Skip to content

Cover the surface_distance backends that had no tests, and fix dask surface_direction (#3725, #3713) - #3729

Open
brendancol wants to merge 2 commits into
mainfrom
deep-sweep-test-coverage-surface_distance-2026-08-16
Open

Cover the surface_distance backends that had no tests, and fix dask surface_direction (#3725, #3713)#3729
brendancol wants to merge 2 commits into
mainfrom
deep-sweep-test-coverage-surface_distance-2026-08-16

Conversation

@brendancol

Copy link
Copy Markdown
Contributor

Closes #3725. Closes #3713.

What this does

test_surface_distance.py had 37 tests spread over three public functions
and four backends, but the matrix had two large holes: nothing ever ran on
dask+cupy, and surface_direction() was only ever called on eager numpy.

Filling the second hole turned up the bug in #3713, so this PR carries the
one-function fix for it alongside the tests. Everything else the audit
found was untested but already correct, and is now pinned.

The source fix

_run_tile() records global source indices:

src_row[r, c] = r + row_offset
src_col[r, c] = c + col_offset

_finalize_direction() then subtracted block-local pixel indices,
built from the chunk's own np.arange(H) / np.arange(W). Every tile
except the top-left one measured its bearings from the wrong origin, and
the error was exactly the tile's (row_offset, col_offset).

On a flat 6x6 raster with one source at (0, 0), chunked (3, 3):

numpy (correct)                      dask+numpy, iterative (before)
[[  0.  270.  270.  270.  270. 270.] [[  0.  270.  270.    0.  270. 270.]
 [360.  315.  296.6 288.4 284. 281.3] [360.  315.  296.6 360.  315. 296.6]
 [360.  333.4 315.  303.7 296.6 291.8][360.  333.4 315.  360.  333.4 315.]
 [360.  341.6 326.3 315.  306.9 301.] [  0.  270.  270.    0.  270. 270.]
 [360.  346.  333.4 323.1 315. 308.7] [360.  315.  296.6 360.  315. 296.6]
 [360.  348.7 338.2 329.  321.3 315.]][360.  333.4 315.  360.  333.4 315.]]

The top-left tile's bearing field tiled across the whole raster. Pixel
(3, 0) reported 0, meaning "you are the source", when the source is
three cells due north.

_finalize_direction() and _extract_output() now take row_offset /
col_offset, and _assemble_sd() passes each tile's own offsets so both
sides of the subtraction sit in the same coordinate space. The defaults
are 0, which is what the eager numpy path and the bounded map_overlap
path want: both seed local indices through _seed_sources(), so global
and local already coincide there.

This is the default path for dask users, since max_distance defaults to
np.inf. dask+cupy is affected too, because
_surface_distance_dask_cupy() routes the unbounded case through
dask+numpy.

The tests

Backend coverage:

  • test_bounded_matches_numpy and test_iterative_matches_numpy_all_modes
    cross-check all three public functions against numpy on dask+numpy,
    cupy, and dask+cupy, over both the map_overlap and iterative branches.
    _surface_distance_dask_cupy() had no coverage at all before this.
  • test_iterative_direction_uses_global_pixel_indices is the targeted
    regression test for the bug above, asserting the actual bearings rather
    than just parity.
  • test_dask_cupy_returns_dask_cupy_array and
    test_cupy_direction_returns_cupy_array pin the output array types.
  • test_target_values_all_backends and
    test_no_sources_all_nan_all_backends reach the cupy target_values
    seeding branch and the cupy no-source early return.

Edge cases:

  • test_inf_elevation_is_a_barrier, test_all_nan_elevation,
    test_single_pixel_raster, test_column_strip, all four backends. The
    existing degenerate-shape tests only covered 1xN rows.

Parameters and metadata:

  • test_connectivity_4_all_backends takes connectivity=4 off the eager
    path and into the dask branches in _compute_seeds_sd() and
    _can_skip_sd().
  • test_geodesic_unsupported_backends_raise and test_invalid_dims cover
    the error paths.
  • test_dataset_input and
    test_dataset_allocation_keeps_per_variable_values exercise
    @supports_dataset, which all three functions carry and none was ever
    called through.
  • test_attrs_and_coords_preserved and test_custom_dim_names_preserved
    assert attrs, coords, and dims survive. _compute() reads res off the
    input attrs to build its edge costs, so chained calls depend on this.

Verification

CUDA is available on this host, so the cupy and dask+cupy tests ran rather
than skipped.

$ python -m pytest xrspatial/tests/test_surface_distance.py -q
106 passed in 5.20s

Fail-before, pass-after on the three direction tests, with the source
change stashed:

FAILED test_iterative_matches_numpy_all_modes[dask+numpy-direction]
FAILED test_iterative_matches_numpy_all_modes[dask+cupy-direction]
FAILED test_iterative_direction_uses_global_pixel_indices
3 failed, 4 passed

Branch coverage of xrspatial.surface_distance, same command before and
after:

NUMBA_DISABLE_JIT=1 python -m pytest xrspatial/tests/test_surface_distance.py \
    --cov=xrspatial.surface_distance --cov-branch -q
statements branch cov
before 570/695 80%
after 604/695 87%

That flag makes the GPU tests fail, so both figures understate what a real
run covers. It is still the right yardstick for the delta. What remains
uncovered is the numba/cupy kernel bodies that the flag hides, plus the
memory-guard fallbacks noted as LOW in the sweep state.

Tests 37 -> 106.

…3725, #3713)

The test file covered three public functions across four backends with 37
tests, but never ran anything on dask+cupy and only ever called
surface_direction() on eager numpy. That second hole was hiding a real
bug: _run_tile() records global source indices while _finalize_direction()
built its pixel grid from block-local ones, so every tile except the
top-left measured its bearings from the wrong origin. On a flat 6x6 raster
chunked (3, 3) the iterative path returned the top-left tile's bearing
field tiled across the whole output.

_finalize_direction() and _extract_output() now take row/col offsets, and
_assemble_sd() passes the tile's own offsets so both sides of the
subtraction live in the same coordinate space. The eager numpy and bounded
map_overlap paths keep the default of 0 because they seed local indices.

New tests cover the holes the audit found: dask+cupy on all three public
functions, surface_direction parity across every backend on both the
bounded and iterative branches, Inf and all-NaN elevation, 1x1 and Nx1
rasters, connectivity=4 off the eager path, target_values on cupy,
Dataset input through @supports_dataset, the geodesic NotImplementedError
per backend, the dims rejection, and attrs/coords/dim preservation.

Tests 37 -> 106, all passing with CUDA present. Branch coverage of
xrspatial.surface_distance 80% -> 87% under NUMBA_DISABLE_JIT=1.
@brendancol

Copy link
Copy Markdown
Contributor Author

Overlap notice: PR #3729 (test-coverage sweep) and PR #3731 (accuracy sweep) both fix the dask surface_direction bug — global source indices measured against chunk-local index grids — found independently in the same /deep-sweep run, along with #3713 filed by the documentation sweep. Both rewrite _finalize_direction and _tile_fn, so they will conflict.

They are not equivalent, and the difference matters:

The trap in merging #3729 alone: its direction tests assert that dask matches numpy. If numpy is itself mirrored, that parity holds while both backends are wrong, so the suite would go green over a live defect. #3731's test_direction_cardinal_points pins bearings against known compass directions instead, which is what actually catches it.

Suggested resolution: take #3731's source changes as the base (superset of the direction fixes), keep #3729's test additions, and reconcile the overlapping hunks in _finalize_direction / _tile_fn by hand. Then run both PRs' full test files against the merged tree — neither PR's own green run proves the combination.

Note for sequencing: PR #3724 adds a docstring caveat describing this bug as unfixed, so it should land after whichever fix merges, with that paragraph dropped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant