Cover the surface_distance backends that had no tests, and fix dask surface_direction (#3725, #3713) - #3729
Conversation
…3725, #3713) The test file covered three public functions across four backends with 37 tests, but never ran anything on dask+cupy and only ever called surface_direction() on eager numpy. That second hole was hiding a real bug: _run_tile() records global source indices while _finalize_direction() built its pixel grid from block-local ones, so every tile except the top-left measured its bearings from the wrong origin. On a flat 6x6 raster chunked (3, 3) the iterative path returned the top-left tile's bearing field tiled across the whole output. _finalize_direction() and _extract_output() now take row/col offsets, and _assemble_sd() passes the tile's own offsets so both sides of the subtraction live in the same coordinate space. The eager numpy and bounded map_overlap paths keep the default of 0 because they seed local indices. New tests cover the holes the audit found: dask+cupy on all three public functions, surface_direction parity across every backend on both the bounded and iterative branches, Inf and all-NaN elevation, 1x1 and Nx1 rasters, connectivity=4 off the eager path, target_values on cupy, Dataset input through @supports_dataset, the geodesic NotImplementedError per backend, the dims rejection, and attrs/coords/dim preservation. Tests 37 -> 106, all passing with CUDA present. Branch coverage of xrspatial.surface_distance 80% -> 87% under NUMBA_DISABLE_JIT=1.
|
Overlap notice: PR #3729 (test-coverage sweep) and PR #3731 (accuracy sweep) both fix the dask They are not equivalent, and the difference matters:
The trap in merging #3729 alone: its direction tests assert that dask matches numpy. If numpy is itself mirrored, that parity holds while both backends are wrong, so the suite would go green over a live defect. #3731's Suggested resolution: take #3731's source changes as the base (superset of the direction fixes), keep #3729's test additions, and reconcile the overlapping hunks in Note for sequencing: PR #3724 adds a docstring caveat describing this bug as unfixed, so it should land after whichever fix merges, with that paragraph dropped. |
Closes #3725. Closes #3713.
What this does
test_surface_distance.pyhad 37 tests spread over three public functionsand four backends, but the matrix had two large holes: nothing ever ran on
dask+cupy, and
surface_direction()was only ever called on eager numpy.Filling the second hole turned up the bug in #3713, so this PR carries the
one-function fix for it alongside the tests. Everything else the audit
found was untested but already correct, and is now pinned.
The source fix
_run_tile()records global source indices:_finalize_direction()then subtracted block-local pixel indices,built from the chunk's own
np.arange(H)/np.arange(W). Every tileexcept the top-left one measured its bearings from the wrong origin, and
the error was exactly the tile's
(row_offset, col_offset).On a flat 6x6 raster with one source at
(0, 0), chunked(3, 3):The top-left tile's bearing field tiled across the whole raster. Pixel
(3, 0)reported 0, meaning "you are the source", when the source isthree cells due north.
_finalize_direction()and_extract_output()now takerow_offset/col_offset, and_assemble_sd()passes each tile's own offsets so bothsides of the subtraction sit in the same coordinate space. The defaults
are 0, which is what the eager numpy path and the bounded
map_overlappath want: both seed local indices through
_seed_sources(), so globaland local already coincide there.
This is the default path for dask users, since
max_distancedefaults tonp.inf. dask+cupy is affected too, because_surface_distance_dask_cupy()routes the unbounded case throughdask+numpy.
The tests
Backend coverage:
test_bounded_matches_numpyandtest_iterative_matches_numpy_all_modescross-check all three public functions against numpy on dask+numpy,
cupy, and dask+cupy, over both the
map_overlapand iterative branches._surface_distance_dask_cupy()had no coverage at all before this.test_iterative_direction_uses_global_pixel_indicesis the targetedregression test for the bug above, asserting the actual bearings rather
than just parity.
test_dask_cupy_returns_dask_cupy_arrayandtest_cupy_direction_returns_cupy_arraypin the output array types.test_target_values_all_backendsandtest_no_sources_all_nan_all_backendsreach the cupytarget_valuesseeding branch and the cupy no-source early return.
Edge cases:
test_inf_elevation_is_a_barrier,test_all_nan_elevation,test_single_pixel_raster,test_column_strip, all four backends. Theexisting degenerate-shape tests only covered 1xN rows.
Parameters and metadata:
test_connectivity_4_all_backendstakesconnectivity=4off the eagerpath and into the dask branches in
_compute_seeds_sd()and_can_skip_sd().test_geodesic_unsupported_backends_raiseandtest_invalid_dimscoverthe error paths.
test_dataset_inputandtest_dataset_allocation_keeps_per_variable_valuesexercise@supports_dataset, which all three functions carry and none was evercalled through.
test_attrs_and_coords_preservedandtest_custom_dim_names_preservedassert attrs, coords, and dims survive.
_compute()readsresoff theinput attrs to build its edge costs, so chained calls depend on this.
Verification
CUDA is available on this host, so the cupy and dask+cupy tests ran rather
than skipped.
Fail-before, pass-after on the three direction tests, with the source
change stashed:
Branch coverage of
xrspatial.surface_distance, same command before andafter:
That flag makes the GPU tests fail, so both figures understate what a real
run covers. It is still the right yardstick for the delta. What remains
uncovered is the numba/cupy kernel bodies that the flag hides, plus the
memory-guard fallbacks noted as LOW in the sweep state.
Tests 37 -> 106.