surface_distance(): the Dijkstra heap is sized H*W, so dense source rasters write out of bounds and abort the process
还没有人认领这个 Issue。
评估
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 新手友好度
- 78/100
- Issue 类型
- 缺陷
- 描述清晰度
- 描述清楚
- 活跃度
- 冷清
- 技术栈
- numpy, python
- 领域
- backend, performance
调研方向
Start in xrspatial/surface_distance.py at _dijkstra and _dijkstra_geodesic, then compare their heap allocation with the corrected sizing and memory comment in xrspatial/cost_distance.py. Reproduce the dense checkerboard case with NUMBA_BOUNDSCHECK=1 and verify both kernels and the _check_memory accounting no longer overflow.
由索引模型根据 Issue 内容生成。
描述
Summary
_dijkstra and _dijkstra_geodesic in xrspatial/surface_distance.py size the
binary-heap backing arrays as
max_heap = height * width # lines 214 and 277
h_keys = np.empty(max_heap, dtype=np.float64)
h_rows = np.empty(max_heap, dtype=np.int64)
h_cols = np.empty(max_heap, dtype=np.int64)
Both kernels are lazy-deletion Dijkstra: a pixel is pushed onto the heap every
time its tentative distance improves, so it can be enqueued once per settled
neighbour rather than once overall. The push count is bounded by
height * width * (n_neighbors + 1), not height * width.
_heap_push (in cost_distance.py) writes keys[pos] at pos = size with no
bounds check. Once the heap outgrows the allocation, the kernel writes past the
end of three numba arrays. numba compiles without bounds checking by default, so
nothing catches it.
cost_distance.py carried the same defect and was already corrected; see the
comment at xrspatial/cost_distance.py:155-161 and the sizing at line 161.
surface_distance.py was never updated.
Reproduction
A checkerboard source raster over random elevation, 8-connectivity, on the
plain numpy backend:
import numpy as np
import xarray as xr
from xrspatial.surface_distance import surface_distance
rng = np.random.default_rng(0)
H = W = 80
elev = rng.random((H, W)) * 5000.0
rr, cc = np.meshgrid(np.arange(H), np.arange(W), indexing='ij')
src = ((rr + cc) % 2 == 0).astype(np.float64)
coords = {'y': np.arange(H, dtype=float), 'x': np.arange(W, dtype=float)}
raster = xr.DataArray(src, dims=('y', 'x'), coords=coords)
elevation = xr.DataArray(elev, dims=('y', 'x'), coords=coords)
surface_distance(raster, elevation, connectivity=8)
Default build (no bounds checking) — the process dies:
$ python repro.py
double free or corruption (!prev)
Aborted (core dumped)
Same script under NUMBA_BOUNDSCHECK=1, which names the defect:
$ NUMBA_BOUNDSCHECK=1 python repro.py
File "xrspatial/surface_distance.py", line 474, in _surface_distance_numpy
_dijkstra(elev_data, H, W, max_distance,
dy, dx, dd, dist, alloc, src_row, src_col)
IndexError: index is out of bounds
Affected backends
Reproduced under NUMBA_BOUNDSCHECK=1 with the same 80x80 input:
| Path | Result |
|---|---|
numpy, planar (_dijkstra) |
IndexError: index is out of bounds |
numpy, geodesic (_dijkstra_geodesic) |
IndexError: index is out of bounds |
dask+numpy, iterative tile path via _run_tile |
IndexError: index is out of bounds |
The dask+numpy bounded map_overlap branch runs _surface_distance_numpy per
chunk, so it is exposed on any chunk whose seeded pixel density is high enough.
The cupy and dask+cupy backends use iterative parallel relaxation with no heap
and are not affected. compute-sanitizer --tool memcheck over nine adversarial
shapes (2x2, 2x97, 97x2, 3x101, 101x3, 61x59, 127x127, 33x512, 512x33) across all
three public functions reports ERROR SUMMARY: 0 errors, and the
_sd_relax_kernel bounds guard at line 496 is correct.
How often it bites
Peak concurrent heap size against the allocated cap, measured with an
instrumented copy of _dijkstra (oversized heap so it can run to completion),
8-connectivity, random elevation:
| Raster | Seeding | Peak heap | Allocated cap | Overflow |
|---|---|---|---|---|
| 60x60 | single source | 452 | 3600 | no (0.13x) |
| 100x100 | 20% random sources | 9810 | 10000 | no (0.98x) |
| 40x40 | checkerboard | 1614 | 1600 | yes (1.01x) |
| 80x80 | checkerboard | 6646 | 6400 | yes (1.04x) |
| 200x200 | checkerboard | 41711 | 40000 | yes (1.04x) |
Sparse single-source rasters stay well clear, which is why the existing tests
pass. Dense source masks are the trigger, and those are ordinary usage:
a rasterized road or stream network, a landcover class mask, or any
target_values= selection that matches a large share of the raster. A 20%
random source density already reaches 0.98x of the cap.
Issue #3709 notes that the module's own benchmark raster makes 99.95% of pixels
sources, so the benchmark suite is already running in the regime that overflows.
Proposed fix
Size the heap to the actual bound in both kernels, matching what
cost_distance.py already does:
max_heap = height * width * (n_neighbors + 1)
The _BYTES_PER_PIXEL = 80 constant used by _check_memory models the heap at
24 bytes/pixel, so it needs to follow the new allocation or the guard added in
#1305 will under-count the eager numpy footprint by roughly 3.4x.
Related, not filed
xrspatial/pathfinding.py:304 uses the same max_heap = height * width sizing
with the same _heap_push. That kernel is single-source A*, and I could not get
it to overflow, so I am recording it here as a site to re-check rather than
claiming a defect.
Found by /deep-sweep security audit of surface_distance on 2026-08-16.
- 主要语言
- Python
- 星标
- 972
- 派生
- 92
- 平均合并
- 2 天 12 小时
- 30 天内合并 PR
- 7
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
xarray-contrib/xarray-spatial 的其他 Issue
-
难度 1/5 1 小时以内 新手友好度 68/100
xarray-contrib/xarray-spatial#3726 ·
-
api area:surface bug severity:medium sweep-api-consistency
难度 2/5 1-3 小时 新手友好度 82/100
xarray-contrib/xarray-spatial#3712 ·
-
难度 1/5 1 小时以内 新手友好度 88/100
xarray-contrib/xarray-spatial#3710 ·
-
bug
难度 1/5 1-3 小时 新手友好度 88/100
xarray-contrib/xarray-spatial#3707 ·
-
area:surface documentation user-guide-example
难度 2/5 1-3 小时 新手友好度 72/100
xarray-contrib/xarray-spatial#3464 ·
查看 xarray-contrib/xarray-spatial 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 88/100
-
难度 2/5 1-3 小时 新手友好度 82/100
-
难度 2/5 1-3 小时 新手友好度 78/100
-
enhancement
难度 2/5 1-3 小时 新手友好度 72/100
-
难度 2/5 1-3 小时 新手友好度 74/100