meta(distributed): multi-node and multi-GPU execution — backend API, IGG halo, tests, slab adjoint

Đang mở
  • Có gì không ổn?
#79 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
25/100
Loại issue
Tính năng
Độ rõ ràng
Cần làm rõ
Mức độ hoạt động
Sôi nổi
Công nghệ
julia
Lĩnh vực
backend-api-design, distributed-systems, performance, testing

Hướng nghiên cứu

Start with sub-issue #47 and the existing _dist_scatter!/_dist_reduce! seam, then read #46, #30, and #77 to understand the backend, multi-node, test, and slab-adjoint work. Check test/multigpu and test/mdla_gpu.jl; done means the listed sub-issues are addressed while preserving transpose invariants, one halo exchange per application, prepare-time coefficient exchange, and no new core dependency.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

enhancement

Umbrella for everything that makes a CartesianGrid operator run across more than one device or node. MDLA (single-node multi-GPU, CUDA-only) is done behind the _dist_scatter! / _dist_reduce! seam; what remains is the API to choose a backend, the multi-node backend itself, and the tests that prove ≥3 partitions. The perf gap on the slab transpose (#77) is closed.

Status (2026-09-30): #77 done via PR #91; #30's fold-in no longer waits on #47/#46; one open DESIGN question on ghost layout (from #81).

Sub-issues, in order

  1. #47 — explicit backend selection (AbstractDistributedBackend, MDLABackend, IGGBackend), mirroring AbstractJVPBackend. First: it is the entry point #46 must implement against, and today a second extension defining prepare_distributed(::AbstractOperator, ::Integer) would be ambiguous.
  2. #46 — ImplicitGlobalGrid halo backend for multi-node. Carries the κ-ghost note from the PR #66 review (the IGG backend owes one setup-time exchange of the coefficient's Interface ghosts) and, from #52, the comm/compute overlap idea: launch the interior stencil, run _exchange_storage! concurrently, then finish the boundary strips. Overlap is not to be built before a blocking exchange is measured.
  3. #30 — instantiate test/multigpu (≥3-GPU distributed tests). Upstream blockers cleared; also lifts the min(NGPUS_MDLA, 2) caps in test/mdla_gpu.jl. Not gated on #47/#46: the fold-in exercises the existing MDLA backend only, so including test/multigpu/mdla_3partition.jl behind the existing NGPUS_MP >= 3 self-skip and lifting the caps can land in parallel with 1–2.
  4. ~~#77 — slab Diffusion adjoint runs the generic masked gather at ~5.7× the forward stencil. Becomes a per-iteration cost the moment distributed reverse-mode AD lands.~~ **Done** — PR #91 (interior stencil + ghost-plane gather for the slab Diffusion adjoint).

Open design question

  • Ghost layout: inline padding vs. ghosts beside the interior (from #81's addendum, Q3). Should Field keep ghosts inline in a padded array, or store the interior contiguously with ghosts held in separate face buffers? Face buffers are what MPI wants regardless — inline ghosts have to be packed into them — and would let a solver vector alias the interior and fill faces in O(surface) instead of two O(volume) staging passes. The cost lands on operator bodies: inline ghosts are what let one broadcast over shifted slices be correct at every cell (the "leaf bodies stay array-level" rule), whereas ghosts-beside splits each sweep into an interior core plus up to 3ᴺ−1 boundary regions, per leaf, per BC kind, and again for every declared adjoint and AD path. The one existing data point is laplacian_7pt_noflux (DESIGN.md §4), a flat halo-free primitive kept as a separate body, pinned to homogeneous no-flux, and locked to the halo-based leaf by a parity test. Outcome: if inline is right, an alternatives-considered note in DESIGN.md so it stops resurfacing; if not, its own issue.

Invariants that bind every sub-issue

  • _dist_reduce! must remain the exact transpose of _dist_scatter! across ranks, including the ∂/∂κ scatter-add — this is the part with no equivalent in the IGG/ParallelStencil ecosystem.
  • One halo exchange per operator application; coefficient exchanges belong at prepare time.
  • No new core dependency: MPI/IGG live in ext/.

—
🤖 Claude struck #77 off this list with great satisfaction.
claude-opus-5-5[1m] (high) · claude-code 2.1.285 · unreviewed

Ngôn ngữ chính
Julia
Star
3
Fork
1
Merge trung bình
4 ngày 19 giờ
Pull request đã merge (30 ngày)
5

Chuẩn bị môi trường

Dự án này không cung cấp dev container, Dockerfile hay hướng dẫn đóng góp, nên bạn cần tự thiết lập môi trường: hãy bắt đầu từ README và xem hướng dẫn đóng góp lần đầu của chúng tôi để biết các bước chung.

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của RallypointOne/MatrixFreeOperators.jl

Tất cả issue của RallypointOne/MatrixFreeOperators.jl

Issue tương tự

Thêm issue về Julia

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.