meta(distributed): multi-node and multi-GPU execution — backend API, IGG halo, tests, slab adjoint

未关闭
  • 哪里不对？
#79 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
25/100
Issue 类型
功能
描述清晰度
需要澄清
活跃度
活跃
技术栈
julia
领域
backend-api-design, distributed-systems, performance, testing

调研方向

Start with sub-issue #47 and the existing _dist_scatter!/_dist_reduce! seam, then read #46, #30, and #77 to understand the backend, multi-node, test, and slab-adjoint work. Check test/multigpu and test/mdla_gpu.jl; done means the listed sub-issues are addressed while preserving transpose invariants, one halo exchange per application, prepare-time coefficient exchange, and no new core dependency.

由索引模型根据 Issue 内容生成。

描述

enhancement

Umbrella for everything that makes a CartesianGrid operator run across more than one device or node. MDLA (single-node multi-GPU, CUDA-only) is done behind the _dist_scatter! / _dist_reduce! seam; what remains is the API to choose a backend, the multi-node backend itself, and the tests that prove ≥3 partitions. The perf gap on the slab transpose (#77) is closed.

Status (2026-09-30): #77 done via PR #91; #30's fold-in no longer waits on #47/#46; one open DESIGN question on ghost layout (from #81).

Sub-issues, in order

  1. #47 — explicit backend selection (AbstractDistributedBackend, MDLABackend, IGGBackend), mirroring AbstractJVPBackend. First: it is the entry point #46 must implement against, and today a second extension defining prepare_distributed(::AbstractOperator, ::Integer) would be ambiguous.
  2. #46 — ImplicitGlobalGrid halo backend for multi-node. Carries the κ-ghost note from the PR #66 review (the IGG backend owes one setup-time exchange of the coefficient's Interface ghosts) and, from #52, the comm/compute overlap idea: launch the interior stencil, run _exchange_storage! concurrently, then finish the boundary strips. Overlap is not to be built before a blocking exchange is measured.
  3. #30 — instantiate test/multigpu (≥3-GPU distributed tests). Upstream blockers cleared; also lifts the min(NGPUS_MDLA, 2) caps in test/mdla_gpu.jl. Not gated on #47/#46: the fold-in exercises the existing MDLA backend only, so including test/multigpu/mdla_3partition.jl behind the existing NGPUS_MP >= 3 self-skip and lifting the caps can land in parallel with 1–2.
  4. ~~#77 — slab Diffusion adjoint runs the generic masked gather at ~5.7× the forward stencil. Becomes a per-iteration cost the moment distributed reverse-mode AD lands.~~ **Done** — PR #91 (interior stencil + ghost-plane gather for the slab Diffusion adjoint).

Open design question

  • Ghost layout: inline padding vs. ghosts beside the interior (from #81's addendum, Q3). Should Field keep ghosts inline in a padded array, or store the interior contiguously with ghosts held in separate face buffers? Face buffers are what MPI wants regardless — inline ghosts have to be packed into them — and would let a solver vector alias the interior and fill faces in O(surface) instead of two O(volume) staging passes. The cost lands on operator bodies: inline ghosts are what let one broadcast over shifted slices be correct at every cell (the "leaf bodies stay array-level" rule), whereas ghosts-beside splits each sweep into an interior core plus up to 3ᴺ−1 boundary regions, per leaf, per BC kind, and again for every declared adjoint and AD path. The one existing data point is laplacian_7pt_noflux (DESIGN.md §4), a flat halo-free primitive kept as a separate body, pinned to homogeneous no-flux, and locked to the halo-based leaf by a parity test. Outcome: if inline is right, an alternatives-considered note in DESIGN.md so it stops resurfacing; if not, its own issue.

Invariants that bind every sub-issue

  • _dist_reduce! must remain the exact transpose of _dist_scatter! across ranks, including the ∂/∂κ scatter-add — this is the part with no equivalent in the IGG/ParallelStencil ecosystem.
  • One halo exchange per operator application; coefficient exchanges belong at prepare time.
  • No new core dependency: MPI/IGG live in ext/.

—
🤖 Claude struck #77 off this list with great satisfaction.
claude-opus-5-5[1m] (high) · claude-code 2.1.285 · unreviewed

主要语言
Julia
星标
3
派生
1
平均合并
1 分钟
30 天内合并 PR
1

环境准备

这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

RallypointOne/MatrixFreeOperators.jl 的其他 Issue

查看 RallypointOne/MatrixFreeOperators.jl 的全部 Issue

相似的 Issue

更多 Julia Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。