Broaden #1003 into a shared CPU/GPU elementwise framework (follow-up to #1067)

Aperta
  • Qualcosa non va?
#1,069 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Refactoring
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
cpp
Ambito
backend, performance

Direzione di ricerca

Start by reading #1003, #1067, and the traversal in cuArithmeticDispatch.cuh, cuiArithmeticDispatch.cuh, and cuCpr_dispatch.cu; compare with the CPU traversal from #1056. The issue lists six redesign items, including shared layout metadata, rank limits, and mapper fast paths. Benchmark the listed operand layouts across sizes and ranks; done means the shared framework addresses the listed correctness and reachability gaps and includes comparative benchmarks.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Context

Follow-up to PR #1067, which converts GPU elementwise arithmetic to typed
std::visit dispatch and removes the legacy per-dtype tables. In his review of
#1067, @ianmccul noted that the non-contiguous mapper kernels introduced/carried
forward there should be treated as transitional code, and that the broader
work tracked in #1003 should be broadened to a shared CPU/GPU elementwise
framework. This issue collects the concrete items from that review so they aren't
lost.

The typed-dispatch cleanup and dtype-correctness fixes in #1067 are independently
worthwhile and can land first; the items below are the follow-up.

Architecture (broaden #1003)

Move to a single shared CPU/GPU elementwise framework:

  • common operation functors and dtype/promotion policies (the functor is already
    shared within one GPU arithmetic family, but traversal is not),
  • common host-side stride/layout preprocessing into one compact,
    trivially-copyable layout descriptor,
  • backend-specific CPU/CUDA execution that consumes the same descriptor.

This removes the traversal duplication that currently exists across
cuArithmeticDispatch.cuh, cuiArithmeticDispatch.cuh, cuCpr_dispatch.cu, and
the analogous CPU traversal from #1056.

Concrete items

  1. Rank-dependent shared-memory launch limit (correctness). The non-contiguous
    kernels request 512 * rank * sizeof(uint64_t) = 4096 * rank bytes of dynamic
    shared memory for per-thread coordinate vectors. Rank 13 already exceeds the
    common 48 KiB limit, so high-rank non-contiguous arithmetic fails at launch with
    no rank check or fallback. (Pre-existing: inherited from the legacy kernels.)
    Fix by not putting the per-thread coordinate vector in dynamic shared memory, or
    at minimum add an explicit supported-rank check / rank-derived block size.

  2. Pass layout metadata as a kernel argument, not via managed allocations. Each
    non-contiguous op currently does 5 cudaMallocManaged allocations + 2 copies +
    launch + 5 frees for a few hundred bytes of metadata. Pack extents/strides/
    mappers into one trivially-copyable struct passed by value (e.g.
    __grid_constant__ LayoutMetadata with std::array<uint64_t, MAX_RANK> fields;
    ~1.3 KiB at rank 32, well under the 32 KiB parameter limit on modern devices),
    or copy once with cudaMemcpyAsync / prefetch. (Gemini's "host write to device
    pointer" flag was a false positive — cuCalloc_gpu is cudaMallocManaged — but
    demand-paging tiny managed allocations is still a poor metadata path.)

  3. In-place: iterate LHS physical order; only map the RHS. For in-place ops the
    LHS is also the output, so traversing LHS physical order guarantees coalesced
    LHS reads and output writes; only the RHS may need an index transform. Compose
    the two permutations host-side into a single LHS-physical -> RHS-physical map.
    The current kernel redundantly decodes a logical index, computes Lidx, then
    Ridx, and writes out[Lidx].

  4. Same-layout fast path. When Lt.shape() == Rt.shape() && Lt.invmapper() == Rt.invmapper(), corresponding logical elements share physical offsets, so the op
    is a plain physical-buffer zip (out[i] = op(lhs[i], rhs[i])) with no mapper --
    true even for non-commutative ops. Today the front end takes the full mapper path
    whenever either operand is individually non-contiguous, so two identically
    permuted operands pay the whole mapper cost unnecessarily.

  5. Uniform non-contiguous reachability. Out-of-place GPU Mul/Div/Mod/Cpr
    front ends still reject (or contiguous-ize) non-contiguous tensor/tensor
    operands; only Add/Sub and the in-place paths reach the mapper-aware kernels.
    The shared framework should make the mapper-aware kernel uniformly reachable.

  6. Benchmarks to guide the redesign. Before treating the mapper path as final,
    benchmark (over multiple sizes/ranks): contiguous a += b; identically permuted
    a += b (should match contiguous once the fast path exists); contiguous LHS +
    permuted RHS; differently permuted operands; and contiguous()-copy-then-zip
    including copy cost.

Credit: findings and suggestions by @ianmccul (posted via OpenAI Codex) on #1067.
Related: #1003, #1067.

Lingua principale
C++
Stelle
62
Fork
23
Merge medio
4g 10h
PR unite (30g)
3

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Cytnx-dev/Cytnx

Tutte le issue di Cytnx-dev/Cytnx

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.