Object id: a per-array full-length kind (49 unique bits, independent hashed home, no home key)

Aperta
  • Qualcosa non va?
#4,040 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
8/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
cpp, hpc
Ambito
distributed-systems

Direzione di ricerca

Read doc/objid64-design.md on the objid-redesign branch (sections 1 and 7), then #3994 and #4021 for the current hashed-kind layout. The change centres on CkLocMgr::homePe(id) and the CkArrayOptions flag. It is sequenced after PR 3, so the first step is to confirm that PR has landed and to agree the design with maintainers. Done means an agreed design plus a test that migrates elements whose two homes sit on different processes.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

enhancement

Follow-up to #3994 (64-bit object id redesign, doc/objid64-design.md on objid-redesign). Proposed by Kale, 2026-10-13; this issue records the idea and its cost so it is not lost. Not for the 9.0 alpha.

The limit this removes

In the hashed kind (arrays without an index compressor), the 49 payload bits are split into an H-bit home key (the hash of the index, so the home is computable from the id alone) and U = 49 - H unique bits handed out in tranches. H is set per run from the process count, clamped to 24, and persisted in checkpoints. At large process counts U gets small: on Frontier with two processes per node (18,816 processes) H = 18, so an array holds at most 2^31 ids for the life of the run, and the first tranche per process is 32,768. This is what drives the tranche cap, the first-grant floor after restart (PR 1b, #4021), and the tranche reclamation follow-up.

Proposal: a per-array "full-length" kind

A third kind, chosen per array at creation (a CkArrayOptions flag), with:

  • payload = 49 unique bits from the tranche allocator, no home key;
  • id home = hash(id) spread over processes and ranks (the same spread as #4039);
  • index home = hash(index), as today, but it holds only the index -> id binding, which never changes after insertion;
  • location is held by the id home only, so migration updates one home as now.

No bit of the id is taken for the kind. The kind is a property of the collection: CkLocMgr::homePe(id) already dispatches on whether the array has a compressor, and this is a third branch of the same dispatch. The three TYPE_TAG bits stay reserved (design doc section 1).

What it costs

  • Insertion (and restore after restart) notifies two homes instead of one: one extra message per element, not an extra hop on any send path.
  • The first index-addressed send from a PE that has never seen that index: today the request goes requester -> home -> requester. With two homes the index home forwards the request (index, id, requester) to the id home, which answers the requester. Three legs instead of two, once per PE per element; after that the cache holds both bindings and sends are one hop as now.
  • Deletion and reclaim (reclaimRemote) clear both homes.
  • recoverIndex(id) for this kind needs the id home to hold id -> index, so the insert notification to the id home carries the index.

What it buys

  • 2^49 ids per array per run at any process count. Frontier at two processes per node: 5.6e14 instead of 2.1e9, and a first tranche per process of 2^30 or more instead of 32,768.
  • The pool consumed by each restart becomes negligible, so tranche reclamation becomes optional.
  • No process count is baked into the id, so the layout persisted in the checkpoint does not constrain restarts of this kind.

When

With or after PR 3 (process-level directory, design section 7), not before. PR 3 rewrites the home's tables into the shared per-process table of section 7.2; separating an index table from an id table is where this change lands naturally. Doing it on today's per-PE tables means rewriting the same code twice. Estimated 3-4 days on top of PR 3: mint path, two-home notification, the forwarded index request, reclaim and restore, and a test that migrates elements whose two homes are on different processes. The default stays the hashed kind; applications opt in per array.

Lingua principale
C++
Stelle
241
Fork
62
Merge medio
1g 1h
PR unite (30g)
121

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di charmplusplus/charm

Tutte le issue di charmplusplus/charm

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.