ShaderLab retains ~37-40 MB of compile-time data after compilation (LALR construction scaffold + object pools never truly released)

Aperta
#3,028 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
50/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
typescript

Direzione di ricerca

Start in packages/shader-lab/src/lalr/State.ts, where the static closureMap and pool Maps keep the LR item graph alive, and find where the parse table is finished in the Parser. Confirm that actionTable and gotoTable hold only numeric ids before clearing the scaffold there. Also read ShaderLabUtils.clearAllShaderLabObjectPool and ClearableObjectPool, which only reset a counter. Done means the scaffold is released after table construction and the shader-lab tests still pass.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Affected package

@galacean/engine-shaderlab — verified on a 2.0-based build (0.0.0-experimental-2.0-game.16); the relevant structures are identical on current main (packages/shader-lab/src/lalr/State.ts).

Most visible on iOS / JavaScriptCore. On V8 the same retention exists but is masked (see note at the end), which is probably why it has gone unnoticed.

Symptom

After all shaders are compiled, the following compile-time data stays alive for the whole engine lifetime (heap snapshot from a production game, Safari Web Inspector):

What Count Memory
StateItem (LR items, each holding a lookaheadSet: Set) 11,296 Set hash buckets total ~25–30 MB on JSC (JSC pre-allocates ~2 KB per Set instance)
State (LR states, each holding _items/cores Sets + _stateItemPool Map) 487 included above
BaseToken / ASTNode / ShaderPosition / ShaderRange pool residents ~150k objects ~9 MB, plus measurable GC mark-phase cost
Total ~160k objects ~37–40 MB
Root cause
  1. The LALR table-construction scaffold is never torn down. State.closureMap / State.pool (static Maps, packages/shader-lab/src/lalr/State.ts L6–7) pin the entire item-set graph. Runtime parsing only uses Parser.actionTable / gotoTable, whose entries are plain numeric ids ({ action, target: newState.id }, gotoTable.set(gs, newState.id)) — fully self-contained. State.pool is write-only (set in the constructor, never read anywhere). So once the parse table is built, the whole State/StateItem/lookaheadSet graph is garbage, kept alive only by those two static Maps.

  2. ShaderLabUtils.clearAllShaderLabObjectPool() does not actually release anything. It calls ClearableObjectPool.clear(), which only resets the used-element counter; _elements keeps every object ever allocated. Pool capacity equals the compilation-time peak (~150k objects in our game) and never shrinks.

Suggested fix
  1. Tear down the scaffold right after the parse table is built (zero API change):
    State.closureMap.clear();
    State.pool.clear();
    
  2. Provide a public API (e.g. ShaderLab.releaseCompilationCache()) that runs garbageCollection() (already exists on the pool base class) over ShaderLabUtils._shaderLabObjectPoolSet. Only the application knows when compilation has converged (e.g. after shader warm-up), so this is best left as an explicit call.
Verification (production game, patched locally)

Applying exactly the two steps above after shader warm-up, same scene before/after:

  • Set instances: 12,858 → 589
  • JSC Cell Butterfly (Set/Map backing): 38.6 MB → 8.0 MB
  • Live heap: −30.6 MB; Safari Memory timeline javascript category (includes JSC heap capacity): −~100 MB
  • Behavior: shader variants compiled after the release still work — the parse table is self-contained, and token/AST pools simply re-allocate on demand (millisecond-level, during loading).
Side note for the engine in general

JSC pre-allocates ~2 KB of hash buckets per Set/Map instance. Any per-instance tiny-collection design (like one lookaheadSet per LR item — or per-bone/per-entity Sets elsewhere) costs two orders of magnitude more on iOS than on V8 (the same ~12k Sets cost 0.3 MB on Chrome vs ~30 MB on JSC). Small arrays are much cheaper for collections that typically hold 0–3 elements.

Lingua principale
TypeScript
Stelle
5.9k
Fork
413
Merge medio
16h 17m
PR unite (30g)
41

Preparare l'ambiente

  • Nessun Dockerfile né file Docker Compose
  • Ha un modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di galacean/engine

Tutte le issue di galacean/engine

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.