HermitCrab analysis: combined stack cut work 14-64% on four real grammars, but R2 is closed; re-measure without it (unmeasured)

Aperta
  • Qualcosa non va?
#611 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Documentazione
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
csharp
Ambito
performance

Direzione di ricerca

Start with docs/hermitcrab-2026-10-27-analysis-round.md at line 364 and review PR #578 for the stress grammars and the related work in #562, #567, and #566. Rerun the proposed stack on the current branches in work steps, excluding closed R2, and record the remeasured results and merge order here.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Claude-drafted.

The combined stack of #546, R1, R3, R2 and R5 cut work by 14.08% to 64.31% on four real grammars, but R2 is now closed, so the stack must be re-measured without it. Against master, less work (analysis events plus synthesis steps): Mbugwe 33.26%, Sena 64.31%, Aweti 27.58%, Amharic 14.08%, Indonesian equal, conformance 2.01%. The synthetic stress grammar bantu-tam: 89.64%. Parses equal master on every completed word. These numbers exist only in the retired PR #490 ledger. Measured before R2's witness-reuse fix.

R1 is the required-slot look-ahead (run on #546's shared walk via #577), R3 its ordered and anchored form (#576), R2 the root-substring prune (#572), R5 the edge-segment prefilter (#574).

Status: R2 (PR #572) closed 2026-10-29. Counting every unit of work as equal it added 7.7% to Sena's total work (closing comment on #572).

What we know:

  • R2 adds 4.74 points of Sena work on top of #546 with R3 (59.57% to 64.31% less). On Mbugwe R2 is off, so the stack equals #546 with R3.
  • R2 standalone on conformance was 15.3% more work (events, each probe counted as one event; ledger row 40).

Not known: the stack's numbers without R2.

Proposed change (C#): rerun the stack on the current branches in work steps, record the numbers in this issue, and list the merge order. Use PR #578 for the stress grammars.

Related: PRs #546, #577 (draft), #574, #578 (open); #576 (merged); #572 (closed); #562, #567, #566 (open). PanGloss: sillsdev/PanGloss#112 (the pointer issue for this one, open); sillsdev/PanGloss#66 (open) ports #546 and #566; sillsdev/PanGloss#23 (open) ports #572.

Next: rerun the stack without R2 in work steps and post the table here.

Evidence -- the full stack table

One census binary, one harness, one word set, every arm against master; census mode, 150 s per word; words aborted or erroring in any arm are dropped from all arms. "Less" is against master, per grammar.

grammar work unification matcher
Mbugwe (40 words) 33.26% 20.96% 38.31%
Sena (51) 64.31% 15.43% 22.45%
Aweti (99) 27.58% 45.64% 55.71%
Amharic (24) 14.08% 10.42% 14.96%
Indonesian (70) equal 5.49% 6.51%
conformance (1,452 runs) 2.01% 3.18% 8.69%
stress bantu-tam (60) 89.64% 64.03% 81.38%
stress early-lexicon (60) 67.72% 83.86% 87.06%

Parity: 0 differences in per-word parse signatures; 0 shadow violations on the full stack. The stack needs one test-only commit (11 toy-grammar tests set RootSubstringPrune off).

Additivity (work, points of master): Mbugwe 33.26 against a sum of 34.23 (near-additive); Sena 64.31 against 103.30 (overlap: #546, R3 and R2 remove the same dead branches); Aweti 27.58 against 13.12 (super-additive).

R2's cost -- why it is closed
  • Standalone (with the witness-reuse fix): Sena 9.05% more unification, conformance 1.00% more, bantu-tam 37.48% more. Events: Sena 17.29% fewer, conformance 0.22% fewer.
  • Before the fix, net of probes: Sena 15.3% less work, conformance 15.3% more (7,237 probes buy 92 events).
  • Stacked, the fix removes about 90% of R2's unification overhead, but R2 still costs 1.03 to 3.71 points of unification work over no R2 (conformance, Sena, bantu-tam).
  • The ledger's landing decision: #572 lands only on top of #577 and #574. The alternative, dropping #572, gives up Sena's 4.74 points.
Sources -- pinned at 2eba2f50 (closed PR #490)
Lingua principale
C#
Stelle
29
Fork
16
Merge medio
5g 21h
PR unite (30g)
9

Preparare l'ambiente

  • Nessun Dockerfile né file Docker Compose
  • Ha un modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di sillsdev/machine

Tutte le issue di sillsdev/machine

Issue simili

Altre issue su C#

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.