Compaction level 2 summarizes every old turn at once, leaving 10–24% of the budget at 96K

Aperta
#394 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
rust

Direzione di ricerca

Start with level2_summarize_old_turns and compact_messages in src/context.rs, then read docs/evals/compaction-defaults.md and run the compaction_sweep example to establish the baseline. Compare the 96K and 26K rows against the acceptance criteria, and add coverage for partial summaries and byte-for-byte stability before deciding whether the measured tradeoffs meet the target.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

At the default budget (96K effective), a compaction leaves far less history than its own target asks for. Level 2 summarizes every turn older than keep_recent in one pass, whatever the target, so the context drops to 10–24% of the budget while the target sits at 30–70%. The model loses the detail of all older turns at once, with most of the budget unused.

Follow-up to #150 (closed by #393, which fixed the small-budget cliff in level 3). Noted in docs/evals/compaction-defaults.md: "level 2's all-or-nothing summarization is the remaining cliff at 96K."

Numbers

cargo run --release --example compaction_sweep (drives the real compact_messages; floor 0.30, compact_headroom_turns: Some(30), keep_recent: 10, 5 seeds × 240 requests), at 96K:

profile growth/turn effective target left after a compaction (mean / min) compactions /100
chat ~0.7K 0.70 (cap) 24.1% / 22.8% 0.4
records (#150's shape) ~1.6K ~0.50 10.5% / 9.3% 1.7
coding ~3.4K 0.30 (floor) 19.7% / 9.9% 3.8
mixed varies — 23.9% / 7.4% 2.3

The effective target is ContextConfig::effective_target_ratio: (budget − 30 × growth) / budget, capped at compact_target_ratio (0.7) and floored at MIN_HEADROOM_RATIO (0.30). For records it asks to keep ~48K and level 2 leaves ~10K. Level 3 never runs: level 2 already lands under the target.

Why

level2_summarize_old_turns(messages, keep_recent) (src/context.rs) replaces every assistant message before the keep_recent boundary with a [Summary] … one-liner and drops its tool results. The target isn't an input. compact_messages checks the target only afterwards (if total_tokens(&compacted) <= target).

Proposed

Summarize the oldest turns only until the history fits the target, the way level 3 drops the smallest span that reaches it. Turns between that point and the keep_recent boundary stay in full detail.

Things to settle with the sweep before changing it:

  • Retention vs compaction count. Keeping more after a compaction means reaching the budget sooner. records @ 96K should move from ~10% toward ~50% kept; measure how much cmp/100 and the ideal hit rate move.
  • Prefix-cache stability. Today a second pass re-summarizes from the same boundary. A partial summary moves the boundary each pass, so check the bytes before it stay put. Summaries are deterministic and inherit the turn's timestamp, so this should hold, but it needs a test like a_second_compaction_keeps_the_front_byte_for_byte.
  • Small budgets. At 26K, level 3 (with #393) already handles the protected set. Confirm the 26K rows don't regress.

Acceptance

  • The sweep's 96K rows: post-compaction size close to the effective target, the compaction count not materially higher, the hit rate not lower.
  • The 26K rows unchanged or better on head- / ask- / mkr.
  • compaction_is_well_formed_within_budget_and_keeps_what_fits still passes, plus a level-2 test that a partial summary keeps the turns after its cut in full.
Lingua principale
Rust
Stelle
327
Fork
46
Merge medio
2h 36m
PR unite (30g)
107

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di yologdev/yoagent

Tutte le issue di yologdev/yoagent

Issue simili

Altre issue su Rust

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.