Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

fix: summarize context window preemptively before dispatch

Aperta
  • Qualcosa non va?
#1,399 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
35/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
javascript, nodejs
Ambito
ai, backend

Direzione di ricerca

Start with the trigger derivation in src/agent/deepAgents.js around lines 458-472, then read createSummarizationMiddlewareFromConfig in src/provider/summarizationMiddleware.js and estimateCost in src/provider/tokenBudgetMiddleware.js (lines 69-80) to see how input and output budgets are counted. Done means a test in tests/unit/provider/summarizationMiddleware.test.js shows a request whose input plus maxTokens exceeds the window being compacted before dispatch. Expect to work through the deepagents library's trigger logic, which needs more context than the issue gives.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug

Summary

The summarization middleware does not compact the context window preemptively. In long-running sessions it fails to compress before the model's context window is exceeded, causing repeated 400 context-window errors.

Environment

  • OS: Linux 7.0.14-19-pve
  • Node.js: v25.8.1
  • madz version: 1.113.1
  • LLM provider: OpenAI gpt-4o

Reproduction

Steps to reproduce the behavior:

  1. Start a long-running madz session with a large system prompt and many registered tools.
  2. Continue the conversation until the input approaches the model's context window.
  3. Observe the model return 400 context-window errors instead of the middleware compacting preemptively.

Expected Behavior

The summarization middleware should estimate the current context window usage (input + expected output) before dispatching, and compact the conversation when the total would exceed the model's context window — the same way the token-budget middleware estimates cost before sending.

Actual Behavior

The middleware only fires when the input token count (estimated via countTokensApproximately, chars/4) exceeds a fixed trigger (80% of context length). It does not account for expected output tokens, so a long response can push input + output over the window, causing 400 context-window errors that are handled reactively (compact + re-send) rather than preemptively.

Additional Context

The trigger is derived at init as Math.floor(contextLength * 0.8). For gpt-4o (128000), that is 102400. The trigger uses countTokensApproximately (chars/4), which differs from the real tokenizer used by the token-budget middleware (model.getNumTokensFromMessages). The token-budget middleware already estimates cost as input + output budget and reserves against a rolling window — the summarization middleware should apply the same preemptive estimate against the context window.

Audit Findings (for Issue #1399)

  • src/agent/deepAgents.js:458 — The summarization trigger is derived at init as Math.floor(contextLength * 0.8). For gpt-4o (128000) that is 102400. This is a fixed fraction of the context window with no output-token allowance.
  • src/agent/deepAgents.js:463 — The derived trigger is written to config.summarization.trigger as { type: "tokens", value: triggerTokens }, then passed to createSummarizationMiddlewareFromConfig.
  • src/agent/deepAgents.js:472 — createSummarizationMiddlewareFromConfig builds the deepagents SummarizationMiddleware, which fires only when the input token count exceeds the trigger. It does not account for expected output tokens.
  • src/provider/tokenBudgetMiddleware.js:69-80 — The token-budget middleware already estimates cost preemptively: inputTokens (via model.getNumTokensFromMessages) + toolTokens + outputBudget (from options.maxTokens). This is the pattern the summarization middleware should mirror.
  • src/provider/summarizationMiddleware.js:44-60 — createSummarizationMiddlewareFromConfig passes only trigger and keep to createSummarizationMiddleware. The deepagents library computes the trigger from countTokensApproximately (chars/4), which diverges from the real tokenizer used by the token-budget middleware.
  • node_modules/deepagents/dist/langsmith--RbtjNdQ.js:3602 — shouldSummarize() compares adjustedTokens against the trigger. adjustedTokens comes from countTotalTokens → countTokensApproximately, which is a character-based approximation, not the model tokenizer.

Fix Steps

  1. Add a preemptive context-window check — In src/provider/summarizationMiddleware.js, before dispatching, estimate the total cost (input + expected output) using the model tokenizer, mirroring estimateCost() in src/provider/tokenBudgetMiddleware.js:69.
  2. Account for expected output — Add the configured maxTokens (output budget) to the estimate so a long response is accounted for, not just the input.
  3. Compact when total exceeds the window — When input + output > contextWindow, trigger summarization (or the existing compaction path) before the request is sent, rather than waiting for a 400.
  4. Use the real tokenizer — Prefer model.getNumTokensFromMessages over countTokensApproximately for the trigger estimate, so the threshold matches the model's actual tokenization.
  5. Add a test — Add tests/unit/provider/summarizationMiddleware.test.js coverage for the preemptive path: a request whose input + output would exceed the window compacts before dispatch.
  6. Verify — Run npm run test, npm run lint, and npm run coverage to confirm no regressions.
Lingua principale
JavaScript
Stelle
2
Fork
0
Merge medio
1h 6m
PR unite (30g)
234

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di avoidwork/madz

Tutte le issue di avoidwork/madz

Issue simili

Altre issue su JavaScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.