Auto-compaction never fires for a custom (BYOK) model: context meter undercounts ~1.4× (657,843 shown vs 921,487 charged), so the session dies on 400s and cannot self-recover

Bắt đầu với các dòng 7523–7549 trong session events.jsonl và các định nghĩa được sinh trong rpc.d.ts cho compactionThreshold, promptTokenLimit, bufferTokens và mcpTools. So sánh các bộ đếm context được báo cáo với các lỗi của provider và checkpoint compaction tại runtime. Được xem là hoàn tất khi các session dùng provider tùy chỉnh kích hoạt compaction trước khi provider từ chối một request quá lớn và ngưỡng buffer có thể ngăn cùng lỗi đó.

Summary

With a custom (BYOK) provider whose tokenizer the app does not model, the local context
meter undercounts tokens by ~40%. Background compaction (trigger "threshold") and the
buffer-exhaustion blocking threshold both evaluate that undercounted number, so neither can
fire before the provider rejects the request. The session hard-fails with 400s until the user
runs /compact manually.

Environment

• GitHub Copilot desktop app 1.1.20 (github.exe, %LOCALAPPDATA%\Programs\GitHub Copilot), Windows
• Agent runtime: copilot 1.0.84-5 (%LOCALAPPDATA%\github-copilot-sdk\cli\1.0.84-5\copilot.exe)
• Provider: custom DeepSeek — authKind: api_key, baseUrl: https://api.deepseek.com,
  wireApi: responses; model id deepseek-flash (provider id ab5a407d-9560-461f-9e02-a63c36bd4c08)
• Model config: Max prompt tokens 840,000, Max output tokens 128,000,
  reasoning efforts none / low / high / max
• MCP servers attached: aspire, github, microsoft-learn, nuget, playwright — all "tools": ["*"]
• Session 53c1a09d-6bdc-407d-b133-b817e4dc0184, 2026-09-14T02:20:32Z → 2026-09-14T14:18:32Z
  (640 assistant turns, 998 tool executions, 18 images, 18 user messages, 86 autopilot objective changes)

Steps to reproduce

1. Configure a custom provider whose model accepts 1,048,576 input tokens; set Max prompt
   tokens = 840,000 and Max output tokens = 128,000.
2. Run one long agentic session (hundreds of turns with large tool outputs and several MCP servers).
3. Compare the app's context meter with the token count the provider reports in its usage/error responses.

Expected

session.compaction_start with "trigger":"threshold" when the meter crosses the compaction
threshold (observed elsewhere at ~80% of tokenLimit; here that would be 672,000), well before
the provider limit — and no 400 "maximum context length" errors.

Actual

• No threshold compaction ever fired during the 12-hour session. The only compaction is
  user-initiated: session.compaction_start … "trigger":"manual", timestamp
  2026-09-14T14:18:05.353Z (app log: command_name="compact" at 14:18:04.923Z).
• At that moment the app's meter read:
  • systemTokens 15,768 + conversationTokens 590,768 + toolDefinitionsTokens 51,304
    = currentTokens 657,843, with tokenLimit 840,000 → 78.3%
  • i.e. 14,157 tokens short of the ~80% (672,000) trip point — the trigger never came close.
• DeepSeek counted the same conversation at 921,487 tokens. Two model calls failed:
  • 2026-09-14T08:25:35.172Z — 400 This model's maximum context length is 1048576 tokens. However, you requested 1049487 tokens (921487 in the messages, 128000 in the completion). Please reduce the length of the messages or completion.
  • 2026-09-14T08:26:11.368Z — same, 1050149 tokens (922149 in the messages, 128000 in the completion)
  • (8 records total, counting the errorOccurred / sessionEnd hook payloads and the runtime stack.)
• Undercount factor: 921,487 / 657,843 = 1.400.
• The manual compaction itself worked: messagesRemoved 1,322, tokensRemoved 572,553,
  590,771 → 18,218 — so compaction is functional; only the automatic trigger's input was wrong.

Control: threshold compaction works for hosted models

Ten sessions on hosted models auto-compacted correctly at ~80% of their limit, e.g.:

• {"currentTokens":102931,"tokenLimit":128000,"trigger":"threshold"}
• {"currentTokens":104691,"tokenLimit":128000,"trigger":"threshold"}
• {"currentTokens":105120,"tokenLimit":128000,"trigger":"threshold"}

Range observed: 102,931–105,120 (80.4%–82.1% of 128,000). The trigger logic is fine; the token
input is wrong for the custom model.

Additional metadata mismatch

The session's model metadata carried max_context_window_tokens: 200000 while the endpoint
enforces 1,048,576 (per the provider's own error), alongside max_prompt_tokens: 840000,
max_output_tokens: 128000.

Impact

• Background compaction and the buffer-exhaustion blocking threshold (documented default 95%,
  SessionContextInfo.bufferTokens) both operate on the undercounted number, so for BYOK models
  they cannot fire before the provider rejects the request.
• The session cannot self-recover: retries repeat the same request and the same 400.
• The context meter is misleading near the limit (~78% shown while the real request was already
  over the window once the 128,000 output reserve is counted).

Likely cause / suggested fixes

• SessionContextAttribution documents that totals are "tokenized against the resolved model
  (feeds the per-model token multiplier)". For this custom model that multiplier does not match
  the provider's tokenizer.
• Consider: calibrating the multiplier from provider-reported usage (the runtime already
  receives compactionTokensUsed.inputTokens, cacheReadTokens, etc.), applying a conservative
  safety factor for custom providers, and/or gating compaction on provider-reported input tokens.
• Re-check attribution of tool/MCP schema mass: five MCP servers at "tools": ["*"] were credited
  only 51,304 tokens here, while the meter/actual gap is ~264,000 tokens.

Evidence

• Session events: %USERPROFILE%\.copilot\session-state\53c1a09d-6bdc-407d-b133-b817e4dc0184\events.jsonl
  (lines 7523–7549: error events, session.compaction_start / session.compaction_complete)
• Runtime log: %USERPROFILE%\.copilot\logs\process-1789352396024-130936.log
  (Persisted compaction checkpoint, Compacted conversation history {"messages_removed":1322,"tokens_removed":572553.0,"generation":1})
• App log: %USERPROFILE%\.copilot\logs\github-app.138040.log (session command compact, 14:18:04.923Z)
• Schema references: %LOCALAPPDATA%\Programs\GitHub Copilot\copilot-sdk\generated\rpc.d.ts
  (SessionContextInfo.compactionThreshold, promptTokenLimit, bufferTokens, categories.mcpTools)