Bug: Copilot CLI 1.0.82 BYOK silently disables prompt caching (~5x cost)
維護者通常 1 天內回覆
還沒有人認領這個 Issue。
評估
研究方向
找出 BYOK request builder，並將其 request 建構與 subscription path 以及 issue 中描述的 1.0.80 行為進行比較。檢查是否存在 prompt-cache declaration，然後針對 Anthropic-model endpoint 執行多輪 BYOK 工作階段，並驗證 cache-write 與 cache-read 指標不再為零。
由索引模型根據 Issue 內容生成。
描述
Summary
Copilot CLI 1.0.82 in BYOK mode sends chat requests that carry no prompt-cache declaration, so prompt caching is effectively disabled for the whole session. Every turn re-sends the entire growing context at full price — provider usage shows cached_tokens=0 and cache_write/cache_creation_tokens=0 on every request across the session, even though the prefix is stable.
Net effect: ~5x cost for normal multi-turn BYOK sessions.
Setup
- Copilot CLI 1.0.82, BYOK mode (custom OpenAI-compatible
/chat/completionsendpoint serving an Anthropic model,claude-sonnet-5) - Compare baseline: same machine, same endpoint/model, 1.0.80 BYOK → prompt caching works (~97% hit)
- Also tested 1.0.82 in GitHub-subscription mode on the same machine → caching works, so the build is cache-capable
Expected
Once a conversation prefix stabilizes, the provider should report cached_tokens > 0 and small cache_creation_tokens for the new tail (that's what 1.0.80 BYOK and subscription mode report).
Actual
Every request reports zero cache metrics:
"prompt_tokens_details": {
"text_tokens": 146022,
"cached_tokens": 0,
"cache_write_tokens": 0,
"cache_creation_tokens": 0,
"cache_creation_token_details": { "ephemeral_5m_input_tokens": 0 }
},
"cache_read_input_tokens": 0
Over a 150-request session, text_tokens climbed monotonically 118,966 → 285,085 while cached_tokens and cache_write/creation_tokens stayed 0 every time. The zero cache-write/creation is the strong signal: even a cold miss should write the prefix so the next turn can hit; a persistent 0 write means the request never asks for a cache at all.
Evidence
| Metric | 1.0.82 BYOK (broken) | 1.0.80 BYOK (baseline) |
|---|---|---|
| Requests | 150 | 320 |
| Input tokens | 30,075,042 | 100,762,401 |
| Cached tokens | 0 | 98,424,636 |
| Cache hit rate | 0% | 97.7% |
| Cost | $61.21 | $28.92 |
With normal caching, 30.1M input ≈ $11.4; at 0% it costs ≈ $60.2 → ~$49 (~5x) wasted in a single session.
Root-cause hypothesis
The 1.0.82 BYOK request builder omits the prompt-cache declaration (cache_control / ephemeral-cache policy) that subscription mode injects. Because BYOK calls the user's endpoint directly, the missing declaration is only visible to BYOK users.
Severity
High — silent runaway cost on BYOK setups; no CLI error, only a much larger bill. Requires monitoring provider cache hit-rate to detect.
Suggested fix
Make the BYOK path send the same prompt-cache declaration used by subscription mode.
Repro note
I cannot reproduce right now (downgraded to 1.0.80 where caching works). Reproducing requires running 1.0.82 in BYOK against an Anthropic-model endpoint and reading provider cache accounting. Happy to help test a newer build backported if there's a fix candidate.
- 主要語言
- Shell
- 星號
- 11.2k
- 分支
- 1.9k
- 平均合併
- 17 小時 6 分鐘
- 30 天內合併 PR
- 5
環境準備
- 沒有 Dockerfile 或 Docker Compose 檔案
- 沒有 Pull Request 範本
- 閱讀貢獻指南
從這裡開始
- 先讀完整個 Issue，再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫，在一個分支上完成修改。
- 送出 Pull Request，並在描述裡引用這個 Issue 編號。
github/copilot-cli 的其他 Issue
-
area:sessions
難度 2/5 1-3 小時 新手友好度 72/100
github/copilot-cli#4996 ·
維護者通常 1 天內回覆
-
triage
難度 1/5 1 小時以內 新手友好度 88/100
github/copilot-cli#4963 · 1 則留言 ·
維護者通常 1 天內回覆
-
triage
難度 2/5 1-3 小時 新手友好度 75/100
github/copilot-cli#4932 ·
維護者通常 1 天內回覆
-
triage
難度 2/5 1-3 小時 新手友好度 78/100
github/copilot-cli#4909 ·
維護者通常 1 天內回覆
-
triage
難度 2/5 1-3 小時 新手友好度 76/100
github/copilot-cli#4906 ·
維護者通常 1 天內回覆
查看 github/copilot-cli 的全部 Issue
相似的 Issue
-
難度 2/5 1-3 小時 新手友好度 88/100
linuxserver/docker-baseimage-selkies#195 · 3 則留言 ·
-
component:skill
難度 2/5 1-3 小時 新手友好度 88/100
agentic-os-org/ANOLISA#4480 · 1 則留言 ·
維護者通常 1 天內回覆
-
📦 Update Request
難度 2/5 1-3 小時 新手友好度 78/100
-
難度 2/5 1-3 小時 新手友好度 92/100
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 88/100