<current_datetime> and random prompt_cache_key still block cross-session prompt caching (SDK 1.0.19 / runtime 1.0.95)

未關閉
  • 哪裡不對？
#2,915 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

評估

難度
4/5
預估耗時
3-5 天
新手友好度
45/100
Issue 類型
功能
描述清晰度
基本清楚
活躍度
活躍
技術堆疊
java, python
領域
backend-api-design

研究方向

Start by reading the runtime request construction for current_datetime and prompt_cache_key, then trace the session options exposed by the SDK. Compare requests from separate sessions to confirm where the changing values enter the prompt. Done means adding a supported way to control the timestamp placement or precision, or to set a stable cache key, with coverage for requests that reuse a stable prompt prefix.

由索引模型根據 Issue 內容生成。

描述

Follow-up to #1297, which was closed as not planned. That issue is still present in the current release and it blocks prompt caching across sessions for workloads that start a new session per request.

Environment

  • github-copilot-sdk 1.0.19 (Python), bundled runtime 1.0.95
  • Copilot billing, gpt-6-luna and gpt-6.1-sol
  • One new session per request, system_message={"mode": "replace", ...}, no tools, enable_session_store=False

What we measured

Five sequential requests with the same synthetic payload (11,454 input tokens each). Usage comes from assistant.usage.

Request Cached input Cache write
A (warm) 0 11,451
A repeated, identical 2,073 9,378
B, same prefix and different tail 2,073 9,378
A with a fixed working directory, twice 2,073 9,378
A with a fixed session_id, three times 2,073 9,378

The results were the same with both models. 2,073 tokens is exactly our system message. Nothing after it is ever read from cache.

What the runtime sends

We pointed a session at a local BYOK provider (type: "openai", wire_api: "responses") that only records requests. When we compared two sessions with the same prompt:

  1. The system message is byte-identical.
  2. The user message starts with <current_datetime>2026-11-11T00:04:49.196-03:00</current_datetime>, so the two requests differ from the first characters after the system message.
  3. prompt_cache_key is a different random value for each session, even when session_id is fixed.
  4. A blob attachment is placed after the datetime too, so no consumer content can come before it.

We found no option or environment variable that disables the datetime or controls prompt_cache_key.

Impact

This covers any consumer that sends a large stable prefix (shared rules, reference context) and opens a new session per request, such as a CI reviewer that splits one pull request into many requests. Those consumers pay for a cache write on every request and never get a cache read beyond the system message. The comment from November 3 on #1297 shows near-100% reuse once the timestamp is fixed, so the timestamp alone accounts for the loss.

Request

Any of the following would solve it for us:

  1. A session option to omit <current_datetime>, or to place it after the user content.
  2. Lower precision, for example the date or the hour, so requests close in time share the prefix.
  3. A way to set prompt_cache_key, or a value derived from the session configuration instead of a random one per session.
主要語言
Java
星號
10.7k
分支
1.5k
30 天內沒有已合併 PR

環境準備

  • 沒有 Dockerfile 或 Docker Compose 檔案
  • 沒有 Pull Request 範本
  • 閱讀貢獻指南

從這裡開始

  1. 先讀完整個 Issue，再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫，在一個分支上完成修改。
  4. 送出 Pull Request，並在描述裡引用這個 Issue 編號。

github/copilot-sdk 的其他 Issue

查看 github/copilot-sdk 的全部 Issue

相似的 Issue

更多 Java Issue

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。