<current_datetime> and random prompt_cache_key still block cross-session prompt caching (SDK 1.0.19 / runtime 1.0.95)

未关闭
  • 哪里不对？
#2,915 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
45/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
java, python
领域
backend-api-design

调研方向

Start by reading the runtime request construction for current_datetime and prompt_cache_key, then trace the session options exposed by the SDK. Compare requests from separate sessions to confirm where the changing values enter the prompt. Done means adding a supported way to control the timestamp placement or precision, or to set a stable cache key, with coverage for requests that reuse a stable prompt prefix.

由索引模型根据 Issue 内容生成。

描述

Follow-up to #1297, which was closed as not planned. That issue is still present in the current release and it blocks prompt caching across sessions for workloads that start a new session per request.

Environment

  • github-copilot-sdk 1.0.19 (Python), bundled runtime 1.0.95
  • Copilot billing, gpt-6-luna and gpt-6.1-sol
  • One new session per request, system_message={"mode": "replace", ...}, no tools, enable_session_store=False

What we measured

Five sequential requests with the same synthetic payload (11,454 input tokens each). Usage comes from assistant.usage.

Request Cached input Cache write
A (warm) 0 11,451
A repeated, identical 2,073 9,378
B, same prefix and different tail 2,073 9,378
A with a fixed working directory, twice 2,073 9,378
A with a fixed session_id, three times 2,073 9,378

The results were the same with both models. 2,073 tokens is exactly our system message. Nothing after it is ever read from cache.

What the runtime sends

We pointed a session at a local BYOK provider (type: "openai", wire_api: "responses") that only records requests. When we compared two sessions with the same prompt:

  1. The system message is byte-identical.
  2. The user message starts with <current_datetime>2026-11-11T00:04:49.196-03:00</current_datetime>, so the two requests differ from the first characters after the system message.
  3. prompt_cache_key is a different random value for each session, even when session_id is fixed.
  4. A blob attachment is placed after the datetime too, so no consumer content can come before it.

We found no option or environment variable that disables the datetime or controls prompt_cache_key.

Impact

This covers any consumer that sends a large stable prefix (shared rules, reference context) and opens a new session per request, such as a CI reviewer that splits one pull request into many requests. Those consumers pay for a cache write on every request and never get a cache read beyond the system message. The comment from November 3 on #1297 shows near-100% reuse once the timestamp is fixed, so the timestamp alone accounts for the loss.

Request

Any of the following would solve it for us:

  1. A session option to omit <current_datetime>, or to place it after the user content.
  2. Lower precision, for example the date or the hour, so requests close in time share the prefix.
  3. A way to set prompt_cache_key, or a value derived from the session configuration instead of a random one per session.
主要语言
Java
星标
10.7k
派生
1.5k
30 天内没有已合并 PR

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

github/copilot-sdk 的其他 Issue

查看 github/copilot-sdk 的全部 Issue

相似的 Issue

更多 Java Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。