<current_datetime> and random prompt_cache_key still block cross-session prompt caching (SDK 1.0.19 / runtime 1.0.95)

Aberta
  • Algo errado?
#2,915 0 comentários 0 reações 0 responsáveis Ver no GitHub

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
45/100
Tipo de issue
Funcionalidade
Clareza
Razoavelmente clara
Status de atividade
Ativa
Stack de tecnologia
java, python
Domínio
backend-api-design

Direção de pesquisa

Start by reading the runtime request construction for current_datetime and prompt_cache_key, then trace the session options exposed by the SDK. Compare requests from separate sessions to confirm where the changing values enter the prompt. Done means adding a supported way to control the timestamp placement or precision, or to set a stable cache key, with coverage for requests that reuse a stable prompt prefix.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Follow-up to #1297, which was closed as not planned. That issue is still present in the current release and it blocks prompt caching across sessions for workloads that start a new session per request.

Environment

  • github-copilot-sdk 1.0.19 (Python), bundled runtime 1.0.95
  • Copilot billing, gpt-6-luna and gpt-6.1-sol
  • One new session per request, system_message={"mode": "replace", ...}, no tools, enable_session_store=False

What we measured

Five sequential requests with the same synthetic payload (11,454 input tokens each). Usage comes from assistant.usage.

Request Cached input Cache write
A (warm) 0 11,451
A repeated, identical 2,073 9,378
B, same prefix and different tail 2,073 9,378
A with a fixed working directory, twice 2,073 9,378
A with a fixed session_id, three times 2,073 9,378

The results were the same with both models. 2,073 tokens is exactly our system message. Nothing after it is ever read from cache.

What the runtime sends

We pointed a session at a local BYOK provider (type: "openai", wire_api: "responses") that only records requests. When we compared two sessions with the same prompt:

  1. The system message is byte-identical.
  2. The user message starts with <current_datetime>2026-11-11T00:04:49.196-03:00</current_datetime>, so the two requests differ from the first characters after the system message.
  3. prompt_cache_key is a different random value for each session, even when session_id is fixed.
  4. A blob attachment is placed after the datetime too, so no consumer content can come before it.

We found no option or environment variable that disables the datetime or controls prompt_cache_key.

Impact

This covers any consumer that sends a large stable prefix (shared rules, reference context) and opens a new session per request, such as a CI reviewer that splits one pull request into many requests. Those consumers pay for a cache write on every request and never get a cache read beyond the system message. The comment from November 3 on #1297 shows near-100% reuse once the timestamp is fixed, so the timestamp alone accounts for the loss.

Request

Any of the following would solve it for us:

  1. A session option to omit <current_datetime>, or to place it after the user content.
  2. Lower precision, for example the date or the hour, so requests close in time share the prefix.
  3. A way to set prompt_cache_key, or a value derived from the session configuration instead of a random one per session.
Linguagem predominante
Java
Estrelas
10.7k
Forks
1.5k
Nenhum PR com merge em 30d

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de github/copilot-sdk

Todas as issues de github/copilot-sdk

Issues semelhantes

Mais issues de Java

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.