<current_datetime> and random prompt_cache_key still block cross-session prompt caching (SDK 1.0.19 / runtime 1.0.95)

Abierto
  • ¿Algo no está bien?
#2,915 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
45/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
java, python
Área
backend-api-design

Línea de trabajo

Start by reading the runtime request construction for current_datetime and prompt_cache_key, then trace the session options exposed by the SDK. Compare requests from separate sessions to confirm where the changing values enter the prompt. Done means adding a supported way to control the timestamp placement or precision, or to set a stable cache key, with coverage for requests that reuse a stable prompt prefix.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Follow-up to #1297, which was closed as not planned. That issue is still present in the current release and it blocks prompt caching across sessions for workloads that start a new session per request.

Environment

  • github-copilot-sdk 1.0.19 (Python), bundled runtime 1.0.95
  • Copilot billing, gpt-6-luna and gpt-6.1-sol
  • One new session per request, system_message={"mode": "replace", ...}, no tools, enable_session_store=False

What we measured

Five sequential requests with the same synthetic payload (11,454 input tokens each). Usage comes from assistant.usage.

Request Cached input Cache write
A (warm) 0 11,451
A repeated, identical 2,073 9,378
B, same prefix and different tail 2,073 9,378
A with a fixed working directory, twice 2,073 9,378
A with a fixed session_id, three times 2,073 9,378

The results were the same with both models. 2,073 tokens is exactly our system message. Nothing after it is ever read from cache.

What the runtime sends

We pointed a session at a local BYOK provider (type: "openai", wire_api: "responses") that only records requests. When we compared two sessions with the same prompt:

  1. The system message is byte-identical.
  2. The user message starts with <current_datetime>2026-11-11T00:04:49.196-03:00</current_datetime>, so the two requests differ from the first characters after the system message.
  3. prompt_cache_key is a different random value for each session, even when session_id is fixed.
  4. A blob attachment is placed after the datetime too, so no consumer content can come before it.

We found no option or environment variable that disables the datetime or controls prompt_cache_key.

Impact

This covers any consumer that sends a large stable prefix (shared rules, reference context) and opens a new session per request, such as a CI reviewer that splits one pull request into many requests. Those consumers pay for a cache write on every request and never get a cache read beyond the system message. The comment from November 3 on #1297 shows near-100% reuse once the timestamp is fixed, so the timestamp alone accounts for the loss.

Request

Any of the following would solve it for us:

  1. A session option to omit <current_datetime>, or to place it after the user content.
  2. Lower precision, for example the date or the hour, so requests close in time share the prefix.
  3. A way to set prompt_cache_key, or a value derived from the session configuration instead of a random one per session.
Lenguaje dominante
Java
Estrellas
10.7k
Forks
1.5k
Sin PR fusionados en 30 d

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de github/copilot-sdk

Todos los issues de github/copilot-sdk

Issues similares

Más issues de Java

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.