Provider API /responses: growing DeepSeek V4.1 Flash context reports 175,680 → 116,559 input tokens as response format switches resp_ → gen_

Aberta
  • Algo errado?
#1,100 0 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
12/100
Tipo de issue
Bug
Clareza
Precisa de esclarecimento
Status de atividade
Ativa
Domínio
ai-infra-agents, backend-api-design

Direção de pesquisa

The payload names no source files, tests or entry points in the repository; the evidence is two archived request bodies and their usage figures from the /provider/v1/responses endpoint. Start by reproducing the request-shape comparison in the report and tracing how the server builds the usage object for response.completed. Done would mean a clear explanation of why resp_ and gen_ responses report different input totals, or a documented accounting difference. The root cause may sit upstream of this repository, so a maintainer should confirm the path first.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Summary

On the native POST /provider/v1/responses endpoint using deepseek/deepseek-v4.1-flash, two successive requests in the same session have a strictly growing input history, but the provider-reported total input usage drops from 175,680 to 116,559 tokens (−59,121, or −33.7%). The first response has an ID beginning resp_ and the second gen_; their SSE reasoning event formats also differ.

This is not a client-side history compaction: I compared the two request JSON bodies captured by a local forwarding gateway. The first 66 input items of the second request are structurally identical to the 66 items in the first request, with only three new items appended. Non-input request fields are identical.

The discrepancy may be an upstream implementation/routing difference, reasoning-history handling difference, or usage-accounting difference; the captures alone cannot establish which.

Environment

  • Date: 2026-11-02 (UTC+08:00)
  • Client: Pi 1.4.0 on Windows
  • Account/access: GOAT plan, with Pi requests forwarded to the Command Code Provider API through a local API gateway (rather than using Command Code's own CLI)
  • Model: deepseek/deepseek-v4.1-flash
  • Protocol: native Responses → Responses on both calls (not a Responses → Chat Completions conversion)
  • The gateway is configured to forward these calls using the native Responses protocol. This report does not assume the gateway is officially supported or endorsed by Command Code.
  • Captured in the forwarding gateway's request archive. No raw prompts, code, headers, keys, or unredacted session identifiers are included here.

Evidence: two successive requests

Metric A: 15:08:00 UTC+8 B: 15:08:11 UTC+8
API /v1/responses /v1/responses
HTTP status 200 200
input item count 66 69
Request-body UTF-8 length 455,034 bytes 458,196 bytes
usage.input_tokens 175,680 116,559
input_tokens_details.cached_tokens 174,976 86,144
Uncached input (difference) 704 30,415
usage.output_tokens 752 28,532
output_tokens_details.reasoning_tokens 491 20,038
Response ID prefix resp_ gen_
Reported service tier default auto
Main reasoning SSE events response.reasoning_summary_text.delta response.reasoning.delta

The input-token counts above are the raw usage.input_tokens in response.completed, including cached input; they are not just uncached/billable tokens. The gateway's summary agrees with the SSE usage returned by the API.

Both captured request bodies are valid, fully parseable JSON, and their UTF-8 lengths exactly match their respective Content-Length headers, even though the gateway's capture metadata contains requestTruncated: true. The second streamed response capture was marked truncated, but it includes the terminal response.completed usage.

Request-shape comparison

I parsed both archived request bodies and verified:

  1. request_B.input[:66] == request_A.input — exact structural equality of every earlier item.
  2. request_B.input[66:] adds exactly:
    • one reasoning item,
    • one function_call (edit),
    • one function_call_output.
  3. All other top-level request fields are equal, including model, stream, store, tools, max_output_tokens, reasoning: {effort: "high", summary: "auto"}, include: ["reasoning.encrypted_content"], and the same prompt_cache_key.
  4. Both responses report the same served model.

No client-side compaction, branching, or content deletion is needed to explain the observed payloads: request B is request A's history plus new items.

Expected behavior

When a request retains every previous input item and adds only three new history items with unchanged model and configuration, its total reported input tokens should not drop by ~59K solely because the response ID/SSE shape changed, unless some history is deliberately removed before upstream inference or there is a documented difference in accounting. Ideally the usage should be consistent and the model should receive the full intended reasoning history.

Actual behavior / concern

  • resp_ response: ~175.7K input tokens, ~99.6% reported cached.
  • gen_ response: ~116.6K input tokens, ~73.9% reported cached.
  • Request B carries more history, including previous reasoning items, yet reports substantially fewer total input tokens.
  • The change coincides with distinct SSE formats and response ID prefixes, which could indicate different backend adapters or accounting paths, but the prefixes alone do not prove different upstream providers.

A local Pi session JSONL also showed several other apparent resp_ → gen_ input-usage drops, followed by increases when resp_ returned; the two archive captures above provide the cleanest direct request-body comparison.

Questions

  1. Do resp_ and gen_ responses on /provider/v1/responses come from different provider adapters or routing paths?
  2. Are historical reasoning items (including summaries) consistently forwarded to the model and counted in input_tokens on both paths?
  3. Could one path omit historical reasoning from the effective prompt, or under-report its usage?
  4. Is there a supported way to identify the actual upstream/adapter for each call without exposing private prompt data?

Related reports

  • #840 — historical reasoning dropped by model slug on Chat Completions (reportedly fixed); the present report concerns native Responses.
  • #949 — deepseek-v4.1-flash Responses reasoning items not round-trippable.
  • #1007 — prompt cache instability / upstream affinity in other Responses models (related symptom, not an identical reproduction).

I can provide narrowly scoped, further sanitized diagnostic details if needed, but cannot publish the raw captures because they contain private source code and conversation content.

Linguagem predominante
Sem dados de linguagem
Estrelas
4.2k
Forks
377
Merge médio
1d 3h
PRs com merge (30d)
6

Preparar o ambiente

Este projeto não oferece contêiner de desenvolvimento, Dockerfile nem guia de contribuição, então a configuração fica por sua conta: comece pelo README e veja nosso guia da primeira contribuição para os passos gerais.

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de CommandCodeAI/command-code

Todas as issues de CommandCodeAI/command-code

Issues semelhantes

Mais issues de AI Infra & Agents

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.