Provider API /responses: growing DeepSeek V4.1 Flash context reports 175,680 → 116,559 input tokens as response format switches resp_ → gen_

Offen
  • Stimmt etwas nicht?
#1,100 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
12/100
Issue-Typ
Bug
Klarheit
Muss geklärt werden
Aktivitätsstatus
Aktiv
Bereich
ai-infra-agents, backend-api-design

Rechercherichtung

The payload names no source files, tests or entry points in the repository; the evidence is two archived request bodies and their usage figures from the /provider/v1/responses endpoint. Start by reproducing the request-shape comparison in the report and tracing how the server builds the usage object for response.completed. Done would mean a clear explanation of why resp_ and gen_ responses report different input totals, or a documented accounting difference. The root cause may sit upstream of this repository, so a maintainer should confirm the path first.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Summary

On the native POST /provider/v1/responses endpoint using deepseek/deepseek-v4.1-flash, two successive requests in the same session have a strictly growing input history, but the provider-reported total input usage drops from 175,680 to 116,559 tokens (−59,121, or −33.7%). The first response has an ID beginning resp_ and the second gen_; their SSE reasoning event formats also differ.

This is not a client-side history compaction: I compared the two request JSON bodies captured by a local forwarding gateway. The first 66 input items of the second request are structurally identical to the 66 items in the first request, with only three new items appended. Non-input request fields are identical.

The discrepancy may be an upstream implementation/routing difference, reasoning-history handling difference, or usage-accounting difference; the captures alone cannot establish which.

Environment

  • Date: 2026-11-02 (UTC+08:00)
  • Client: Pi 1.4.0 on Windows
  • Account/access: GOAT plan, with Pi requests forwarded to the Command Code Provider API through a local API gateway (rather than using Command Code's own CLI)
  • Model: deepseek/deepseek-v4.1-flash
  • Protocol: native Responses → Responses on both calls (not a Responses → Chat Completions conversion)
  • The gateway is configured to forward these calls using the native Responses protocol. This report does not assume the gateway is officially supported or endorsed by Command Code.
  • Captured in the forwarding gateway's request archive. No raw prompts, code, headers, keys, or unredacted session identifiers are included here.

Evidence: two successive requests

Metric A: 15:08:00 UTC+8 B: 15:08:11 UTC+8
API /v1/responses /v1/responses
HTTP status 200 200
input item count 66 69
Request-body UTF-8 length 455,034 bytes 458,196 bytes
usage.input_tokens 175,680 116,559
input_tokens_details.cached_tokens 174,976 86,144
Uncached input (difference) 704 30,415
usage.output_tokens 752 28,532
output_tokens_details.reasoning_tokens 491 20,038
Response ID prefix resp_ gen_
Reported service tier default auto
Main reasoning SSE events response.reasoning_summary_text.delta response.reasoning.delta

The input-token counts above are the raw usage.input_tokens in response.completed, including cached input; they are not just uncached/billable tokens. The gateway's summary agrees with the SSE usage returned by the API.

Both captured request bodies are valid, fully parseable JSON, and their UTF-8 lengths exactly match their respective Content-Length headers, even though the gateway's capture metadata contains requestTruncated: true. The second streamed response capture was marked truncated, but it includes the terminal response.completed usage.

Request-shape comparison

I parsed both archived request bodies and verified:

  1. request_B.input[:66] == request_A.input — exact structural equality of every earlier item.
  2. request_B.input[66:] adds exactly:
    • one reasoning item,
    • one function_call (edit),
    • one function_call_output.
  3. All other top-level request fields are equal, including model, stream, store, tools, max_output_tokens, reasoning: {effort: "high", summary: "auto"}, include: ["reasoning.encrypted_content"], and the same prompt_cache_key.
  4. Both responses report the same served model.

No client-side compaction, branching, or content deletion is needed to explain the observed payloads: request B is request A's history plus new items.

Expected behavior

When a request retains every previous input item and adds only three new history items with unchanged model and configuration, its total reported input tokens should not drop by ~59K solely because the response ID/SSE shape changed, unless some history is deliberately removed before upstream inference or there is a documented difference in accounting. Ideally the usage should be consistent and the model should receive the full intended reasoning history.

Actual behavior / concern

  • resp_ response: ~175.7K input tokens, ~99.6% reported cached.
  • gen_ response: ~116.6K input tokens, ~73.9% reported cached.
  • Request B carries more history, including previous reasoning items, yet reports substantially fewer total input tokens.
  • The change coincides with distinct SSE formats and response ID prefixes, which could indicate different backend adapters or accounting paths, but the prefixes alone do not prove different upstream providers.

A local Pi session JSONL also showed several other apparent resp_ → gen_ input-usage drops, followed by increases when resp_ returned; the two archive captures above provide the cleanest direct request-body comparison.

Questions

  1. Do resp_ and gen_ responses on /provider/v1/responses come from different provider adapters or routing paths?
  2. Are historical reasoning items (including summaries) consistently forwarded to the model and counted in input_tokens on both paths?
  3. Could one path omit historical reasoning from the effective prompt, or under-report its usage?
  4. Is there a supported way to identify the actual upstream/adapter for each call without exposing private prompt data?

Related reports

  • #840 — historical reasoning dropped by model slug on Chat Completions (reportedly fixed); the present report concerns native Responses.
  • #949 — deepseek-v4.1-flash Responses reasoning items not round-trippable.
  • #1007 — prompt cache instability / upstream affinity in other Responses models (related symptom, not an identical reproduction).

I can provide narrowly scoped, further sanitized diagnostic details if needed, but cannot publish the raw captures because they contain private source code and conversation content.

Vorherrschende Sprache
Keine Sprachdaten
Sterne
4.2k
Forks
377
Ø Merge
1 T. 3 Std.
Gemergte PRs (30 T.)
6

Entwicklungsumgebung

Dieses Projekt bietet weder Dev-Container noch Dockerfile noch Beitragsleitfaden – die Einrichtung liegt bei Ihnen. Beginnen Sie mit der README; die allgemeinen Schritte stehen in unserem Leitfaden für den ersten Beitrag.

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus CommandCodeAI/command-code

Alle Issues in CommandCodeAI/command-code

Ähnliche Issues

Weitere Issues zu AI Infra & Agents

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.