Provider API /responses: growing DeepSeek V4.1 Flash context reports 175,680 → 116,559 input tokens as response format switches resp_ → gen_

Đang mở
  • Có gì không ổn?
#1,100 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức phù hợp với người mới
12/100
Loại issue
Lỗi
Độ rõ ràng
Cần làm rõ
Mức độ hoạt động
Sôi nổi
Lĩnh vực
ai-infra-agents, backend-api-design

Hướng nghiên cứu

The payload names no source files, tests or entry points in the repository; the evidence is two archived request bodies and their usage figures from the /provider/v1/responses endpoint. Start by reproducing the request-shape comparison in the report and tracing how the server builds the usage object for response.completed. Done would mean a clear explanation of why resp_ and gen_ responses report different input totals, or a documented accounting difference. The root cause may sit upstream of this repository, so a maintainer should confirm the path first.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Summary

On the native POST /provider/v1/responses endpoint using deepseek/deepseek-v4.1-flash, two successive requests in the same session have a strictly growing input history, but the provider-reported total input usage drops from 175,680 to 116,559 tokens (−59,121, or −33.7%). The first response has an ID beginning resp_ and the second gen_; their SSE reasoning event formats also differ.

This is not a client-side history compaction: I compared the two request JSON bodies captured by a local forwarding gateway. The first 66 input items of the second request are structurally identical to the 66 items in the first request, with only three new items appended. Non-input request fields are identical.

The discrepancy may be an upstream implementation/routing difference, reasoning-history handling difference, or usage-accounting difference; the captures alone cannot establish which.

Environment

  • Date: 2026-11-02 (UTC+08:00)
  • Client: Pi 1.4.0 on Windows
  • Account/access: GOAT plan, with Pi requests forwarded to the Command Code Provider API through a local API gateway (rather than using Command Code's own CLI)
  • Model: deepseek/deepseek-v4.1-flash
  • Protocol: native Responses → Responses on both calls (not a Responses → Chat Completions conversion)
  • The gateway is configured to forward these calls using the native Responses protocol. This report does not assume the gateway is officially supported or endorsed by Command Code.
  • Captured in the forwarding gateway's request archive. No raw prompts, code, headers, keys, or unredacted session identifiers are included here.

Evidence: two successive requests

Metric A: 15:08:00 UTC+8 B: 15:08:11 UTC+8
API /v1/responses /v1/responses
HTTP status 200 200
input item count 66 69
Request-body UTF-8 length 455,034 bytes 458,196 bytes
usage.input_tokens 175,680 116,559
input_tokens_details.cached_tokens 174,976 86,144
Uncached input (difference) 704 30,415
usage.output_tokens 752 28,532
output_tokens_details.reasoning_tokens 491 20,038
Response ID prefix resp_ gen_
Reported service tier default auto
Main reasoning SSE events response.reasoning_summary_text.delta response.reasoning.delta

The input-token counts above are the raw usage.input_tokens in response.completed, including cached input; they are not just uncached/billable tokens. The gateway's summary agrees with the SSE usage returned by the API.

Both captured request bodies are valid, fully parseable JSON, and their UTF-8 lengths exactly match their respective Content-Length headers, even though the gateway's capture metadata contains requestTruncated: true. The second streamed response capture was marked truncated, but it includes the terminal response.completed usage.

Request-shape comparison

I parsed both archived request bodies and verified:

  1. request_B.input[:66] == request_A.input — exact structural equality of every earlier item.
  2. request_B.input[66:] adds exactly:
    • one reasoning item,
    • one function_call (edit),
    • one function_call_output.
  3. All other top-level request fields are equal, including model, stream, store, tools, max_output_tokens, reasoning: {effort: "high", summary: "auto"}, include: ["reasoning.encrypted_content"], and the same prompt_cache_key.
  4. Both responses report the same served model.

No client-side compaction, branching, or content deletion is needed to explain the observed payloads: request B is request A's history plus new items.

Expected behavior

When a request retains every previous input item and adds only three new history items with unchanged model and configuration, its total reported input tokens should not drop by ~59K solely because the response ID/SSE shape changed, unless some history is deliberately removed before upstream inference or there is a documented difference in accounting. Ideally the usage should be consistent and the model should receive the full intended reasoning history.

Actual behavior / concern

  • resp_ response: ~175.7K input tokens, ~99.6% reported cached.
  • gen_ response: ~116.6K input tokens, ~73.9% reported cached.
  • Request B carries more history, including previous reasoning items, yet reports substantially fewer total input tokens.
  • The change coincides with distinct SSE formats and response ID prefixes, which could indicate different backend adapters or accounting paths, but the prefixes alone do not prove different upstream providers.

A local Pi session JSONL also showed several other apparent resp_ → gen_ input-usage drops, followed by increases when resp_ returned; the two archive captures above provide the cleanest direct request-body comparison.

Questions

  1. Do resp_ and gen_ responses on /provider/v1/responses come from different provider adapters or routing paths?
  2. Are historical reasoning items (including summaries) consistently forwarded to the model and counted in input_tokens on both paths?
  3. Could one path omit historical reasoning from the effective prompt, or under-report its usage?
  4. Is there a supported way to identify the actual upstream/adapter for each call without exposing private prompt data?

Related reports

  • #840 — historical reasoning dropped by model slug on Chat Completions (reportedly fixed); the present report concerns native Responses.
  • #949 — deepseek-v4.1-flash Responses reasoning items not round-trippable.
  • #1007 — prompt cache instability / upstream affinity in other Responses models (related symptom, not an identical reproduction).

I can provide narrowly scoped, further sanitized diagnostic details if needed, but cannot publish the raw captures because they contain private source code and conversation content.

Ngôn ngữ chính
Không có dữ liệu ngôn ngữ
Star
4.2k
Fork
377
Merge trung bình
1 ngày 3 giờ
Pull request đã merge (30 ngày)
6

Chuẩn bị môi trường

Dự án này không cung cấp dev container, Dockerfile hay hướng dẫn đóng góp, nên bạn cần tự thiết lập môi trường: hãy bắt đầu từ README và xem hướng dẫn đóng góp lần đầu của chúng tôi để biết các bước chung.

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của CommandCodeAI/command-code

Tất cả issue của CommandCodeAI/command-code

Issue tương tự

Thêm issue về AI Infra & Agents

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.