[Bug]: Document whether `prompt_tokens` includes `cache_read_tokens` in accumulated token usage
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 72/100
- Issue type
- Documentation
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- python
- Domain
- documentation, observability
Research direction
Start in openhands-sdk/openhands/sdk/llm/utils/metrics.py: read the TokenUsage field definitions for prompt_tokens and cache_read_tokens, then MetricsSnapshot.cache_hit_rate, which already encodes the denominator rule. Read normalize_usage in telemetry.py to confirm the OpenAI-style mapping. Done when the field descriptions state the overlap rule consistently with cache_hit_rate, and tests/sdk/llm/test_llm_metrics.py still passes unchanged.
Written by the indexing model from the issue text.
Description
Is there an existing issue for the same bug?
- I have searched existing issues and this is not a duplicate.
Bug Description
The SDK exposes accumulated token usage with both prompt_tokens and cache_read_tokens, but the public field-level documentation does not state whether cached tokens are included inside prompt_tokens or reported as a disjoint bucket.
This matters for anyone computing input-token totals or cache-hit rates from accumulated_token_usage: adding prompt_tokens + cache_read_tokens double-counts for providers whose prompt_tokens already includes cached reads, while treating them as fully disjoint undercounts for providers that report cached reads separately.
From code inspection, the intended semantics appear to be provider-shape-dependent:
- For LiteLLM/OpenAI Chat Completions usage,
normalize_usagecopiesusage.prompt_tokenstoprompt_tokensandprompt_tokens_details.cached_tokenstocache_read_tokens; cached reads are therefore a detail/subset of the prompt total. - The
cache_hit_rateimplementation already encodes this interpretation: it normally usesprompt_tokensas the denominator and only switches toprompt_tokens + cache_read_tokenswhencache_read_tokens > prompt_tokens(the separately-reported/ACP-style case).
That invariant is currently only discoverable by reading the metrics/telemetry implementation. It should be explicit in the model field documentation and/or SDK metrics docs so API consumers do not have to infer it.
Steps to Reproduce
- Inspect the token-usage model fields for
accumulated_token_usage(prompt_tokens,cache_read_tokens). - Try to determine from the public field descriptions whether
cache_read_tokensis included inprompt_tokens. - Observe that the field descriptions only say the values must be non-negative; the overlap rule is not stated there.
Actual Behavior
Public consumers cannot tell whether to compute total input tokens as prompt_tokens or as prompt_tokens + cache_read_tokens without reading SDK internals.
Expected Behavior
The overlap semantics are documented at the point of use:
- For LiteLLM/OpenAI-style usage:
cache_read_tokensis included inprompt_tokens. - For providers/ACP-style reporting where cached reads are reported separately: consumers should follow the same denominator rule used by
cache_hit_raterather than assuming the fields are always disjoint or always overlapping.
Relevant Code
openhands-sdk/openhands/sdk/llm/utils/telemetry.py:normalize_usagemaps provider usage intoprompt_tokensandcache_read_tokens.openhands-sdk/openhands/sdk/llm/utils/metrics.py:TokenUsagefields andMetricsSnapshot.cache_hit_ratedenominator logic.TokenUsage.__add__sums the fields independently, which is consistent withprompt_tokensbeing the prompt total andcache_read_tokensbeing a cached-read detail for OpenAI-style usage.
Acceptance Criteria
-
TokenUsage.prompt_tokensandTokenUsage.cache_read_tokensfield descriptions (or the SDK metrics documentation) explicitly state when cached reads are included inprompt_tokensand when they are reported separately. - The documented total-input-token guidance matches the denominator rule already implemented by
MetricsSnapshot.cache_hit_rate. - Existing normalization/accumulation behavior is covered by a test or doc example showing an OpenAI-style usage object with cached tokens and verifying that consumers should not double-count them.
- No behavior change is required unless maintainers decide the current provider-dependent semantics should be normalized instead; in that case the migration/behavior change is called out explicitly in the fix.
Additional Context
This came from a documentation/API-clarity review of the SDK metrics surface. No private user data, account information, or conversation data is included.
This issue was filed by an AI agent (OpenHands) on behalf of the user.
OpenHands AI triage
The following comments and acceptance criteria were added by the OpenHands AI agent.
Triage
This is an in-scope documentation-clarity issue owned by this repository. TokenUsage lives in openhands-sdk/openhands/sdk/llm/utils/metrics.py, and normalize_usage in openhands-sdk/openhands/sdk/llm/utils/telemetry.py maps LiteLLM/OpenAI prompt_tokens plus prompt_tokens_details.cached_tokens into prompt_tokens and cache_read_tokens. The intended rule is already encoded in MetricsSnapshot.cache_hit_rate: cached reads are a subset of prompt_tokens for OpenAI-style usage, and are reported separately for provider/ACP-style usage where cache_read_tokens can exceed prompt_tokens.
Smallest coherent scope: state the overlap rule in the public TokenUsage field descriptions and align the wording with the existing cache_hit_rate denominator rule, so a consumer can compute input totals without reading internals. No in-repo metrics markdown guide exists (SDK docs live at docs.openhands.dev), so the field descriptions are the authoritative in-repo surface.
Non-goals: no change to normalize_usage, TokenUsage.__add__, merge, diff, or cache_hit_rate behavior; no new token fields; no provider-specific normalization; no change to cache_write_tokens semantics.
Acceptance Criteria
-
TokenUsage.prompt_tokensandTokenUsage.cache_read_tokensfield descriptions state that, for LiteLLM/OpenAI-style usage,cache_read_tokensis included in (a subset of)prompt_tokens, and that for provider/ACP-style usage cached reads are reported separately, socache_read_tokensmay exceedprompt_tokens. - The documented guidance for total input tokens matches the
MetricsSnapshot.cache_hit_ratedenominator rule: useprompt_tokens, switching toprompt_tokens + cache_read_tokensonly whencache_read_tokens > prompt_tokens. - The
cache_hit_ratedocstring and the new field descriptions use consistent terminology (included/subset versus reported separately) so the two surfaces do not contradict each other. - A documentation example (docstring or SDK docs) or a behavior test shows an OpenAI-style usage object with cached tokens and states that consumers must not add
cache_read_tokenstoprompt_tokensagain; if written as a test it asserts the resulting total orcache_hit_rate, not docstring text. - Existing tests in
tests/sdk/llm/test_llm_metrics.pypass unchanged, confirming no runtime behavior change.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 797
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 354
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from OpenHands/software-agent-sdk
-
[Bug]: `list_directory` reports a complete listing as truncated when a directory has exactly 500 entriesPossibly taken @balabommablock-cpu claimed this 29 days ago. Openbug priority:low ready-for-dev
Difficulty 2/5 1-3 hours Newbie friendliness 87/100
OpenHands/software-agent-sdk#6133 ·
Maintainers usually reply within 1 day
-
[Bug]: Command hook that exits 1 is treated as blocking when its stdout is a JSON `deny` or `continue: false`Possibly taken A pull request linked to this issue is open or already merged. Openbug priority:low ready-for-dev
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
OpenHands/software-agent-sdk#6116 · 3 comments ·
Maintainers usually reply within 1 day
-
[Bug]: TypeScript GET-with-body requests remain pending after a truncated responsePossibly taken @BlueRaddish claimed this 34 days ago. Openbug priority:low ready-for-dev
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
OpenHands/software-agent-sdk#6059 ·
Maintainers usually reply within 1 day
-
priority:low
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
OpenHands/software-agent-sdk#5708 · 1 comment ·
Maintainers usually reply within 1 day
-
[Bug] Unauthenticated model-info lookup still emits GET /v1/model/info when key is NonePossibly taken @daviskay75 claimed this 49 days ago. Openpriority:low
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
OpenHands/software-agent-sdk#5454 · 3 comments ·
Maintainers usually reply within 1 day
All issues in OpenHands/software-agent-sdk
Similar issues
-
[Bug]: reme CLI crashes when the running server used a relative config pathPossibly taken @nurlanmnn claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
agentscope-ai/ReMe#770 ·
Maintainers usually reply within 1 day
-
Pi: paused model calls are reported as "not signed in" when the model's provider comes from an extensionPossibly taken @balcsida claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
microsoft/TokenEconomizer#288 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 67/100
sickn33/agentic-awesome-skills#2437 ·
Maintainers usually reply within 1 day
-
site-bug triage
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
ethereum/consensus-specs#5846 ·
Maintainers usually reply within 1 day