OpenAI cost calculation ignores cached input tokens (over-costs cache hits)
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 74/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Quiet
- Tech stack
- typescript
Research direction
Start in core/llm/utils/calculateRequestCost.ts at calculateOpenAICost and compare it with the Anthropic branch's cache-token handling. Review the OpenAI pricing entries and promptTokensDetails usage fields, then verify that cached and uncached input are priced separately and the reported total matches the provider's charge.
Written by the indexing model from the issue text.
Description
Summary
calculateOpenAICost in core/llm/utils/calculateRequestCost.ts bills the full promptTokens at the standard input rate and never accounts for cached input tokens, so requests that hit OpenAI's prompt cache are over-costed. The Anthropic branch in the same file already handles cache tokens; the OpenAI branch does not.
Detail
OpenAI reports cached input as usage.prompt_tokens_details.cached_tokens, and prompt_tokens includes those cached tokens. Cached input is billed at a discount (for example gpt-4o cached input is half the standard input rate). The cost function charges every prompt token at full rate:
const inputCost = (usage.promptTokens / 1_000_000) * modelPricing.input;
// no use of usage.promptTokensDetails.cachedTokens
Compare calculateAnthropicCost, which reads usage.promptTokensDetails and prices cachedTokens / cacheWriteTokens at their own rates.
Effect
For an OpenAI request with cached input (common with long, stable system prompts), the reported cost is higher than the actual OpenAI charge — the cached portion is billed at full price instead of the cache-read discount.
Suggested direction
Give the OpenAI pricing table a cachedInput rate and subtract the cached tokens from the full-rate input, pricing them separately, the way the Anthropic branch does:
const cachedTokens = usage.promptTokensDetails?.cachedTokens ?? 0;
const uncachedInput = Math.max(0, usage.promptTokens - cachedTokens);
const inputCost = (uncachedInput / 1_000_000) * modelPricing.input
+ (cachedTokens / 1_000_000) * modelPricing.cachedInput;
Happy to open a PR with the cached-input rates for the models already listed if that direction sounds right.
- Dominant language
- TypeScript
- Stars
- 36k
- Forks
- 5.4k
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from continuedev/continue
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
continuedev/continue#13291 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
continuedev/continue#13254 · 2 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
continuedev/continue#13233 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 85/100
continuedev/continue#13198 · 1 comment ·
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
continuedev/continue#13197 ·
All issues in continuedev/continue
Similar issues
-
calcite-components needs triage refactor
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Esri/calcite-design-system#15203 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 91/100
-
community first-timers-only good first issue hacktoberfest help wanted low hanging fruit up-for-grabs
Difficulty 1/5 Under an hour Newbie friendliness 95/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Automattic/studio#4908 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100