[Bug]: Bedrock LLM prompt_tokens leaves out cache read/write tokens

Aperta
  • Qualcosa non va?
#9,190 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@lalith0407 ci sta già lavorando.

Dal 6/11/2026.

  • #9191 di @lalith0407 — aperta
  • #9269 di @2Zest — aperta

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
25/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Ferma
Stack tecnologico
python
Ambito
backend-api-design

Direzione di ricerca

Start in the Bedrock provider at BedrockCompletion._track_token_usage_internal, named in the issue, and compare its accounting with the Anthropic provider’s cache token handling. Check the mocked Converse usage example and the existing token usage tests; done means prompt_tokens includes input, cache read, and cache write, cached_prompt_tokens and cache_creation_tokens track their respective fields, and the totals add up when consistent.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Description

When using the native Bedrock provider with prompt caching, UsageMetrics.prompt_tokens undercounts billed input. Bedrock Converse reports usage.inputTokens as the non-cached input only; cache hits and writes are separate (cacheReadInputTokens / cacheWriteInputTokens). AWS documents total input as the sum of those three, and totalTokens already includes them.

BedrockCompletion._track_token_usage_internal currently does:

self._token_usage["prompt_tokens"] += input_tokens  # inputTokens only
self._token_usage["cached_prompt_tokens"] += cached_tokens  # read only
# cache writes never recorded; prompt+completion != totalTokens

The Anthropic provider in this repo already folds cache_read_input_tokens + cache_creation_input_tokens into input_tokens before tracking.

Steps to Reproduce

Offline with a mocked converse response (no live AWS needed):

from unittest.mock import patch
from crewai.llm import LLM

llm = LLM(model="bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0")
usage = {
    "inputTokens": 10,
    "outputTokens": 5,
    "totalTokens": 18906,
    "cacheReadInputTokens": 18891,
    "cacheWriteInputTokens": 0,
}
with patch.object(llm._client, "converse") as mock_converse:
    mock_converse.return_value = {
        "output": {"message": {"role": "assistant", "content": [{"text": "OK"}]}},
        "usage": usage,
        "stopReason": "end_turn",
    }
    llm.call("Hello")

print(llm._token_usage)
# main: prompt_tokens=10, cached_prompt_tokens=18891, total_tokens=18906
# expected: prompt_tokens=18901 (== 10+18891), prompt+completion == total

Same gap on the first (cache-write) turn: cacheWriteInputTokens is ignored entirely (cache_creation_tokens stays 0).

Expected behavior
  • prompt_tokens == inputTokens + cacheRead + cacheWrite
  • cached_prompt_tokens == cacheRead
  • cache_creation_tokens == cacheWrite
  • prompt_tokens + completion_tokens == total_tokens (when totalTokens is consistent)
Actual behavior
  • prompt_tokens = non-cached inputTokens only
  • cache writes not tracked
  • prompt + completion != total on cached turns
CrewAI version

main @ 7abc196 (2026-11-06)

Additional context

HUMAN: I hit this while checking Bedrock prompt-caching metrics in crewAI: on a cache-hit turn, prompt_tokens stayed at the non-cached input count while total_tokens already included the cache read, so the parts didn't add up. Looking at the Converse usage fields and how the Anthropic path in this repo folds cache read/write into input, Bedrock wasn't doing the same. Happy to take a fix once this is confirmed.

Lingua principale
Python
Stelle
60.6k
Fork
9.1k
Merge medio
1g 4h
PR unite (30g)
214

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di crewAIInc/crewAI

Tutte le issue di crewAIInc/crewAI

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.