[Bug]: Bedrock LLM prompt_tokens leaves out cache read/write tokens

Abierto
  • ¿Algo no está bien?
#9,190 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

@lalith0407 ya está trabajando en esto.

Desde el 6/11/2026.

  • #9191 de @lalith0407 — abierto
  • #9269 de @2Zest — abierto

Evaluación

Dificultad
2/5
Tiempo estimado
1-3 horas
Aptitud para principiantes
25/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Estancado
Stack tecnológico
python
Área
backend-api-design

Línea de trabajo

Start in the Bedrock provider at BedrockCompletion._track_token_usage_internal, named in the issue, and compare its accounting with the Anthropic provider’s cache token handling. Check the mocked Converse usage example and the existing token usage tests; done means prompt_tokens includes input, cache read, and cache write, cached_prompt_tokens and cache_creation_tokens track their respective fields, and the totals add up when consistent.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Description

When using the native Bedrock provider with prompt caching, UsageMetrics.prompt_tokens undercounts billed input. Bedrock Converse reports usage.inputTokens as the non-cached input only; cache hits and writes are separate (cacheReadInputTokens / cacheWriteInputTokens). AWS documents total input as the sum of those three, and totalTokens already includes them.

BedrockCompletion._track_token_usage_internal currently does:

self._token_usage["prompt_tokens"] += input_tokens  # inputTokens only
self._token_usage["cached_prompt_tokens"] += cached_tokens  # read only
# cache writes never recorded; prompt+completion != totalTokens

The Anthropic provider in this repo already folds cache_read_input_tokens + cache_creation_input_tokens into input_tokens before tracking.

Steps to Reproduce

Offline with a mocked converse response (no live AWS needed):

from unittest.mock import patch
from crewai.llm import LLM

llm = LLM(model="bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0")
usage = {
    "inputTokens": 10,
    "outputTokens": 5,
    "totalTokens": 18906,
    "cacheReadInputTokens": 18891,
    "cacheWriteInputTokens": 0,
}
with patch.object(llm._client, "converse") as mock_converse:
    mock_converse.return_value = {
        "output": {"message": {"role": "assistant", "content": [{"text": "OK"}]}},
        "usage": usage,
        "stopReason": "end_turn",
    }
    llm.call("Hello")

print(llm._token_usage)
# main: prompt_tokens=10, cached_prompt_tokens=18891, total_tokens=18906
# expected: prompt_tokens=18901 (== 10+18891), prompt+completion == total

Same gap on the first (cache-write) turn: cacheWriteInputTokens is ignored entirely (cache_creation_tokens stays 0).

Expected behavior
  • prompt_tokens == inputTokens + cacheRead + cacheWrite
  • cached_prompt_tokens == cacheRead
  • cache_creation_tokens == cacheWrite
  • prompt_tokens + completion_tokens == total_tokens (when totalTokens is consistent)
Actual behavior
  • prompt_tokens = non-cached inputTokens only
  • cache writes not tracked
  • prompt + completion != total on cached turns
CrewAI version

main @ 7abc196 (2026-11-06)

Additional context

HUMAN: I hit this while checking Bedrock prompt-caching metrics in crewAI: on a cache-hit turn, prompt_tokens stayed at the non-cached input count while total_tokens already included the cache read, so the parts didn't add up. Looking at the Converse usage fields and how the Anthropic path in this repo folds cache read/write into input, Bedrock wasn't doing the same. Happy to take a fix once this is confirmed.

Lenguaje dominante
Python
Estrellas
60.6k
Forks
9.1k
Merge medio
1 d 2 h
PR fusionados (30 d)
214

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de crewAIInc/crewAI

Todos los issues de crewAIInc/crewAI

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.