🐛 Bug Report: OpenAI instrumentation drops gen_ai.usage.cache_read.input_tokens when usage.prompt_tokens_details is None
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 85/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- python
- Domain
- observability
Research direction
The failing lookup is in _set_response_attributes in opentelemetry/instrumentation/openai/shared/init.py, near line 285, where prompt_tokens_details is read from the usage dict. Compare it with the completion_tokens_details handling in chat_wrappers.py (around line 339), which already guards against None. Done means that a usage object with prompt_tokens_details set to None still produces gen_ai.usage.cache_read.input_tokens = 0, which you can check with the reproduction's httpx MockTransport against the existing span tests.
Written by the indexing model from the issue text.
Description
Which component is this bug for?
OpenAI Instrumentation
📜 Description
_set_response_attributes in opentelemetry/instrumentation/openai/shared/__init__.py (line 285 at df4921a / 0.62.4) does:
prompt_tokens_details = dict(usage.get("prompt_tokens_details", {}))
prompt_tokens_details is optional in the Chat Completions usage object. When a response doesn't include it, the openai SDK still has the key, set to None (CompletionUsage.prompt_tokens_details = None). So usage.get("prompt_tokens_details", {}) returns None, and dict(None) raises TypeError: 'NoneType' object is not iterable.
@dont_throw catches the error and logs it only at DEBUG ("OpenLLMetry failed to trace in _set_response_attributes"), so nothing is visible by default. Because this is the last statement in the function, the span keeps model, id, finish reasons and token counts, but gen_ai.usage.cache_read.input_tokens is never set. It happens for every chat completion without prompt_tokens_details: non-streaming and streaming, sync and async.
The completion_tokens_details lookup in chat_wrappers.py (around line 339) already handles None correctly (if tokens_details:); the prompt side doesn't.
👟 Reproduction steps
pip install openai opentelemetry-sdk opentelemetry-instrumentation-openai
import json, httpx, openai
from opentelemetry.instrumentation.openai import OpenAIInstrumentor
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
def respond(request):
# usage without prompt_tokens_details (the field is optional)
return httpx.Response(200, json={
"id": "chatcmpl-1", "object": "chat.completion", "created": 0, "model": "gpt-4o-mini",
"choices": [{"index": 0, "message": {"role": "assistant", "content": "hi"}, "finish_reason": "stop"}],
"usage": {"prompt_tokens": 7, "completion_tokens": 3, "total_tokens": 10}})
exporter = InMemorySpanExporter()
provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(exporter))
OpenAIInstrumentor().instrument(tracer_provider=provider)
client = openai.OpenAI(api_key="x", http_client=httpx.Client(transport=httpx.MockTransport(respond)))
r = client.chat.completions.create(model="gpt-4o-mini", messages=[{"role": "user", "content": "hello"}])
print("usage.prompt_tokens_details:", r.usage.prompt_tokens_details)
attrs = exporter.get_finished_spans()[0].attributes
print("gen_ai.usage.input_tokens:", attrs.get("gen_ai.usage.input_tokens"))
print("gen_ai.usage.cache_read.input_tokens:", attrs.get("gen_ai.usage.cache_read.input_tokens", "MISSING"))
Enable DEBUG logging for opentelemetry.instrumentation.openai to see the swallowed TypeError. The same happens with stream=True, stream_options={"include_usage": True}.
👍 Expected behavior
A missing or None prompt_tokens_details is treated like an empty one, and the span gets gen_ai.usage.cache_read.input_tokens = 0, as it does when the field is present. For example:
prompt_tokens_details = dict(usage.get("prompt_tokens_details") or {})
👎 Actual Behavior with Screenshots
usage.prompt_tokens_details: None
gen_ai.usage.input_tokens: 7
gen_ai.usage.cache_read.input_tokens: MISSING
DEBUG log: OpenLLMetry failed to trace in _set_response_attributes, error: ... TypeError: 'NoneType' object is not iterable
When the response includes "prompt_tokens_details": {"cached_tokens": 4}, the attribute is recorded correctly (4).
🤖 Python Version
3.12
📃 Provide any additional context for the Bug.
opentelemetry-instrumentation-openai 0.62.4 (also main at df4921a), openai 2.54.0, opentelemetry-sdk 1.x.
Affects any OpenAI-compatible endpoint that omits prompt_tokens_details from usage.
👀 Have you spent some time to check if this bug has been raised before?
- I checked and didn't find similar issue
Are you willing to submit PR?
None
- Dominant language
- Python
- Stars
- 7.5k
- Forks
- 1.2k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 2
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from traceloop/openllmetry
-
bug python
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
traceloop/openllmetry#4717 ·
Maintainers usually reply within 1 day
-
🐛 Bug Report: Incomplete cache token migration after #4243 breaks Bedrock and Groq instrumentationPossibly taken @Quantum-Enigma-dot claimed this 21 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
traceloop/openllmetry#4665 · 1 comment ·
Maintainers usually reply within 1 day
-
🐛 Bug Report: OpenAI instrumentation drops gen_ai.usage.reasoning_tokens on streamed chat completionsPossibly taken @jayfeng7 claimed this 23 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
traceloop/openllmetry#4660 · 1 comment ·
Maintainers usually reply within 1 day
-
Bug Report: Failed dataset deletes still remove local rows and columnsPossibly taken @inchang-ing claimed this 32 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
traceloop/openllmetry#4630 ·
Maintainers usually reply within 1 day
-
🐛 Bug Report: LiteLLM instrumentation does not record seed, stop, frequency_penalty, presence_penaltyPossibly taken @shantam77 claimed this 38 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
traceloop/openllmetry#4598 · 2 comments ·
Maintainers usually reply within 1 day
All issues in traceloop/openllmetry
Similar issues
-
opensubtitlescom: provider rejects pt-PT even though its own converter maps itPossibly taken @paulo-0308 claimed this today. Open
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Diaoul/subliminal#1405 ·
-
Internal ErrorOpenbug
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
frappe/frappe_docker#2071 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
The-Pocket/PocketFlow#161 ·
-
comp/desktop P2 type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
NousResearch/hermes-agent#157702 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
zai-org/zcode-plugins#202 ·
Maintainers usually reply within 1 day