🐛 Bug Report: OpenAI instrumentation drops gen_ai.usage.cache_read.input_tokens when usage.prompt_tokens_details is None

Open Beginner friendly
#4,583 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@rupak-eng is already working on this.

Since Oct 9, 2026.

  • #4586 by @rupak-eng — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
85/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
observability

Research direction

The failing lookup is in _set_response_attributes in opentelemetry/instrumentation/openai/shared/init.py, near line 285, where prompt_tokens_details is read from the usage dict. Compare it with the completion_tokens_details handling in chat_wrappers.py (around line 339), which already guards against None. Done means that a usage object with prompt_tokens_details set to None still produces gen_ai.usage.cache_read.input_tokens = 0, which you can check with the reproduction's httpx MockTransport against the existing span tests.

Written by the indexing model from the issue text.

Description

Which component is this bug for?

OpenAI Instrumentation

📜 Description

_set_response_attributes in opentelemetry/instrumentation/openai/shared/__init__.py (line 285 at df4921a / 0.62.4) does:

prompt_tokens_details = dict(usage.get("prompt_tokens_details", {}))

prompt_tokens_details is optional in the Chat Completions usage object. When a response doesn't include it, the openai SDK still has the key, set to None (CompletionUsage.prompt_tokens_details = None). So usage.get("prompt_tokens_details", {}) returns None, and dict(None) raises TypeError: 'NoneType' object is not iterable.

@dont_throw catches the error and logs it only at DEBUG ("OpenLLMetry failed to trace in _set_response_attributes"), so nothing is visible by default. Because this is the last statement in the function, the span keeps model, id, finish reasons and token counts, but gen_ai.usage.cache_read.input_tokens is never set. It happens for every chat completion without prompt_tokens_details: non-streaming and streaming, sync and async.

The completion_tokens_details lookup in chat_wrappers.py (around line 339) already handles None correctly (if tokens_details:); the prompt side doesn't.

👟 Reproduction steps

pip install openai opentelemetry-sdk opentelemetry-instrumentation-openai

import json, httpx, openai
from opentelemetry.instrumentation.openai import OpenAIInstrumentor
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter

def respond(request):
    # usage without prompt_tokens_details (the field is optional)
    return httpx.Response(200, json={
        "id": "chatcmpl-1", "object": "chat.completion", "created": 0, "model": "gpt-4o-mini",
        "choices": [{"index": 0, "message": {"role": "assistant", "content": "hi"}, "finish_reason": "stop"}],
        "usage": {"prompt_tokens": 7, "completion_tokens": 3, "total_tokens": 10}})

exporter = InMemorySpanExporter()
provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(exporter))
OpenAIInstrumentor().instrument(tracer_provider=provider)

client = openai.OpenAI(api_key="x", http_client=httpx.Client(transport=httpx.MockTransport(respond)))
r = client.chat.completions.create(model="gpt-4o-mini", messages=[{"role": "user", "content": "hello"}])
print("usage.prompt_tokens_details:", r.usage.prompt_tokens_details)

attrs = exporter.get_finished_spans()[0].attributes
print("gen_ai.usage.input_tokens:", attrs.get("gen_ai.usage.input_tokens"))
print("gen_ai.usage.cache_read.input_tokens:", attrs.get("gen_ai.usage.cache_read.input_tokens", "MISSING"))

Enable DEBUG logging for opentelemetry.instrumentation.openai to see the swallowed TypeError. The same happens with stream=True, stream_options={"include_usage": True}.

👍 Expected behavior

A missing or None prompt_tokens_details is treated like an empty one, and the span gets gen_ai.usage.cache_read.input_tokens = 0, as it does when the field is present. For example:

prompt_tokens_details = dict(usage.get("prompt_tokens_details") or {})
👎 Actual Behavior with Screenshots

usage.prompt_tokens_details: None
gen_ai.usage.input_tokens: 7
gen_ai.usage.cache_read.input_tokens: MISSING

DEBUG log: OpenLLMetry failed to trace in _set_response_attributes, error: ... TypeError: 'NoneType' object is not iterable

When the response includes "prompt_tokens_details": {"cached_tokens": 4}, the attribute is recorded correctly (4).

🤖 Python Version

3.12

📃 Provide any additional context for the Bug.

opentelemetry-instrumentation-openai 0.62.4 (also main at df4921a), openai 2.54.0, opentelemetry-sdk 1.x.
Affects any OpenAI-compatible endpoint that omits prompt_tokens_details from usage.

👀 Have you spent some time to check if this bug has been raised before?
  • I checked and didn't find similar issue
Are you willing to submit PR?

None

Dominant language
Python
Stars
7.5k
Forks
1.2k
Avg merge
2d 6h
Merged PRs (30d)
2

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from traceloop/openllmetry

All issues in traceloop/openllmetry

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.