🐛 Bug Report: OpenAI instrumentation drops gen_ai.usage.cache_read.input_tokens when usage.prompt_tokens_details is None

未关闭 适合新手
#4,583 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

@rupak-eng 已经在做这个了。

开始于 2026年10月9日。

  • #4586 来自 @rupak-eng —— 未关闭

评估

难度
2/5
预计耗时
1-3 小时
新手友好度
85/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
python
领域
observability

调研方向

失败的查找位于 opentelemetry/instrumentation/openai/shared/init.py 中第 285 行附近的 _set_response_attributes,这里从 usage 字典中读取 prompt_tokens_details。请将其与 chat_wrappers.py 中第 339 行附近的 completion_tokens_details 处理方式进行比较,后者已经对 None 做了防护。当 prompt_tokens_details 为 None 的 usage 对象仍然生成 gen_ai.usage.cache_read.input_tokens = 0 时即为完成,可以使用复现代码中的 httpx MockTransport 对照现有的 span 测试进行验证。

由索引模型根据 Issue 内容生成。

描述

Which component is this bug for?

OpenAI Instrumentation

📜 Description

_set_response_attributes in opentelemetry/instrumentation/openai/shared/__init__.py (line 285 at df4921a / 0.62.4) does:

prompt_tokens_details = dict(usage.get("prompt_tokens_details", {}))

prompt_tokens_details is optional in the Chat Completions usage object. When a response doesn't include it, the openai SDK still has the key, set to None (CompletionUsage.prompt_tokens_details = None). So usage.get("prompt_tokens_details", {}) returns None, and dict(None) raises TypeError: 'NoneType' object is not iterable.

@dont_throw catches the error and logs it only at DEBUG ("OpenLLMetry failed to trace in _set_response_attributes"), so nothing is visible by default. Because this is the last statement in the function, the span keeps model, id, finish reasons and token counts, but gen_ai.usage.cache_read.input_tokens is never set. It happens for every chat completion without prompt_tokens_details: non-streaming and streaming, sync and async.

The completion_tokens_details lookup in chat_wrappers.py (around line 339) already handles None correctly (if tokens_details:); the prompt side doesn't.

👟 Reproduction steps

pip install openai opentelemetry-sdk opentelemetry-instrumentation-openai

import json, httpx, openai
from opentelemetry.instrumentation.openai import OpenAIInstrumentor
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter

def respond(request):
    # usage without prompt_tokens_details (the field is optional)
    return httpx.Response(200, json={
        "id": "chatcmpl-1", "object": "chat.completion", "created": 0, "model": "gpt-4o-mini",
        "choices": [{"index": 0, "message": {"role": "assistant", "content": "hi"}, "finish_reason": "stop"}],
        "usage": {"prompt_tokens": 7, "completion_tokens": 3, "total_tokens": 10}})

exporter = InMemorySpanExporter()
provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(exporter))
OpenAIInstrumentor().instrument(tracer_provider=provider)

client = openai.OpenAI(api_key="x", http_client=httpx.Client(transport=httpx.MockTransport(respond)))
r = client.chat.completions.create(model="gpt-4o-mini", messages=[{"role": "user", "content": "hello"}])
print("usage.prompt_tokens_details:", r.usage.prompt_tokens_details)

attrs = exporter.get_finished_spans()[0].attributes
print("gen_ai.usage.input_tokens:", attrs.get("gen_ai.usage.input_tokens"))
print("gen_ai.usage.cache_read.input_tokens:", attrs.get("gen_ai.usage.cache_read.input_tokens", "MISSING"))

Enable DEBUG logging for opentelemetry.instrumentation.openai to see the swallowed TypeError. The same happens with stream=True, stream_options={"include_usage": True}.

👍 Expected behavior

A missing or None prompt_tokens_details is treated like an empty one, and the span gets gen_ai.usage.cache_read.input_tokens = 0, as it does when the field is present. For example:

prompt_tokens_details = dict(usage.get("prompt_tokens_details") or {})
👎 Actual Behavior with Screenshots

usage.prompt_tokens_details: None
gen_ai.usage.input_tokens: 7
gen_ai.usage.cache_read.input_tokens: MISSING

DEBUG log: OpenLLMetry failed to trace in _set_response_attributes, error: ... TypeError: 'NoneType' object is not iterable

When the response includes "prompt_tokens_details": {"cached_tokens": 4}, the attribute is recorded correctly (4).

🤖 Python Version

3.12

📃 Provide any additional context for the Bug.

opentelemetry-instrumentation-openai 0.62.4 (also main at df4921a), openai 2.54.0, opentelemetry-sdk 1.x.
Affects any OpenAI-compatible endpoint that omits prompt_tokens_details from usage.

👀 Have you spent some time to check if this bug has been raised before?
  • I checked and didn't find similar issue
Are you willing to submit PR?

None

主要语言
Python
星标
7.5k
派生
1.2k
平均合并
1 天 23 小时
30 天内合并 PR
3

环境准备

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

traceloop/openllmetry 的其他 Issue

查看 traceloop/openllmetry 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。