🐛 Bug Report: OpenAI Responses: with_raw_response.create() raises when the body can't be parsed, though the uninstrumented call returns

Abierto Apto para principiantes
#4,717 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
2/5
Tiempo estimado
1-3 horas
Aptitud para principiantes
76/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
python

Línea de trabajo

Empieza en v1/responses_wrappers.py, en las llamadas síncrona y asíncrona a parse_response alrededor de las líneas 674 y 867, y después inspecciona el fallback existente del wrapper cuando falla la creación de TracedData. Añade cobertura usando el transporte HTTP simulado del issue para ambos casos; se considera terminado cuando el JSON malformado devuelve la respuesta sin procesar sin lanzar una excepción ni registrar métricas.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

bug python
Which component is this bug for?

OpenAI Instrumentation

📜 Description

client.responses.with_raw_response.create() is meant to return the raw HTTP response without parsing the body. The user decides whether to call .parse().

The Responses wrapper in v1/responses_wrappers.py calls parse_response(response) itself, so it can read the response ID, model and usage. That call sits outside the wrapper's try/except (around line 674 for sync and line 867 for async). If the body can't be parsed, the exception from our .parse() goes up to the user. The same call returns normally when the instrumentation is off.

The async path (async_parse_response) has the same problem.

👟 Reproduction steps
import httpx, openai
from opentelemetry.instrumentation.openai import OpenAIInstrumentor

def bad(req):
    return httpx.Response(200, headers={"content-type": "application/json"}, content=b"{not json")

def call():
    c = openai.OpenAI(api_key="k", max_retries=0,
                      http_client=httpx.Client(transport=httpx.MockTransport(bad)))
    return c.responses.with_raw_response.create(model="m", input="hi")

print(type(call()).__name__)   # LegacyAPIResponse
OpenAIInstrumentor().instrument()
call()                         # raises JSONDecodeError
👍 Expected behavior

The instrumentation never changes what the call returns or raises. If it can't parse the raw response, it should return the response untraced and record no metrics. The wrapper already does this when building TracedData fails (except Exception: return response).

👎 Actual Behavior with Screenshots
uninstrumented: returned LegacyAPIResponse status 200
instrumented:   raised JSONDecodeError
📃 Provide any additional context for the Bug.
  • This has been there since the first Responses instrumentation (#3052).
  • CodeRabbit raised it on #4713. It suggested recording duration and error.type and then re-raising. That would still break the user's call, and it would count a successful API call as a failed operation. So the fix is to guard the parse, not to record it as an error.
  • Suggested fix: wrap parse_response / async_parse_response in try/except Exception: return response in both the sync and async wrappers. Add a test using the mock transport above that checks the instrumented call returns the same object as the uninstrumented one.
  • Related: #4504, #4551, #4476 (other with_raw_response issues in the Responses wrapper).
👀 Have you spent some time to check if this bug has been raised before?
  • I checked and didn't find similar issue
Lenguaje dominante
Python
Estrellas
7.5k
Forks
1.2k
Merge medio
2 d 6 h
PR fusionados (30 d)
2

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de traceloop/openllmetry

Todos los issues de traceloop/openllmetry

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.