[BOT ISSUE] OpenAI Responses API stream_raw() not instrumented

Aberta
  • Algo errado?
#138 0 comentários 0 reações 0 responsáveis Ver no GitHub

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
58/100
Tipo de issue
Bug
Clareza
Razoavelmente clara
Status de atividade
Pouca atividade
Stack de tecnologia
ruby
Domínio
observability

Direção de pesquisa

Comece por lib/braintrust/contrib/openai/instrumentation/responses.rb e compare seus wrappers create e stream com lib/braintrust/contrib/openai/instrumentation/chat.rb:60-72. Em seguida, inspecione ResponsesPatcher e o tratamento de Internal::Stream em lib/braintrust/contrib/openai/patcher.rb; está concluído quando Responses stream_raw produzir tracing com a entrada especificada, a saída agregada, os metadados e as métricas.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Summary

The official OpenAI Ruby gem provides client.responses.stream_raw() which returns an OpenAI::Internal::Stream of raw server-sent events. This method is not instrumented, meaning users who call stream_raw instead of stream on the Responses API get zero tracing.

This is inconsistent with the Chat Completions integration in this same SDK, which already instruments stream_raw alongside create and stream.

What is missing

  • OpenAI::Resources::Responses#stream_raw — not wrapped, no context is set on the returned stream object
  • Although OpenAI::Internal::Stream is patched by the ChatPatcher, the InternalStream module only traces when ctx[:stream_type] == :raw — a context value that only the Chat stream_raw wrapper sets. Responses stream_raw calls pass through completely untraced.
How the Chat integration handles this (for reference)

The Chat integration wraps stream_raw at lib/braintrust/contrib/openai/instrumentation/chat.rb:60-72, storing context on the stream object with stream_type: :raw. The ChatPatcher then patches OpenAI::Internal::Stream at lib/braintrust/contrib/openai/patcher.rb:58-63 to create spans on consumption.

A similar pattern would be needed for Responses:

  1. Wrap Responses#stream_raw to store context (with a distinct stream_type) on the returned stream
  2. Either extend the existing InternalStream module to handle the responses stream type, or create a dedicated stream handler
What a span should capture
  • Input: the response input (text, conversation items, etc.)
  • Output: aggregated output items from response.completed event (same as ResponseStream)
  • Metadata: model, instructions, tools, provider, endpoint, stream flag
  • Metrics: input_tokens, output_tokens, time_to_first_token

Braintrust docs status

not_found — Braintrust tracing docs at https://www.braintrust.dev/docs/guides/traces and OpenAI integration docs do not specifically mention stream_raw for any provider.

Upstream sources

  • Official OpenAI Ruby SDK: https://github.com/openai/openai-ruby — OpenAI::Resources::Responses defines create, stream, stream_raw, retrieve, retrieve_streaming, delete, cancel, and compact
  • stream_raw returns OpenAI::Internal::Stream (yielding raw SSE events), distinct from stream which returns OpenAI::Helpers::Streaming::ResponseStream

Local repo files inspected

  • lib/braintrust/contrib/openai/instrumentation/responses.rb — wraps only create (line 35) and stream (line 59); no stream_raw method
  • lib/braintrust/contrib/openai/patcher.rb — ResponsesPatcher (line 76) patches ResponseStream but does not patch Internal::Stream for responses
  • lib/braintrust/contrib/openai/instrumentation/chat.rb:60-72 — shows how Chat's stream_raw is wrapped (the pattern to follow)
  • lib/braintrust/contrib/openai/patcher.rb:58-63 — shows Internal::Stream patching for Chat (with stream_type: :raw guard at chat.rb:242)
Linguagem predominante
Ruby
Estrelas
9
Forks
11
Merge médio
1d 3h
PRs com merge (30d)
4

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de braintrustdata/braintrust-sdk-ruby

Todas as issues de braintrustdata/braintrust-sdk-ruby

Issues semelhantes

Mais issues de Ruby

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.