[bot] OpenAI ResponseStream `text` method bypasses instrumentation

Open Beginner friendly
#155 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
ruby

Research direction

Start in lib/braintrust/contrib/openai/instrumentation/responses.rb:118-183, where ResponseStream currently wraps each, and compare the corresponding text handling in lib/braintrust/contrib/openai/instrumentation/chat.rb:140-225. Review lib/braintrust/contrib/openai/patcher.rb:113-119 to confirm the patched class, then verify that consuming ResponseStream via text produces tracing without double-tracing when each is used.

Written by the indexing model from the issue text.

Description

Summary

The OpenAI ResponseStream instrumentation only wraps the each method, but the upstream OpenAI::Helpers::Streaming::ResponseStream class also exposes a text convenience method that bypasses each and returns text deltas directly. Users who consume the Responses API stream via stream.text.each { |t| ... } get zero tracing.

This is inconsistent within the same SDK: the Chat Completions integration wraps both each and text on ChatCompletionStream, but the Responses integration only wraps each on ResponseStream.

What is missing

ResponseStream#text is not wrapped. The upstream SDK's text method returns a fused chain yielding ResponseTextDeltaEvent content that reads from the underlying HTTP stream directly, bypassing the prepended each method on the ResponseStream object.

How the Chat integration handles this (for reference)

The Chat integration wraps both methods on ChatCompletionStream at lib/braintrust/contrib/openai/instrumentation/chat.rb:152-169:

module ChatCompletionStream
  module InstanceMethods
    def each(&block)        # line 152 — wrapped ✓
      ...
    end

    def text                # line 159 — wrapped ✓
      ...
      Enumerator.new do |y|
        trace_consumption(ctx) do
          original_enum.each { |t| y << t }
        end
      end
    end
  end
end

The same pattern should be applied to ResponseStream. The ctx[:consumed] guard already prevents double-tracing when both text and each are wrapped on the same object.

Upstream ResponseStream methods

The upstream class (OpenAI::Helpers::Streaming::ResponseStream) exposes:

  • each — wrapped ✓
  • textNOT wrapped ✗ — bypasses each, reads from internal stream directly
  • until_done — covered (calls each internally)
  • get_final_response — covered (consumes stream via each)
  • get_output_text — covered (delegates to get_final_response)

Only text needs a wrapper because it is the only consumption method that bypasses each.

Braintrust docs status

not_found — Braintrust docs do not mention text as a stream consumption method or differentiate between stream consumption patterns.

Upstream sources

Local repo files inspected

  • lib/braintrust/contrib/openai/instrumentation/responses.rb:118-183ResponseStream module only wraps each (line 128)
  • lib/braintrust/contrib/openai/instrumentation/chat.rb:140-225ChatCompletionStream module wraps both each (line 152) and text (line 159)
  • lib/braintrust/contrib/openai/patcher.rb:113-119ResponsesPatcher.patch_response_stream patches ResponseStream class
Dominant language
Ruby
Stars
9
Forks
10
Avg merge
7h 32m
Merged PRs (30d)
2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from braintrustdata/braintrust-sdk-ruby

All issues in braintrustdata/braintrust-sdk-ruby

Similar issues

More Ruby issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.