[bot] OpenAI ResponseStream `text` method bypasses instrumentation
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Quiet
- Tech stack
- ruby
- Domain
- observability-sre
Research direction
Start in lib/braintrust/contrib/openai/instrumentation/responses.rb:118-183, where ResponseStream currently wraps each, and compare the corresponding text handling in lib/braintrust/contrib/openai/instrumentation/chat.rb:140-225. Review lib/braintrust/contrib/openai/patcher.rb:113-119 to confirm the patched class, then verify that consuming ResponseStream via text produces tracing without double-tracing when each is used.
Written by the indexing model from the issue text.
Description
Summary
The OpenAI ResponseStream instrumentation only wraps the each method, but the upstream OpenAI::Helpers::Streaming::ResponseStream class also exposes a text convenience method that bypasses each and returns text deltas directly. Users who consume the Responses API stream via stream.text.each { |t| ... } get zero tracing.
This is inconsistent within the same SDK: the Chat Completions integration wraps both each and text on ChatCompletionStream, but the Responses integration only wraps each on ResponseStream.
What is missing
ResponseStream#text is not wrapped. The upstream SDK's text method returns a fused chain yielding ResponseTextDeltaEvent content that reads from the underlying HTTP stream directly, bypassing the prepended each method on the ResponseStream object.
How the Chat integration handles this (for reference)
The Chat integration wraps both methods on ChatCompletionStream at lib/braintrust/contrib/openai/instrumentation/chat.rb:152-169:
module ChatCompletionStream
module InstanceMethods
def each(&block) # line 152 — wrapped ✓
...
end
def text # line 159 — wrapped ✓
...
Enumerator.new do |y|
trace_consumption(ctx) do
original_enum.each { |t| y << t }
end
end
end
end
end
The same pattern should be applied to ResponseStream. The ctx[:consumed] guard already prevents double-tracing when both text and each are wrapped on the same object.
Upstream ResponseStream methods
The upstream class (OpenAI::Helpers::Streaming::ResponseStream) exposes:
each— wrapped ✓text— NOT wrapped ✗ — bypasseseach, reads from internal stream directlyuntil_done— covered (callseachinternally)get_final_response— covered (consumes stream viaeach)get_output_text— covered (delegates toget_final_response)
Only text needs a wrapper because it is the only consumption method that bypasses each.
Braintrust docs status
not_found — Braintrust docs do not mention text as a stream consumption method or differentiate between stream consumption patterns.
Upstream sources
- OpenAI Ruby SDK
ResponseStream: https://github.com/openai/openai-ruby/blob/main/lib/openai/helpers/streaming/response_stream.rb — definestext,until_done,get_final_response,get_output_text - OpenAI Ruby SDK
ChatCompletionStream: https://github.com/openai/openai-ruby/blob/main/lib/openai/helpers/streaming/chat_completion_stream.rb — definestext(already wrapped by this SDK)
Local repo files inspected
lib/braintrust/contrib/openai/instrumentation/responses.rb:118-183—ResponseStreammodule only wrapseach(line 128)lib/braintrust/contrib/openai/instrumentation/chat.rb:140-225—ChatCompletionStreammodule wraps botheach(line 152) andtext(line 159)lib/braintrust/contrib/openai/patcher.rb:113-119—ResponsesPatcher.patch_response_streampatchesResponseStreamclass
- Dominant language
- Ruby
- Stars
- 9
- Forks
- 10
- Avg merge
- 7h 32m
- Merged PRs (30d)
- 2
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from braintrustdata/braintrust-sdk-ruby
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 5/5 Over a week Newbie friendliness 38/100
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
-
Eval::Runner's per-case force_flush adds 10-25s+ per case even when no scorer/classifier uses trace: Openbug ruby
Difficulty 3/5 1-2 days Newbie friendliness 25/100
All issues in braintrustdata/braintrust-sdk-ruby
Similar issues
-
user-reported
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Kong/developer.konghq.com#7316 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
TheOdinProject/curriculum#31408 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
notch8/utk_knapsack#148 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 78/100
Homebrew/homebrew-cask#288729 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100