[BOT ISSUE] OpenAI Chat Completions missing reasoning model and web search metadata parameters
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 76/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Quiet
- Tech stack
- ruby
- Domain
- observability
Research direction
Start with METADATA_FIELDS in lib/braintrust/contrib/openai/instrumentation/chat.rb and lib/braintrust/contrib/ruby_openai/instrumentation/chat.rb, then compare the existing pattern in instrumentation/responses.rb. Update the Chat Completions metadata coverage for the three named parameters and verify that reasoning and web-search configuration appear in span metadata without changing the existing token parser.
Written by the indexing model from the issue text.
Description
Summary
The OpenAI Chat Completions instrumentation in both the openai and ruby-openai integrations does not capture several parameters that are critical for reasoning models and web search, because the METADATA_FIELDS lists have not been updated for newer API additions. Users of reasoning models (o1, o3, o4-mini) and web-search-enabled completions get incomplete observability — these parameters are silently dropped from span metadata.
What is missing
The METADATA_FIELDS constant in both Chat Completions instrumentations currently includes:
METADATA_FIELDS = %i[
model frequency_penalty logit_bias logprobs max_tokens n
presence_penalty response_format seed service_tier stop
stream stream_options temperature top_p top_logprobs
tools tool_choice parallel_tool_calls user functions function_call
].freeze
The following parameters accepted by the upstream OpenAI Chat Completions API are not captured:
| Parameter | Why it matters |
|---|---|
max_completion_tokens |
Required for reasoning models (o1, o3, o4-mini) — replaces max_tokens which doesn't work for these models. Without this, users can't see the output token limit in their traces. |
reasoning_effort |
Controls reasoning depth (low/medium/high) for reasoning models. Directly affects cost, latency, and output quality — essential for observability. |
web_search_options |
Configures web search during generation (OpenAI's built-in web search tool for Chat Completions). Without this, users can't distinguish web-search-enabled generations from regular ones. |
Inconsistency with Responses API
The Responses API instrumentation in this SDK already captures the reasoning parameter (which contains effort and other reasoning config). The Chat Completions API uses different parameter names (max_completion_tokens, reasoning_effort) for equivalent functionality, but these are not captured.
Braintrust docs status
not_found — The Braintrust docs at https://www.braintrust.dev/docs/instrument/wrap-providers do not mention max_completion_tokens, reasoning_effort, reasoning models, or web_search_options.
Upstream sources
- Official OpenAI Ruby SDK: https://github.com/openai/openai-ruby —
OpenAI::Resources::Chat::Completions#createacceptsmax_completion_tokens,reasoning_effort,prediction,audio, andweb_search_optionsparameters - OpenAI Chat Completions API docs: https://platform.openai.com/docs/api-reference/chat/create — documents all parameters including reasoning and web search
- OpenAI reasoning models guide: https://platform.openai.com/docs/guides/reasoning — documents
max_completion_tokensandreasoning_effortas the primary configuration parameters
Local repo files inspected
lib/braintrust/contrib/openai/instrumentation/chat.rb(lines 27–32) —METADATA_FIELDSmissingmax_completion_tokens,reasoning_effort,web_search_optionslib/braintrust/contrib/ruby_openai/instrumentation/chat.rb(lines 32–37) — sameMETADATA_FIELDSlist, same omissionslib/braintrust/contrib/openai/instrumentation/responses.rb(lines 26–31) — Responses API already capturesreasoningparameter, showing the pattern existslib/braintrust/contrib/support/openai.rb(lines 14–68) — usage token parser already handles*_tokens_detailsincludingreasoning_tokens, so metrics capture reasoning token usage but metadata doesn't capture the reasoning configuration
- Dominant language
- Ruby
- Stars
- 9
- Forks
- 10
- Avg merge
- 7h 32m
- Merged PRs (30d)
- 2
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from braintrustdata/braintrust-sdk-ruby
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 5/5 Over a week Newbie friendliness 38/100
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
-
Eval::Runner's per-case force_flush adds 10-25s+ per case even when no scorer/classifier uses trace: Openbug ruby
Difficulty 3/5 1-2 days Newbie friendliness 25/100
All issues in braintrustdata/braintrust-sdk-ruby
Similar issues
-
user-reported
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Kong/developer.konghq.com#7316 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
TheOdinProject/curriculum#31408 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
notch8/utk_knapsack#148 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 78/100
Homebrew/homebrew-cask#288729 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100