[BOT ISSUE] OpenAI Chat Completions missing reasoning model and web search metadata parameters

Open Beginner friendly
#143 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
76/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
ruby
Domain
observability

Research direction

Start with METADATA_FIELDS in lib/braintrust/contrib/openai/instrumentation/chat.rb and lib/braintrust/contrib/ruby_openai/instrumentation/chat.rb, then compare the existing pattern in instrumentation/responses.rb. Update the Chat Completions metadata coverage for the three named parameters and verify that reasoning and web-search configuration appear in span metadata without changing the existing token parser.

Written by the indexing model from the issue text.

Description

Summary

The OpenAI Chat Completions instrumentation in both the openai and ruby-openai integrations does not capture several parameters that are critical for reasoning models and web search, because the METADATA_FIELDS lists have not been updated for newer API additions. Users of reasoning models (o1, o3, o4-mini) and web-search-enabled completions get incomplete observability — these parameters are silently dropped from span metadata.

What is missing

The METADATA_FIELDS constant in both Chat Completions instrumentations currently includes:

METADATA_FIELDS = %i[
  model frequency_penalty logit_bias logprobs max_tokens n
  presence_penalty response_format seed service_tier stop
  stream stream_options temperature top_p top_logprobs
  tools tool_choice parallel_tool_calls user functions function_call
].freeze

The following parameters accepted by the upstream OpenAI Chat Completions API are not captured:

Parameter Why it matters
max_completion_tokens Required for reasoning models (o1, o3, o4-mini) — replaces max_tokens which doesn't work for these models. Without this, users can't see the output token limit in their traces.
reasoning_effort Controls reasoning depth (low/medium/high) for reasoning models. Directly affects cost, latency, and output quality — essential for observability.
web_search_options Configures web search during generation (OpenAI's built-in web search tool for Chat Completions). Without this, users can't distinguish web-search-enabled generations from regular ones.
Inconsistency with Responses API

The Responses API instrumentation in this SDK already captures the reasoning parameter (which contains effort and other reasoning config). The Chat Completions API uses different parameter names (max_completion_tokens, reasoning_effort) for equivalent functionality, but these are not captured.

Braintrust docs status

not_found — The Braintrust docs at https://www.braintrust.dev/docs/instrument/wrap-providers do not mention max_completion_tokens, reasoning_effort, reasoning models, or web_search_options.

Upstream sources

Local repo files inspected

  • lib/braintrust/contrib/openai/instrumentation/chat.rb (lines 27–32) — METADATA_FIELDS missing max_completion_tokens, reasoning_effort, web_search_options
  • lib/braintrust/contrib/ruby_openai/instrumentation/chat.rb (lines 32–37) — same METADATA_FIELDS list, same omissions
  • lib/braintrust/contrib/openai/instrumentation/responses.rb (lines 26–31) — Responses API already captures reasoning parameter, showing the pattern exists
  • lib/braintrust/contrib/support/openai.rb (lines 14–68) — usage token parser already handles *_tokens_details including reasoning_tokens, so metrics capture reasoning token usage but metadata doesn't capture the reasoning configuration
Dominant language
Ruby
Stars
9
Forks
10
Avg merge
7h 32m
Merged PRs (30d)
2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from braintrustdata/braintrust-sdk-ruby

All issues in braintrustdata/braintrust-sdk-ruby

Similar issues

More Ruby issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.