Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[bot] OpenAI Responses API instrumentation only captures response `id`, missing status/service_tier/model/etc.

Open
  • Is something wrong?
#219 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
65/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
ruby
Domain
devtools, observability-sre

Research direction

Examine the two files mentioned: lib/braintrust/contrib/openai/instrumentation/responses.rb and lib/braintrust/contrib/ruby_openai/instrumentation/responses.rb. Compare their finalize_metadata methods with the sibling chat.rb implementation to see the pattern for capturing response fields. Add the missing fields (status, incomplete_details, error, model, service_tier, created_at, background) from the OpenAI::Models::Responses::Response object. Verify the changes by running any existing tests for the instrumentation or creating a small script to ensure the metadata is populated correctly.

Written by the indexing model from the issue text.

Description

Summary

The Responses API instrumentation in both the openai and ruby-openai integrations captures almost none of the response object's fields into span metadata after the call completes. finalize_metadata only ever sets metadata["id"]. This is materially less detail than the sibling Chat Completions instrumentation in this same SDK, which captures id, created, model, system_fingerprint, and service_tier from the response object after every call (including streaming).

This is a distinct gap from #145 (which covers wrong/missing request-side METADATA_FIELDS, e.g. max_tokens vs max_output_tokens). This issue is about response-side fields that are dropped after the call returns.

What is missing

The upstream Responses API response object (OpenAI::Models::Responses::Response) exposes at minimum:

  • status — e.g. completed, incomplete, failed, in_progress — critical for knowing whether a generation actually finished successfully
  • incomplete_details — explains why a response is incomplete (e.g. hit max_output_tokens) — directly useful for debugging truncated generations
  • error — error details when the response object itself reports a failure
  • model — the actual model that served the request (may differ from the requested model, e.g. due to aliasing)
  • service_tier — the processing tier that actually served the request (may differ from the requested tier)
  • created_at — response creation timestamp
  • background — whether the response was run in background/async mode

None of these are captured. The current finalize_metadata implementations:

# lib/braintrust/contrib/openai/instrumentation/responses.rb:109-112
def finalize_metadata(span, metadata, response)
  metadata["id"] = response.id if response.respond_to?(:id) && response.id
  Support::OTel.set_json_attr(span, "braintrust.metadata", metadata)
end

# lib/braintrust/contrib/ruby_openai/instrumentation/responses.rb:137-140
def finalize_metadata(span, metadata, response_data)
  metadata["id"] = response_data["id"] if response_data["id"]
  Support::OTel.set_json_attr(span, "braintrust.metadata", metadata)
end

only ever set id. The same gap exists in the streaming path (ResponseStream#finalize_stream_span in lib/braintrust/contrib/openai/instrumentation/responses.rb:164-181, and Common.aggregate_responses_chunks/aggregate_responses_events in the respective common.rb files), which likewise only propagate id through to metadata.

For contrast, the parallel Chat Completions finalize_metadata (lib/braintrust/contrib/openai/instrumentation/chat.rb:129-136) captures id, created, model, system_fingerprint, and service_tier — showing this is an internal inconsistency between two instrumented surfaces for the same underlying gem, not a hard technical limitation.

Braintrust docs status

unclear — https://www.braintrust.dev/docs/guides/tracing states that Braintrust auto-instrumentation captures "Model parameters (model name, temperature, etc.)" generally, and the Braintrust docs' own Responses API example (openai.responses.create(...)) is presented as fully auto-traced with "inputs, outputs, latency, tokens, and cost" — but no page enumerates which specific response-object fields (status, incomplete_details, service_tier, etc.) should end up in span metadata for the Responses API.

Upstream sources

Braintrust docs sources checked

Local repo files inspected

  • lib/braintrust/contrib/openai/instrumentation/responses.rb (lines 109-112, 164-181) — finalize_metadata and streaming finalize_stream_span only ever set metadata["id"]
  • lib/braintrust/contrib/ruby_openai/instrumentation/responses.rb (lines 137-140) — identical finalize_metadata, same gap
  • lib/braintrust/contrib/openai/instrumentation/chat.rb (lines 129-136) — sibling Chat Completions finalize_metadata captures id, created, model, system_fingerprint, service_tier, demonstrating the established pattern this SDK already uses elsewhere
  • lib/braintrust/contrib/openai/instrumentation/common.rb (aggregate_responses_events, lines 112-129) — only returns id, output, usage from the completed event's response, dropping status/incomplete_details/model/service_tier/error
  • lib/braintrust/contrib/ruby_openai/instrumentation/common.rb — analogous streaming aggregator for ruby-openai (not re-quoted here, same field-dropping pattern)
Dominant language
Ruby
Stars
9
Forks
10
Avg merge
22h 10m
Merged PRs (30d)
6

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from braintrustdata/braintrust-sdk-ruby

All issues in braintrustdata/braintrust-sdk-ruby

Similar issues

More Ruby issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.