[bot] OpenAI inline moderation (`moderation` param/field) not captured on Chat Completions or Responses API spans

Offen
  • Stimmt etwas nicht?
#235 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
3/5
Geschätzter Aufwand
1-2 Tage
Anfängerfreundlichkeit
64/100
Issue-Typ
Feature
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
ruby
Bereich
ai-infra-agents, observability

Rechercherichtung

Start by reading the four named instrumentation files: the Chat and Responses implementations for both openai and ruby-openai. Trace how each METADATA_FIELDS list and finalize_metadata method collects request and response data, then inspect the related tests to see how metadata is asserted. Done means inline moderation configuration and results are captured on both API spans across both integrations.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

<!-- provider-gap-audit: openai-inline-moderation-not-captured -->

Summary

OpenAI's Chat Completions and Responses APIs support an official moderation request parameter that runs moderation on the request input and generated output in the same call, returning the results in a moderation field on the response. This is distinct from the standalone moderations.create endpoint (which this SDK already instruments). Neither the request parameter nor the response field is captured anywhere in this repo's openai or ruby-openai instrumentation, for either Chat Completions or the Responses API.

What is missing

Request parameter
  • moderation: { model:, policy: { input: { mode: }, output: { mode: } } } — configures which moderation model to run and whether to just score or actively block input/output.
Response field
  • ChatCompletion#moderation — { input: ModerationResult|Error, output: ModerationResult|Error }
  • Response#moderation — same shape, on the Responses API's Response object

Neither field appears in any METADATA_FIELDS list, and neither finalize_metadata method reads the response's moderation attribute, so a moderated call's safety signals are silently dropped from the trace in all four affected instrumentation files.

Braintrust docs status

not_found — https://docs.braintrust.dev/integrations/ai-providers/openai documents a dedicated "Moderation" span type tied to moderations.create (and openai.moderations.create for Ruby) as a standalone endpoint, but has no mention of inline moderation results being captured as part of chat completion or Responses spans, for any language SDK.

Upstream sources

Local repo files inspected

  • lib/braintrust/contrib/openai/instrumentation/chat.rb (METADATA_FIELDS at lines 27–32; finalize_metadata at lines 129–136) — no moderation
  • lib/braintrust/contrib/openai/instrumentation/responses.rb (METADATA_FIELDS at lines 26–31; finalize_metadata at lines 109–112) — no moderation
  • lib/braintrust/contrib/ruby_openai/instrumentation/chat.rb (METADATA_FIELDS at lines 32–37; finalize_metadata at lines 137–143) — no moderation
  • lib/braintrust/contrib/ruby_openai/instrumentation/responses.rb (METADATA_FIELDS at lines 32–37; finalize_metadata at lines 137–140) — no moderation
  • lib/braintrust/contrib/openai/instrumentation/moderations.rb — confirms this SDK already treats moderation as a first-class signal worth tracing, just not when it arrives inline on a generation call
Vorherrschende Sprache
Ruby
Sterne
9
Forks
11
Ø Merge
1 T. 11 Std.
Gemergte PRs (30 T.)
3

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus braintrustdata/braintrust-sdk-ruby

Alle Issues in braintrustdata/braintrust-sdk-ruby

Ähnliche Issues

Weitere Issues zu Ruby

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.