[bot] OpenAI inline moderation (`moderation` param/field) not captured on Chat Completions or Responses API spans

Ouverte
  • Quelque chose ne va pas ?
#235 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

Évaluation

Difficulté
3/5
Temps estimé
1-2 jours
Accessibilité débutants
64/100
Type d'issue
Fonctionnalité
Clarté
Clairement spécifiée
Activité
Active
Stack technique
ruby
Domaine
ai-infra-agents, observability

Piste de recherche

Start by reading the four named instrumentation files: the Chat and Responses implementations for both openai and ruby-openai. Trace how each METADATA_FIELDS list and finalize_metadata method collects request and response data, then inspect the related tests to see how metadata is asserted. Done means inline moderation configuration and results are captured on both API spans across both integrations.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

<!-- provider-gap-audit: openai-inline-moderation-not-captured -->

Summary

OpenAI's Chat Completions and Responses APIs support an official moderation request parameter that runs moderation on the request input and generated output in the same call, returning the results in a moderation field on the response. This is distinct from the standalone moderations.create endpoint (which this SDK already instruments). Neither the request parameter nor the response field is captured anywhere in this repo's openai or ruby-openai instrumentation, for either Chat Completions or the Responses API.

What is missing

Request parameter
  • moderation: { model:, policy: { input: { mode: }, output: { mode: } } } — configures which moderation model to run and whether to just score or actively block input/output.
Response field
  • ChatCompletion#moderation — { input: ModerationResult|Error, output: ModerationResult|Error }
  • Response#moderation — same shape, on the Responses API's Response object

Neither field appears in any METADATA_FIELDS list, and neither finalize_metadata method reads the response's moderation attribute, so a moderated call's safety signals are silently dropped from the trace in all four affected instrumentation files.

Braintrust docs status

not_found — https://docs.braintrust.dev/integrations/ai-providers/openai documents a dedicated "Moderation" span type tied to moderations.create (and openai.moderations.create for Ruby) as a standalone endpoint, but has no mention of inline moderation results being captured as part of chat completion or Responses spans, for any language SDK.

Upstream sources

Local repo files inspected

  • lib/braintrust/contrib/openai/instrumentation/chat.rb (METADATA_FIELDS at lines 27–32; finalize_metadata at lines 129–136) — no moderation
  • lib/braintrust/contrib/openai/instrumentation/responses.rb (METADATA_FIELDS at lines 26–31; finalize_metadata at lines 109–112) — no moderation
  • lib/braintrust/contrib/ruby_openai/instrumentation/chat.rb (METADATA_FIELDS at lines 32–37; finalize_metadata at lines 137–143) — no moderation
  • lib/braintrust/contrib/ruby_openai/instrumentation/responses.rb (METADATA_FIELDS at lines 32–37; finalize_metadata at lines 137–140) — no moderation
  • lib/braintrust/contrib/openai/instrumentation/moderations.rb — confirms this SDK already treats moderation as a first-class signal worth tracing, just not when it arrives inline on a generation call
Langage dominant
Ruby
Étoiles
9
Forks
11
Merge moyen
1 j 11 h
PR mergées (30 j)
3

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de braintrustdata/braintrust-sdk-ruby

Toutes les issues de braintrustdata/braintrust-sdk-ruby

Issues similaires

Plus d'issues Ruby

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.