[bot] OpenAI inline moderation (`moderation` param/field) not captured on Chat Completions or Responses API spans

未关闭
  • 哪里不对？
#235 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

评估

难度
3/5
预计耗时
1-2 天
新手友好度
64/100
Issue 类型
功能
描述清晰度
描述清楚
活跃度
活跃
技术栈
ruby
领域
ai-infra-agents, observability

调研方向

Start by reading the four named instrumentation files: the Chat and Responses implementations for both openai and ruby-openai. Trace how each METADATA_FIELDS list and finalize_metadata method collects request and response data, then inspect the related tests to see how metadata is asserted. Done means inline moderation configuration and results are captured on both API spans across both integrations.

由索引模型根据 Issue 内容生成。

描述

<!-- provider-gap-audit: openai-inline-moderation-not-captured -->

Summary

OpenAI's Chat Completions and Responses APIs support an official moderation request parameter that runs moderation on the request input and generated output in the same call, returning the results in a moderation field on the response. This is distinct from the standalone moderations.create endpoint (which this SDK already instruments). Neither the request parameter nor the response field is captured anywhere in this repo's openai or ruby-openai instrumentation, for either Chat Completions or the Responses API.

What is missing

Request parameter
  • moderation: { model:, policy: { input: { mode: }, output: { mode: } } } — configures which moderation model to run and whether to just score or actively block input/output.
Response field
  • ChatCompletion#moderation — { input: ModerationResult|Error, output: ModerationResult|Error }
  • Response#moderation — same shape, on the Responses API's Response object

Neither field appears in any METADATA_FIELDS list, and neither finalize_metadata method reads the response's moderation attribute, so a moderated call's safety signals are silently dropped from the trace in all four affected instrumentation files.

Braintrust docs status

not_found — https://docs.braintrust.dev/integrations/ai-providers/openai documents a dedicated "Moderation" span type tied to moderations.create (and openai.moderations.create for Ruby) as a standalone endpoint, but has no mention of inline moderation results being captured as part of chat completion or Responses spans, for any language SDK.

Upstream sources

Local repo files inspected

  • lib/braintrust/contrib/openai/instrumentation/chat.rb (METADATA_FIELDS at lines 27–32; finalize_metadata at lines 129–136) — no moderation
  • lib/braintrust/contrib/openai/instrumentation/responses.rb (METADATA_FIELDS at lines 26–31; finalize_metadata at lines 109–112) — no moderation
  • lib/braintrust/contrib/ruby_openai/instrumentation/chat.rb (METADATA_FIELDS at lines 32–37; finalize_metadata at lines 137–143) — no moderation
  • lib/braintrust/contrib/ruby_openai/instrumentation/responses.rb (METADATA_FIELDS at lines 32–37; finalize_metadata at lines 137–140) — no moderation
  • lib/braintrust/contrib/openai/instrumentation/moderations.rb — confirms this SDK already treats moderation as a first-class signal worth tracing, just not when it arrives inline on a generation call
主要语言
Ruby
星标
9
派生
11
平均合并
1 天 11 小时
30 天内合并 PR
3

环境准备

  • 提供 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

braintrustdata/braintrust-sdk-ruby 的其他 Issue

查看 braintrustdata/braintrust-sdk-ruby 的全部 Issue

相似的 Issue

更多 Ruby Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。