[FEATURE] Support content capture natively for OpenTelemetry
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 50/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- ruby
- Domain
- observability
Research direction
Start by reading RubyLLM's OpenTelemetry instrumentation and the linked GenAI semantic conventions, then trace where system instructions, message inputs, and model outputs are recorded. Define an opt-in setting, such as the proposed option or environment variable, and check that content is captured only when enabled; done means users can opt in to include this content in traces.
Written by the indexing model from the issue text.
Description
Scope check
- This is core LLM communication (not application logic)
- This benefits most users (not just my use case)
- This can't be solved in application code with current RubyLLM
- I read the Contributing Guide
Due diligence
- I searched existing issues
- I checked the documentation
What problem does this solve?
Thank you for adding native support to OpenTelemetry!
When using OpenTelemetry to instrument LLM content, you may want to include system instructions, message input and output into the traces - this could be useful for troubleshooting, tracing content in non-production environments or if you're running evals against traces in a third party platform.
OTel's GenAI semantic conventions describe:
[... about model instructions, user messages, and model outputs]
OpenTelemetry instrumentations SHOULD NOT capture them by default, but SHOULD
provide an option for users to opt in.
Proposed solution
We could offer an opt-in for users to capture content in specific scenarios, i.e.:
RubyLLM::OpenTelemetry.enable, capture_content: true
or via environment variable.
Why this belongs in RubyLLM
Since RubyLLM added support to OpenTelemetry following OTel's GenAI semantic conventions for spans and owns the interfaces for model inputs and outputs, it would make sense to add that support, too.
I'd be happy to open a PR to make that improvement.
- Dominant language
- Ruby
- Stars
- 4.5k
- Forks
- 523
- Avg merge
- 13h 36m
- Merged PRs (30d)
- 239
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
Maintainers usually reply within 1 day
-
L: javascript
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
dependabot/dependabot-core#16744 ·
Maintainers usually reply within 1 day
-
area:core-lib P2 size:S
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Maintainers usually reply within 1 day