Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

[bot] RubyLLM.rerank not instrumented

Aperta
  • Qualcosa non va?
#229 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
Mezza giornata
Idoneità per principianti
62/100
Tipo di issue
Funzionalità
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
ruby
Ambito
observability

Direzione di ricerca

Start with patchers in lib/braintrust/contrib/ruby_llm/integration.rb (around line 49), which currently returns only ChatPatcher, and read patcher.rb and instrumentation/chat.rb to see how that patcher is structured. Add a rerank patcher and instrumentation alongside them that wraps RubyLLM::Rerank.rerank and records the query, document count, provider, model, top_n and returned scores. Done when a rerank call produces a span with those attributes.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

ruby

<!-- provider-gap-audit: ruby-llm-rerank-not-instrumented -->

Summary

ruby_llm ships a standalone reranking API, RubyLLM.rerank / RubyLLM::Rerank.rerank, available across multiple provider backends (Cohere, Bedrock, Vertex AI, Azure, OpenRouter, GPUStack). Like embeddings, it is a distinct, non-streaming model-inference call (query + candidate documents in, relevance-ordered results + scores out) outside Chat, and it is completely uninstrumented.

This is the same category of gap already tracked for RubyLLM.embed/.paint/.transcribe in #136, which did not cover reranking.

What is missing

The ruby_llm integration registers exactly one patcher (ChatPatcher; see lib/braintrust/contrib/ruby_llm/integration.rb:49), and lib/braintrust/contrib/ruby_llm/instrumentation/ contains only chat.rb and common.rb. There is no patcher for RubyLLM::Rerank.rerank.

RubyLLM.rerank(query, documents, model:, provider: nil, top_n: nil, ...) requires an explicit model: (no default, since rerank catalogs differ per provider) and returns a RubyLLM::Rerank::Result set with index, document, score per entry, plus model and raw provider response. Expected instrumentation: a span per rerank call capturing the query, document count, resolved provider/model, top_n, and the returned ordering/scores — mirroring the model/provider/usage capture already done for ChatPatcher.

Braintrust docs status

not_found — https://www.braintrust.dev/docs/integrations/sdk-integrations/ruby-llm documents chat completion spans, tool execution spans, token usage, streaming, and multimodal content for the ruby_llm integration. No mention of reranking anywhere on the page, and no embedding-specific surface is mentioned either (consistent with #136 covering that gap separately).

Upstream sources

Local repo files inspected

  • lib/braintrust/contrib/ruby_llm/integration.rb (line 49) — patchers returns only [ChatPatcher]
  • lib/braintrust/contrib/ruby_llm/patcher.rb — only defines ChatPatcher
  • lib/braintrust/contrib/ruby_llm/instrumentation/ — contains only chat.rb, common.rb; no rerank.rb
  • Grep for rerank/Rerank across lib/ — zero matches
  • Appraisals — ruby_llm "latest" constraint is >= 2.0, i.e. the version range this SDK already tests against includes rerank support
Lingua principale
Ruby
Stelle
9
Fork
10
Merge medio
3g 43m
PR unite (30g)
9

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di braintrustdata/braintrust-sdk-ruby

Tutte le issue di braintrustdata/braintrust-sdk-ruby

Issue simili

Altre issue su Ruby

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.