Hacktoberfest 2026: die Issues, die Maintainer für den Oktober markiert haben – offen und einsteigerfreundlich. Hacktoberfest-Issues durchsuchen

[bot] RubyLLM.rerank not instrumented

Offen
  • Stimmt etwas nicht?
#229 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
3/5
Geschätzter Aufwand
Ein halber Tag
Anfängerfreundlichkeit
62/100
Issue-Typ
Feature
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
ruby
Bereich
observability

Rechercherichtung

Start with patchers in lib/braintrust/contrib/ruby_llm/integration.rb (around line 49), which currently returns only ChatPatcher, and read patcher.rb and instrumentation/chat.rb to see how that patcher is structured. Add a rerank patcher and instrumentation alongside them that wraps RubyLLM::Rerank.rerank and records the query, document count, provider, model, top_n and returned scores. Done when a rerank call produces a span with those attributes.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

ruby

<!-- provider-gap-audit: ruby-llm-rerank-not-instrumented -->

Summary

ruby_llm ships a standalone reranking API, RubyLLM.rerank / RubyLLM::Rerank.rerank, available across multiple provider backends (Cohere, Bedrock, Vertex AI, Azure, OpenRouter, GPUStack). Like embeddings, it is a distinct, non-streaming model-inference call (query + candidate documents in, relevance-ordered results + scores out) outside Chat, and it is completely uninstrumented.

This is the same category of gap already tracked for RubyLLM.embed/.paint/.transcribe in #136, which did not cover reranking.

What is missing

The ruby_llm integration registers exactly one patcher (ChatPatcher; see lib/braintrust/contrib/ruby_llm/integration.rb:49), and lib/braintrust/contrib/ruby_llm/instrumentation/ contains only chat.rb and common.rb. There is no patcher for RubyLLM::Rerank.rerank.

RubyLLM.rerank(query, documents, model:, provider: nil, top_n: nil, ...) requires an explicit model: (no default, since rerank catalogs differ per provider) and returns a RubyLLM::Rerank::Result set with index, document, score per entry, plus model and raw provider response. Expected instrumentation: a span per rerank call capturing the query, document count, resolved provider/model, top_n, and the returned ordering/scores — mirroring the model/provider/usage capture already done for ChatPatcher.

Braintrust docs status

not_found — https://www.braintrust.dev/docs/integrations/sdk-integrations/ruby-llm documents chat completion spans, tool execution spans, token usage, streaming, and multimodal content for the ruby_llm integration. No mention of reranking anywhere on the page, and no embedding-specific surface is mentioned either (consistent with #136 covering that gap separately).

Upstream sources

Local repo files inspected

  • lib/braintrust/contrib/ruby_llm/integration.rb (line 49) — patchers returns only [ChatPatcher]
  • lib/braintrust/contrib/ruby_llm/patcher.rb — only defines ChatPatcher
  • lib/braintrust/contrib/ruby_llm/instrumentation/ — contains only chat.rb, common.rb; no rerank.rb
  • Grep for rerank/Rerank across lib/ — zero matches
  • Appraisals — ruby_llm "latest" constraint is >= 2.0, i.e. the version range this SDK already tests against includes rerank support
Vorherrschende Sprache
Ruby
Sterne
9
Forks
10
Ø Merge
3 T. 43 Min.
Gemergte PRs (30 T.)
9

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus braintrustdata/braintrust-sdk-ruby

Alle Issues in braintrustdata/braintrust-sdk-ruby

Ähnliche Issues

Weitere Issues zu Ruby

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.