Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

[bot] RubyLLM.rerank not instrumented

Abierto
  • ¿Algo no está bien?
#229 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
3/5
Tiempo estimado
Medio día
Aptitud para principiantes
62/100
Tipo de issue
Nueva funcionalidad
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
ruby
Área
observability

Línea de trabajo

Start with patchers in lib/braintrust/contrib/ruby_llm/integration.rb (around line 49), which currently returns only ChatPatcher, and read patcher.rb and instrumentation/chat.rb to see how that patcher is structured. Add a rerank patcher and instrumentation alongside them that wraps RubyLLM::Rerank.rerank and records the query, document count, provider, model, top_n and returned scores. Done when a rerank call produces a span with those attributes.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

ruby

<!-- provider-gap-audit: ruby-llm-rerank-not-instrumented -->

Summary

ruby_llm ships a standalone reranking API, RubyLLM.rerank / RubyLLM::Rerank.rerank, available across multiple provider backends (Cohere, Bedrock, Vertex AI, Azure, OpenRouter, GPUStack). Like embeddings, it is a distinct, non-streaming model-inference call (query + candidate documents in, relevance-ordered results + scores out) outside Chat, and it is completely uninstrumented.

This is the same category of gap already tracked for RubyLLM.embed/.paint/.transcribe in #136, which did not cover reranking.

What is missing

The ruby_llm integration registers exactly one patcher (ChatPatcher; see lib/braintrust/contrib/ruby_llm/integration.rb:49), and lib/braintrust/contrib/ruby_llm/instrumentation/ contains only chat.rb and common.rb. There is no patcher for RubyLLM::Rerank.rerank.

RubyLLM.rerank(query, documents, model:, provider: nil, top_n: nil, ...) requires an explicit model: (no default, since rerank catalogs differ per provider) and returns a RubyLLM::Rerank::Result set with index, document, score per entry, plus model and raw provider response. Expected instrumentation: a span per rerank call capturing the query, document count, resolved provider/model, top_n, and the returned ordering/scores — mirroring the model/provider/usage capture already done for ChatPatcher.

Braintrust docs status

not_found — https://www.braintrust.dev/docs/integrations/sdk-integrations/ruby-llm documents chat completion spans, tool execution spans, token usage, streaming, and multimodal content for the ruby_llm integration. No mention of reranking anywhere on the page, and no embedding-specific surface is mentioned either (consistent with #136 covering that gap separately).

Upstream sources

Local repo files inspected

  • lib/braintrust/contrib/ruby_llm/integration.rb (line 49) — patchers returns only [ChatPatcher]
  • lib/braintrust/contrib/ruby_llm/patcher.rb — only defines ChatPatcher
  • lib/braintrust/contrib/ruby_llm/instrumentation/ — contains only chat.rb, common.rb; no rerank.rb
  • Grep for rerank/Rerank across lib/ — zero matches
  • Appraisals — ruby_llm "latest" constraint is >= 2.0, i.e. the version range this SDK already tests against includes rerank support
Lenguaje dominante
Ruby
Estrellas
9
Forks
11
Merge medio
3 d 43 min
PR fusionados (30 d)
9

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de braintrustdata/braintrust-sdk-ruby

Todos los issues de braintrustdata/braintrust-sdk-ruby

Issues similares

Más issues de Ruby

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.