Feature: OrcaRouter provider support for LettuceDetect

Aperta
  • Qualcosa non va?
#112 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
28/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
python
Ambito
ai

Direzione di ricerca

The provider switch lives in make_llm_client(), which accepts openai or bedrock and passes a base_url to the OpenAI-compatible client. Read that function and the LLMDetector class to see how a provider value is chosen and how api_key and base_url reach the client. Before writing code, the issue needs a maintainer decision on whether a new provider is wanted, since no one has replied yet. If it is accepted, done means a new provider value that builds the client with the existing OpenAI-compatible shape, plus a test that checks the client configuration.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

LettuceDetect tackles a hard problem: instead of a clean-or-flagged verdict on an AI answer, it returns the exact character spans that are unsupported by or contradict the source evidence, across RAG prose, coding-agent answers, and tool output. Span-level precision backed by fast local encoders and generative detectors that type each span is why teams building grounded systems rely on it - and your v2 results on code-agent answers, where the fine-tuned Qwen-2B detector beats large LLM judges that over-flag generated code, make a concrete case for domain-tuned grounding checks.

Your LLM-based detector already routes through an OpenAI-compatible client with a provider switch (openai / bedrock) and a base_url override, so an OrcaRouter provider would let teams using method="llm" judges point the same detector at whichever model best fits their cost, language, or throughput needs - more choice, no change to the LettuceDetect API.

Problem or use case

LLM judges are limited today to the providers hard-coded in make_llm_client(). Teams that want to run grounding checks against a different frontier model, or need one API surface across budgets, must wire their own gateway. I'd like to propose OrcaRouter as an optional provider for this detector.

Proposed solution

Treat OrcaRouter like any other provider value in the LLMDetector abstraction. OrcaRouter exposes an OpenAI-compatible API and uses standard API-key authentication, so it fits the existing OpenAIClient shape (api_key plus base_url) - no new protocol or SDK. A user would construct an LLM-based detector with OrcaRouter as the backend and keep the rest of the LettuceDetect API unchanged.

The most relevant OrcaRouter capabilities for your users are: access to many chat and reasoning models through one endpoint, automatic model routing with provider failover, prompt caching for repeated context/answer pairs, and usage tracking with budgets for evaluation pipelines.

I'm an engineer on the OrcaRouter team and would be glad to draft the implementation PR if maintainers are open to it - nothing here is implemented or tested yet; this issue is only to gauge fit.

Alternatives considered

Teams can already point the OpenAI client at any OpenAI-compatible gateway via base_url. OrcaRouter adds a managed routing layer with failover, caching, and per-team usage controls, plus an optional partner program for open-source projects.

Disclosure

OrcaRouter runs an optional open-source partner program: approved OSS projects can receive a 5% revenue share from OrcaRouter usage attributed to their integration. Participation is not a prerequisite, and I am happy to follow whatever disclosure or governance policy the project prefers. Existing integrations are listed at https://www.orcarouter.ai/built-with; RAGFlow, Dify, and promptfoo list OrcaRouter as an optional provider.

Maintainers: would an OrcaRouter provider be a welcome addition? If so, I will follow up with a focused implementation PR.

Lingua principale
Python
Stelle
615
Fork
60
Merge medio
1g 8h
PR unite (30g)
10

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di KRLabsOrg/LettuceDetect

Tutte le issue di KRLabsOrg/LettuceDetect

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.