Run many requests with detect_batch and adetect_batch

Abierto
  • ¿Algo no está bien?
#175 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
30/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python, pytorch
Área
backend, machine-learning

Línea de trabajo

Start with the batch result type and failure reporting decision referenced in #126, then review the detector contract and the blocked work in #148, #156, #162, #166, and #169. Implement and check the batch contract across each detector, including concurrency, encoder batching, and the predict_prompt_batch adapters. Done means the listed acceptance criteria and existing tests pass.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

ready-for-agent

Parent

#126

Branch

Branch from stateless_core and open the pull request against it, not main. Close this ticket by hand once its pull request merges there: GitHub only closes issues automatically on merges into main.

What to build

A service developer sends a list of DetectRequest items to any detector and gets one result per request in input order, with the encoder scoring them in padded batches and the LLM-backed detectors running them concurrently under a limit the caller sets. One failed request doesn't throw away the results of the others, reported in the way the batch result decision defines.

The old predict_prompt_batch on every detector becomes an adapter over detect_batch. It keeps today's output, and it keeps raising on the first failure, as today.

Acceptance criteria

  • detect_batch(requests, *, min_score=0.0, include_tokens=False, include_detail=False) and adetect_batch exist on every detector, take any mix of SourcesInput and PromptInput, and return one result per request in input order. An empty sequence returns an empty result.
  • The result type and the way a failed request is reported are the ones the batch result decision defines. Whatever it defines, the results of requests that succeeded are returned alongside the failures.
  • EncoderDetector scores requests in padded groups of config.batch_size in one forward pass per group, and each result equals the result of a single detect call for the same request (including usage for that request).
  • Gathered adetect calls and adetect_batch calls on one EncoderDetector, single and padded-batch mixed, return the same results as the same requests run one by one (the encoder serialises scoring with its own lock, see task #148).
  • LLMJudgeDetector and GenerativeDetector run requests concurrently with at most config.max_concurrency client calls in flight per detector instance (max_concurrency defaults to 30), counted across detect, adetect, detect_batch and adetect_batch together, and each result equals the single detect result.
  • RAGFactCheckerDetector runs its requests through the same concurrent path, bounded by its max_concurrency keyword on concurrent detect calls.
  • include_tokens=True and include_detail=True raise InvalidInputError for the whole batch on detectors that don't support them, before any request runs.
  • adetect_batch is a coroutine function that runs detect_batch in a worker thread and returns an equal result (results, order, per-request failures). It is inherited by every detector, including those that override detect_batch.
  • adetect_batch leaves the event loop free while it waits. Its whole-batch errors (unsupported flags) come from the awaited call before any request runs, an empty sequence returns an empty result, and max_concurrency still bounds in-flight calls, also across gathered adetect_batch calls and a batch running next to single detect or adetect calls.
  • The detector contract suite gains a batch section that runs for every detector: order, empty input, equal to single calls, adetect_batch equal to detect_batch, and the failure reporting once decided.
  • predict_prompt_batch on TransformerDetector, LLMDetector and RAGFactCheckerDetector calls detect_batch and returns today's output (golden fixtures); a failed request raises the first error, as today.
  • The existing tests pass unchanged.
  • The changed code follows the Errors and logging section of CONTRIBUTING.md, and no customer text is logged outside lettucedetect.content.

Blocked by

#148, #156, #162, #166, #169, and the decision elsewhere on the batch result type and failure reporting (see Further Notes in #126).

User stories covered

35, 36, 37, 38, 40, 57

Lenguaje dominante
Python
Estrellas
615
Forks
60
Merge medio
1 d 8 h
PR fusionados (30 d)
10

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de KRLabsOrg/LettuceDetect

Todos los issues de KRLabsOrg/LettuceDetect

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.