Run many requests… | KRLabsOrg/LettuceDetect

Start with the batch result type and failure reporting decision referenced in #126, then review the detector contract and the blocked work in #148, #156, #162, #166, and #169. Implement and check the batch contract across each detector, including concurrency, encoder batching, and the predict_prompt_batch adapters. Done means the listed acceptance criteria and existing tests pass.

Parent

#126

Branch

Branch from stateless_core and open the pull request against it, not main. Close this ticket by hand once its pull request merges there: GitHub only closes issues automatically on merges into main.

What to build

A service developer sends a list of DetectRequest items to any detector and gets one result per request in input order, with the encoder scoring them in padded batches and the LLM-backed detectors running them concurrently under a limit the caller sets. One failed request doesn't throw away the results of the others, reported in the way the batch result decision defines.

The old predict_prompt_batch on every detector becomes an adapter over detect_batch. It keeps today's output, and it keeps raising on the first failure, as today.

Acceptance criteria

detect_batch(requests, *, min_score=0.0, include_tokens=False, include_detail=False) and adetect_batch exist on every detector, take any mix of SourcesInput and PromptInput, and return one result per request in input order. An empty sequence returns an empty result.
The result type and the way a failed request is reported are the ones the batch result decision defines. Whatever it defines, the results of requests that succeeded are returned alongside the failures.
EncoderDetector scores requests in padded groups of config.batch_size in one forward pass per group, and each result equals the result of a single detect call for the same request (including usage for that request).
Gathered adetect calls and adetect_batch calls on one EncoderDetector, single and padded-batch mixed, return the same results as the same requests run one by one (the encoder serialises scoring with its own lock, see task #148).
LLMJudgeDetector and GenerativeDetector run requests concurrently with at most config.max_concurrency client calls in flight per detector instance (max_concurrency defaults to 30), counted across detect, adetect, detect_batch and adetect_batch together, and each result equals the single detect result.
RAGFactCheckerDetector runs its requests through the same concurrent path, bounded by its max_concurrency keyword on concurrent detect calls.
include_tokens=True and include_detail=True raise InvalidInputError for the whole batch on detectors that don't support them, before any request runs.
adetect_batch is a coroutine function that runs detect_batch in a worker thread and returns an equal result (results, order, per-request failures). It is inherited by every detector, including those that override detect_batch.
adetect_batch leaves the event loop free while it waits. Its whole-batch errors (unsupported flags) come from the awaited call before any request runs, an empty sequence returns an empty result, and max_concurrency still bounds in-flight calls, also across gathered adetect_batch calls and a batch running next to single detect or adetect calls.
The detector contract suite gains a batch section that runs for every detector: order, empty input, equal to single calls, adetect_batch equal to detect_batch, and the failure reporting once decided.
predict_prompt_batch on TransformerDetector, LLMDetector and RAGFactCheckerDetector calls detect_batch and returns today's output (golden fixtures); a failed request raises the first error, as today.
The existing tests pass unchanged.
The changed code follows the Errors and logging section of CONTRIBUTING.md, and no customer text is logged outside lettucedetect.content.

Blocked by

#148, #156, #162, #166, #169, and the decision elsewhere on the batch result type and failure reporting (see Further Notes in #126).

User stories covered

35, 36, 37, 38, 40, 57

Run many requests with detect_batch and adetect_batch

Đánh giá

Hướng nghiên cứu

Mô tả

Parent

Branch

What to build

Acceptance criteria

Blocked by

User stories covered

Chuẩn bị môi trường

Bắt đầu từ đâu

Issue khác của KRLabsOrg/LettuceDetect

Issue tương tự

Run many requests with detect_batch and adetect_batch

Có gì không ổn?

Đánh giá

Hướng nghiên cứu

Mô tả

Parent

Branch

What to build

Acceptance criteria

Blocked by

User stories covered

Chuẩn bị môi trường

Bắt đầu từ đâu

Issue khác của KRLabsOrg/LettuceDetect

Có gì không ổn?

Có gì không ổn?

Có gì không ổn?

Có gì không ổn?

Có gì không ổn?

Issue tương tự

Có gì không ổn?

Có gì không ổn?

Có gì không ổn?

Có gì không ổn?

Có gì không ổn?

Nhận issue mới trong hộp thư của bạn