Add EncoderDetector and EncoderConfig with detect spans

Aperta
  • Qualcosa non va?
#149 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
45/100
Tipo di issue
Funzionalità
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python, pytorch
Ambito
backend, machine-learning, testing

Direzione di ricerca

Start with lettucedetect/detectors/transformer.py:28-83 and the scoring methods at :84, :241, :341, and :383; run uv run --extra dev pytest tests/test_inference_pytest.py -q. Then review the #128 golden span output, the #143 contract suite, and the local_wordpiece_tokenizer fixture. Done means the new detector passes those contracts, configuration errors carry the specified codes and parameters, and the concurrency tests confirm per-instance serialization and a single lock under racing first calls.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

ready-for-agent

Parent: #148 · Epic: #126 · Blocked by: #128, #143
Branch from stateless_core; open the PR against it.

Problem

TransformerDetector.__init__ downloads the tokenizer and model inside the constructor, so a service cannot build a detector from objects it already holds, and every option is a keyword whose conflicts are checked at runtime.

Current behavior

  • lettucedetect/detectors/transformer.py:28-83 constructor: batch_size < 1 raises ValueError (:58), AutoTokenizer.from_pretrained (:62), AutoModelForTokenClassification.from_pretrained (:63).
  • Scoring: _predict_single (:241), _predict_chunked (:341, max() over passage groups), _group_passages_into_chunks (:84), _build_spans_from_tokens (:383).
  • Spans carry confidence; no score, usage or warnings.

Steps

  1. Add EncoderConfig (frozen pydantic, extra="forbid"): max_length=4096, batch_size=16, lang="en". Unknown option → ConfigurationError (invalid_config); unknown language → invalid_language; batch_size < 1 → invalid_batch_size; each with param.
  2. Add EncoderDetector(model, tokenizer, typer=None, *, config=None, model_id=None, model_version=None). It downloads nothing, reads no environment variable, writes no file.
  3. Implement _detect for SourcesInput and PromptInput by moving the scoring code (private method names stay) behind it. score is the maximum token probability of the span; text equals the answer between offsets.
  4. Plug into the #143 contract suite with a stub model and the local_wordpiece_tokenizer fixture.
  5. Guard the scoring path with one private threading.RLock: _detect takes it around tokenizing, the forward pass and aggregation, so concurrent detect and adetect calls on one instance run one at a time. Reason: transformers rewrites the fast tokenizer's truncation and padding settings on every call, so concurrent calls on one tokenizer can return wrong tokens or raise ValueError (reproduced locally by forcing thread switches; rare at default interpreter settings), and threads sharing one model gain little on a CPU or GPU. It is an RLock so that detect_batch (#177) can fall back to the single-request path while it holds the lock. The lock is created on first use by a private helper, not in __init__, because a detector built with __new__ and its attributes set by hand never runs __init__ (story 56); a module-level guard lock makes two racing first calls get the same lock. No config field and no public argument. The private scoring methods that existing tests patch don't take the lock themselves.
  6. Add the two concurrency tests below.

Acceptance criteria

  • Constructing with a stub model offline in an empty cwd succeeds and leaves it empty.
  • detect/adetect spans equal the #128 golden "spans" output for the same inputs, with score = old confidence; multi-group sources are aggregated with max() as today.
  • await adetect(...) equals detect for both request kinds (the model call runs in adetect's worker thread; no override).
  • asyncio.gather over at least 16 adetect calls with different inputs on one EncoderDetector (stub model, the real local_wordpiece_tokenizer fixture, inputs long enough to truncate and pad) returns each call's own result, equal to the same inputs run one by one with detect, and a stub model that records entries and exits shows no two scoring calls overlapping.
  • Racing first calls from several threads (released together by a barrier) on a detector built with __new__ and its attributes set by hand create exactly one lock.
  • Config errors carry the listed codes and param.
  • EncoderDetector passes the contract suite.
  • Existing tests pass; follows CONTRIBUTING "Errors and logging".

Start here

uv run --extra dev pytest tests/test_inference_pytest.py -q
Lingua principale
Python
Stelle
615
Fork
60
Merge medio
1g 8h
PR unite (30g)
10

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di KRLabsOrg/LettuceDetect

Tutte le issue di KRLabsOrg/LettuceDetect

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.