Make TransformerDetector a deprecated subclass of EncoderDetector

Aperta
  • Qualcosa non va?
#152 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
54/100
Tipo di issue
Refactoring
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python, pytorch
Ambito
backend, machine-learning

Direzione di ricerca

Start by reading lettucedetect/detectors/transformer.py and the EncoderDetector implementation, then run the listed pytest command. Check how TransformerDetector construction, prediction, and __new__ callers behave, including the stated golden files and threaded case. Done means the acceptance criteria pass, including the deprecation warning behavior and the documented deprecation note.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

ready-for-agent

Parent: #148 · Epic: #126 · Blocked by: #149, #150, #151
Branch from stateless_core; open the PR against it.

Problem

TransformerDetector and EncoderDetector must be one implementation. Old callers (including HallucinationDetector and an evaluation script that uses __new__) must keep working unchanged.

Current behavior

  • lettucedetect/detectors/transformer.py:19 TransformerDetector(BaseDetector) with constructor :28 taking model_path, max_length, device, lang, taxonomy_head, include_taxonomy, batch_size, **tok_kwargs.
  • predict (:449), predict_prompt (:489) and predict_prompt_batch (:528) hold the scoring and conversion logic.
  • An evaluation script builds one with __new__ and sets model, tokenizer, device, max_length, lang, typer by hand.

Steps

  1. Change TransformerDetector to subclass EncoderDetector; its constructor calls from_pretrained logic and keeps attributes model, tokenizer, device, max_length, lang, batch_size, typer and the private method names (_predict_single, _predict_batch, _predict_chunked, …) that existing tests patch.
  2. Mark it deprecated (helper from #135) naming EncoderDetector.from_pretrained; in-package calls stay silent.
  3. predict and predict_prompt delegate through detect; predict_prompt_batch stays as is until #178.
  4. Keep **tok_kwargs.
  5. TransformerDetector exposes detect and adetect by inheriting them from EncoderDetector; they take the same arguments, support include_tokens, and return the same Detection as an EncoderDetector built from the same model. Calling them emits no deprecation warning (the warning is on construction). Its model_id is model_path.

Acceptance criteria

  • TransformerDetector(model_path=...) from a test warns LettuceDetectDeprecationWarning; from inside the package it doesn't.
  • predict/predict_prompt output equals the #128 golden files.
  • detect and adetect on a TransformerDetector return a result equal to an EncoderDetector built from the same stub model, with include_tokens=True working, and calling them adds no warning beyond the one from construction.
  • A TransformerDetector made with __new__ and only the six attributes set still answers predict with spans.
  • A TransformerDetector made with __new__ answers predict from several threads at once with the right results (the lock from #149 is created on first use).
  • Existing tests pass unchanged; docstring has the .. deprecated:: note.

Start here

uv run --extra dev pytest tests/test_inference_pytest.py tests/test_custom_taxonomy_pytest.py tests/test_golden_transformer_pytest.py -q
Lingua principale
Python
Stelle
615
Fork
60
Merge medio
1g 8h
PR unite (30g)
10

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di KRLabsOrg/LettuceDetect

Tutte le issue di KRLabsOrg/LettuceDetect

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.