Type spans in EncoderDetector.detect and meter the typer

クローズ
  • 何か問題がありますか？
#155 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
54/100
issue の種類
機能追加
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
python, pytorch
領域
backend-api-design, machine-learning

調査の方向性

Start with EncoderDetector._detect, EncoderDetector.adetect, EncoderDetector.from_pretrained, and TransformerDetector; the issue also points to taxonomy_head.py:96 and transformer.py:383. Run the listed pytest command first, then trace how spans and usage are built and inspect the typed golden fixtures from #128. Done when typed and untyped outputs, both usage entries, and the stated test suites match the acceptance criteria.

索引モデルが issue の本文から書いたものです。

説明

ready-for-agent

Parent: #153 · Epic: #126 · Blocked by: #154, #150
Branch from stateless_core; open the PR against it.

Problem

A typed call is not metered completely and typed spans don't exist in Detection: the typer's work is invisible in usage, and category/subcategory appear only in the old dicts.

Current behavior

  • Typing happens in the old predict path via TaxonomyTyper.type_spans (taxonomy_head.py:96) after _build_spans_from_tokens (transformer.py:383); context is the sources joined by newline (or the prompt).
  • Detection has no typer usage entry.

Steps

  1. In EncoderDetector._detect, when a typer is set, call it for the spans and fill DetectedSpan.category and subcategory; without a typer both are None. Context: sources joined by "\n" for SourcesInput, the prompt for PromptInput.
  2. Add a second Usage for the typer: its model_id, model_version, unit spans (0 when there are no spans).
  3. adetect on a typed EncoderDetector needs no override: it runs detect, including the typer call, in one worker thread, so the typed result is the same as the sync one.
  4. EncoderDetector.from_pretrained(taxonomy_head=..., include_taxonomy=...) builds the typer through TaxonomyTyper.from_pretrained, all four include_taxonomy forms.
  5. TransformerDetector(taxonomy_head=...) still types spans; output equals the typed golden fixtures from #128.

Acceptance criteria

  • With a stub typer every span has category and subcategory; without one both None.
  • await adetect(...) on a typed EncoderDetector equals detect: same category/subcategory on every span and the same two usage entries; the loop stays free while it waits.
  • Two usage entries with a typer, one without; typer entry reports spans = 0 when there are none.
  • Typed golden fixtures and tests/test_custom_taxonomy_pytest.py pass unchanged.
  • Contract suite passes for a typed EncoderDetector; follows CONTRIBUTING "Errors and logging".

Start here

uv run --extra dev pytest tests/test_custom_taxonomy_pytest.py tests/test_golden_transformer_pytest.py -q
主要言語
Python
スター
615
フォーク
60
平均マージ
2日 18時間
マージ済み PR（30日）
12

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

KRLabsOrg/LettuceDetect のほかの issue

KRLabsOrg/LettuceDetect の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。