novelty: a capped run indexes spans and writes atoms.json from the sample only

Chiusa
  • Qualcosa non va?
#371 0 commenti 0 reazioni 1 assegnatario Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@antonwolfy ci sta già lavorando.

Dal 3/11/2026.

  • #380 di @antonwolfy — integrata

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
18/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
tooling

Direzione di ricerca

The fix spans extraction, span indexing and the atoms.json writer, and the payload names no concrete file or test. Start from where extract_atoms caps its output and where save_atoms writes atoms.json, then trace which code reads spans.capped_atom_set. Done would need a decision on the new atoms schema version first, which the issue leaves open.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

A capped novelty run extracts every span since #358 (decision D017), but extract_atoms returns only the spread sample of n atoms. So everything downstream of extraction sees only the sample:

  • index_atoms / span_summary
  • atoms.json (save_atoms)
  • novelty shorten

On a review host, 200 one-atom spans at cap 60 (--tier standard) give 60 atoms back. Of the spans that yield atoms, 140 keep atom_ids=[], and frac_carrying_nothing reads 0.70. Those spans count as carrying nothing and are never sole carriers, so shorten --out can cut them as free. Before #358 these spans were the unread tail of the input; now they are spread over the whole document. The extraction that found their atoms is already paid for, but novelty probe --atoms atoms.json --all-atoms, or a run with a larger cap, cannot reuse it.

Raised by @vlad-perevezentsev in https://github.com/intel/dokimi/pull/358#discussion_r4445936559.

What a fix takes:

  • Spans, attribution and shortening are indexed from all M atoms, while the oracle scores only the sample. Otherwise shortening's oracle pairs grow with M instead of with the cap; build_candidates would need to split structural atoms from scored ones.
  • atoms.json records which atoms are the sample, so novelty probe / novelty shorten probe only those by default and --all-atoms can reuse the rest. That is a new atoms schema version if the change rejects files already written.
  • spans.capped_atom_set gets a new meaning, and the novelty gate's dead-token and harmful_atoms checks read it.
  • Runs that index every span drop the frac_carrying_nothing and shortening forfeits. The docs (skill-novelty.md, D017) and the settings-parity fixture change with them.

Until then, the shortening and frac_carrying_nothing forfeits on a sampled run say what it cannot claim.

Lingua principale
Python
Stelle
11
Fork
9
Merge medio
1g 11h
PR unite (30g)
318

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di intel/dokimi

Tutte le issue di intel/dokimi

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.