microskills: a proposer reply with `"extractors": null` (or another wrongly typed field) aborts the fold, and the next session start pays for the proposer again

Aperta
  • Qualcosa non va?
#173 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
72/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python
Ambito
ai, backend

Direzione di ricerca

Start with parse_payload and _rx in tokeneconomizer/microskills/distiller.py, then read the existing coverage in tests/test_generic_microskills.py. Reproduce the malformed proposer replies described in the issue and check how Distiller.update and fold_locked handle them. Done means wrongly typed entries are safely ignored, invalid patterns do not abort folding, and tests cover these cases.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

What happens

parse_payload (tokeneconomizer/microskills/distiller.py) checks that the proposer's reply is JSON, but not the types inside it:

  • {"extractors": null, "facts": []} raises TypeError: 'NoneType' object is not iterable.
  • An extractor whose pattern is a list raises TypeError: unhashable type: 'list' in _rx, which only catches re.error.
  • A fact whose detector is a string raises AttributeError in _ground.

The guard added for #131 only covers the runner call. These errors come later, out of Distiller.update, so they abort fold_locked before folded.json or the ledger is written. The session stays pending, so the next session start folds it again and calls the proposer again.

Repro

Offline: six sessions through hook.main, with TE_MICROSKILLS=on, the default k = 3, and the proposer stubbed to answer {"extractors": null, "facts": []}.

paid proposer calls: 9        (2 expected)
folded.json: ['s0']
ledger records: 1
errors.log tail: TypeError: 'NoneType' object is not iterable

Why it matters

  • A model that writes null for "no extractors" makes every later session start call the proposer again. Each of those calls is billed, and none of them reach the ledger.
  • The envfacts layer, which makes no model calls, stops learning too.

A side note, not part of the fix

PROPOSER_SYSTEM asks for "ONLY a JSON list of <=4 such objects", and a few lines later for "a JSON OBJECT {extractors, facts}". A reply that follows the first instruction is dropped without any notice. The prompt is pinned by a hash in tests/test_generic_microskills.py, so I've left it alone.

I have a fix with tests ready: parse_payload keeps only entries with the expected types, and _rx returns None for a non-string pattern. Happy to open a PR if that approach works for you.

Lingua principale
Python
Stelle
5.6k
Fork
344
Merge medio
14h 41m
PR unite (30g)
54

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di microsoft/TokenEconomizer

Tutte le issue di microsoft/TokenEconomizer

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.