Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

[Bug]: reasoning_loop_recovery: recover never fires for a low or medium request (focused_recovery_prompt only matches the template's xhigh sentence)

Chiusa Adatta ai principianti
  • Qualcosa non va?
#2,906 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 3 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
78/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
ai, backend

Direzione di ricerca

Inizia da focused_recovery_prompt() e dal suo chiamante intorno a serve/server.py:196; lo script di riproduzione nell'issue mostra come esercitare entrambe le posizioni del prompt senza un engine. Aggiorna recovery per sostituire la frase di sforzo effettiva della richiesta, con un messaggio di skip chiaro quando nessuna frase corrisponde. Aggiungi o esegui test mirati che copiano il recovery con sforzo low, medium e xhigh, in modo che il ramo recover non restituisca più silenziosamente senza agire.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

What happened

With "reasoning_loop_recovery": "recover" in a model config, a reply whose thinking repeats whole passages is detected and then does nothing: the server logs the detection and the thinking carries on to reasoning_budget_tokens (or to max_tokens when no budget is set). The reply is not continued with the low-effort instruction, and nothing says why.

Measured on 0.1.44.1 and again on 0.1.45, a community qwen4exp fine-tune (RVN abliterated IQ3_S), reasoning_effort: low, one prompt that makes the model re-verify the same facts, max_tokens 8192, reasoning_budget_tokens 8192 then 4096:

run budget detection what happened next
1 8192 repeated reasoning detected at 3072 tokens (coverage=0.277) thought on to 6144, then the budget wrapped it up
2 8192 repeated reasoning detected at 3072 tokens (coverage=0.289) the same
3 4096 repeated reasoning detected at 3072 tokens (coverage=0.343) thought on to 4096, then the budget wrapped it up

So the detector works (it is the log line above); the recovery step is what never runs.

Why

focused_recovery_prompt() (serve/server.py:196 in 0.1.45) builds the spliced prompt by replacing the template's own xhigh sentence (HIGH_EFFORT, line 160) in the first system message with the low one, and returns None when that sentence is not there as whole tokens:

old, new = tok.encode(HIGH_EFFORT, parse_special=False), tok.encode(LOW_EFFORT, parse_special=False)
for i in range(len(head) - len(old) + 1):
    if head[i:i + len(old)] == old:
        return ids[:i] + new + ids[i + len(old):] + list(generated)
return None

With "effort_position": "start" (the default) a request that asks for low or medium is rendered with that level's sentence at the top of the first system message, so the xhigh sentence is absent and the function returns None. The caller only acts on a non-None result:

recover_prompt = focused_recovery_prompt(self.tok, ids, raw_ids)
if recover_prompt is not None:
    break

recover is therefore a silent no-op for every low/medium request — which is the common case as soon as a config sets "reasoning_effort": "low" (as mine does, to keep a local model from burning its output budget on thinking) or a client picks Low in its own selector. "reasoning_loop_recovery": "stop" is unaffected (it does not use this function and did end the reply in my runs).

How I checked it (no engine needed)

Rendering one request both ways with the server's own functions and then calling focused_recovery_prompt on it:

--- effort_position="start" (default)
    prompt tokens                : 63
    xhigh sentence in the prompt : False
    low   sentence in the prompt : True
    focused_recovery_prompt()    : None -> the recover branch cannot fire
--- effort_position="end"
    prompt tokens                : 105
    xhigh sentence in the prompt : True
    low   sentence in the prompt : True
    focused_recovery_prompt()    : 93 ids -> it can fire

Script: build the tokenizer and the ChatTemplate from a pack's tokenizer/ exactly as main() does, then

low_text  = tpl.render(msgs, reasoning_effort="low", add_generation_prompt=True)          # start
hist      = tpl.render(msgs, add_generation_prompt=False)                                 # end, default (xhigh)
full      = tpl.render(msgs, reasoning_effort="low", add_generation_prompt=True)
mine      = tpl.render(msgs, reasoning_effort="low", add_generation_prompt=False)
end_text  = hist + EFFORT_TURN.format(EFFORT_TEXT["low"]) + full[len(mine):]
for t in (low_text, end_text):
    print(focused_recovery_prompt(tok, tok.encode(t, parse_special=True), []))
Workaround I use now

"effort_position": "end" in the model config: the history is then rendered with the template's default (xhigh) and the real level follows in a short system turn before the answer, so the xhigh sentence is in the prompt and the recovery can splice it. With it, the detection line is followed by resuming the same output with the low-effort instruction (reasoning_loop_recovery) (measured twice, coverage 0.277 and 0.343).

Two honest notes on that: the nudge is not a cure — in both runs the model went on thinking for another ~1000-3000 tokens to the budget — and effort_position: "end" is a separate opt-in with its own behaviour (the engine checkpoints in front of the tail turn).

Suggested fix

Pass the request's level into focused_recovery_prompt() and look for that sentence (the template knows low, medium, xhigh; high maps to xhigh), or fall back to the low sentence when the level's own is not found. If neither is found, print one line saying the recovery was skipped and why, instead of failing silently — the current behaviour costs a full thinking budget with no hint.

Environment

0.1.44.1 and 0.1.45, prebuilt CUDA 13.0 engine (sm_75/86/89/120) · RTX 5080 16 GB · Windows 11 26200 · Core Ultra 7 270K Plus, 95 GB RAM. Config: reasoning_effort: low, reasoning_budget_tokens: 4096/8192, reasoning_loop_recovery: recover, reasoning_close_retry: true.

Not tested

AMD HIP and SYCL (CUDA only here); a high/xhigh request under effort_position: "start" (that one renders the xhigh sentence at the top, so the recovery should fire — I did not run it); the Anthropic route's output_config.effort path; and whether the model's repetition after the nudge would end on its own with a different prompt.

Lingua principale
C++
Stelle
11.6k
Fork
1k
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Niko1221/Strata

Tutte le issue di Niko1221/Strata

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.