Eval bug: reasoning loop guard force-closes legitimate reasoning early (looks like reasoning budget exhausted)

オープン
#202 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
45/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
活発
技術スタック
cpp

調査の方向性

Start at the /v1/chat/completions entry point and trace how reasoning_loop_guard, reasoning_loop_min_tokens, and the force-close message interact with reasoning-token accounting. Reproduce the reported early cut with the provided 8000-token budget and compare against guard-off behavior. Done means legitimate restatements do not trigger premature closure, actual budget limits still apply, and reasoning-token usage remains accurate when the guard is off.

索引モデルが issue の本文から書いたものです。

説明

Name and Version

llama-server version: 0.4.9 (build 12930, commit b46c53e54), Windows x64 CUDA 13.3 release binary, router mode.

Operating systems

Windows 11

GGML backends

CUDA

Hardware

RTX 4090 24 GB

Models

unsloth Qwen3.8-27B-GGUF, UD-Q4_K_XL (both the August upload and Dynamic v3.1). The same thing happens with UD-IQ4_XS. Settings: --spec-type draft-mtp --spec-draft-n-max 3, KV kvarn6, --reasoning-budget 8000, --reasoning-budget-message "... reasoning budget exceeded, need to answer.\n", reasoning_effort: medium.

Problem description & steps to reproduce

With the default --reasoning-loop-guard force-close, hidden reasoning is often force-closed well before the reasoning budget is reached. With a budget of 8000, we saw the budget message injected after only about 1.1K–1.9K reasoning tokens. Because the force-close uses the budget message, it looks like a broken budget counter. Budgets of 500 and 2000 are honoured exactly.

All the early cuts we looked at happen at the same point: the model re-states content it wrote earlier in the same reasoning, on purpose. In our case it re-copies a 3×3 OKLab matrix with 10-digit coefficients to double-check it. That is legitimate verification, not a degenerate loop. It is also a very predictable stretch, where MTP drafts get accepted almost entirely.

Test setup: one coding prompt (implement OKLab colour mixing, stdlib only, with stated edge cases), thinking_budget_tokens: 8000, non-streaming /v1/chat/completions, temp 1.0 / top_p 0.95 / top_k 20.

per-request setting early cut (budget message in reasoning) usage.completion_tokens_details.reasoning_tokens
default 3/6 (at 1100–1930 tokens) reported
"reasoning_loop_guard": "off" 0/6 always 0
"reasoning_loop_window": 2048, "reasoning_loop_min_coverage": 1024, "reasoning_loop_min_tokens": 2048 7/8, six of them at exactly 2060 tokens (the first check after min_tokens) reported
"reasoning_loop_min_tokens": 100000 0/6 (2.8K–5.4K tokens of reasoning, finishing normally) reported

An earlier run with the same prompt (budget 8000, 6 requests per model version) cut 4/6 on the August GGUF and 1/6 on v3.1. In a small coding benchmark, the share of tasks with all tests passing dropped from 97 % to 74 %, and every failure traced back to one of these early cuts.

Observations / possible issues:

  1. False positives on legitimate restatements. A single re-copy of a ~200–300-token block is enough to trigger. A larger window and a higher coverage threshold make it worse (7/8). That suggests the detector counts one repetition of a long span as "coverage", not a repetition that happens many times.
  2. Misleading message. The force-close injects the reasoning-budget message. From the client side, a loop-guard intervention can't be told apart from an exhausted budget. The loop_guard telemetry block did not appear in the OpenAI-compatible chat completion response, so we only found the cause by reading the source.
  3. reasoning_loop_guard: off also turns off reasoning-token accounting (reasoning_tokens = 0 in usage), apparently because reasoning_output_tokens is only updated in the loop-guard accept callback.

Suggestions:

  • Require several repetitions (period × count) before triggering, instead of a single restated span.
  • Use a separate message, or none, for loop-guard force-close, and expose the loop_guard event in chat-completion responses.
  • Keep reasoning-token accounting independent of the guard mode.

Workaround in use: reasoning-loop-min-tokens = 100000 in the preset. The reasoning budget still caps real loops.

First Bad Commit

Not bisected. The behaviour is present in v0.4.9.

Relevant log output

Example tail of reasoning_content at a cut (budget 8000, 1484 reasoning tokens):

r = 4.0767416621f * l - 3.3077115913f * m + 0.2309699292f * s
g = -... reasoning budget exceeded, need to answer.\n
主要言語
C++
スター
1.2k
フォーク
80
平均マージ
4日 20時間
マージ済み PR(30日)
2

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

Anbeeld/beellama.cpp のほかの issue

Anbeeld/beellama.cpp の issue をすべて見る

似ている issue

C++ の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。