Local AI: Effort Off doesn't stop llama-server thinking, and newer gateways return HTTP 500 for unsupported levels

Aperta
  • Qualcosa non va?
#1,976 1 commento 1 reazione 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@turutupa ci sta già lavorando.

Dal 4/11/2026.

  • #2012 di @turutupa — aperta

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
18/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
csharp, ollama
Ambito
api, backend

Direzione di ricerca

Start with LlamaServerRouterConfiguration.cs near line 295, where the llama-server preset sets reasoning = on, and LocalAiGatewayProviderDefinition.BuildProviderJson for the missing compat.supportsReasoningEffort fields. Also check GatewayActionTransport.cs near line 122, which sends thinking: "low" on canvas actions. Done means Effort Off reaches llama-server as reasoning_effort none, per-model level hiding is declared, and canvas actions stop changing the session effort, verified against the gateway configuration writes.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug clawsweeper:needs-live-repro clawsweeper:needs-product-decision clawsweeper:no-new-fix-pr impact:auth-provider impact:session-state issue-rating: 🐚 platinum hermit local-models P2

Summary

Reasoning control is broken for Local AI (the llama-server the app launches and registers as the llamacpp provider):

  1. Effort Off / /think off does not stop thinking. The model keeps reasoning; the gateway only hides the output, so Off still costs the full thinking time.
  2. Newer gateways can fail outright. Gateway 2026.10.5 beta sends reasoning_effort to local OpenAI-compatible providers by default. With the app's current provider entry, /think minimal on Qwen3.8 returns HTTP 500 from llama-server.

Root cause

  • LlamaServerRouterConfiguration.cs:295 writes reasoning = on into the llama-server preset. With --reasoning on, llama.cpp ignores reasoning_effort: "none", so the per-request off switch has no effect. This is an upstream llama.cpp bug (still on master); under the default --reasoning auto, "none" turns thinking off for every template in the Local AI catalog.
  • LocalAiGatewayProviderDefinition.BuildProviderJson declares reasoning = true but no compat.supportsReasoningEffort / supportedReasoningEfforts. The gateway therefore either sends nothing the server honors (older gateways) or sends levels the model template does not support, such as minimal (2026.10.5 beta).
  • GatewayActionTransport.cs:122 sends thinking: "low" with every canvas (A2UI) action. The gateway persists that level on the chat session, so one canvas action silently switches the user's Effort to Low and turns Off back into thinking.

Proposed fix (standard reasoning_effort, not model-specific)

Use OpenAI's reasoning_effort field, which llama-server implements and the gateway already supports per model, with Off sent as "none":

  1. Drop reasoning = on from the llama-server preset (fall back to the default auto).
  2. Declare per-model effort support in the provider entry, sourced from the model catalog for every entry:
    • Qwen3.8: supportedReasoningEfforts: [none, low, medium, high], hide Minimal (its template has low / medium / xhigh, and accepts high as xhigh).
    • Qwen3.6 and Qwen3.5: [none, medium], hide Minimal and High (on/off only).
    • "none" must be listed, or the gateway drops Off.
    • Do not hide Low: the gateway rejects an explicitly requested hidden level rather than clamping it, and paired phones request Low.
  3. Stop canvas actions from sending thinking: "low".

Resulting pickers: Off / Low / Medium / High / Ultra for Qwen3.8, Off / Low / Medium / Ultra for Qwen3.6 (Ultra is a gateway mode and cannot be hidden).

Both app config write paths (config set --batch-file and config.patch) preserve the hidden-level settings and pass the exact readback check. Existing installs need a provider-entry migration so an upgrade does not hit the "changed outside the companion" repair state.

Trade-offs to decide

  • Default effort drops for Qwen3.8 from xhigh to medium (the gateway default). In return, switching between Medium and Off reuses the prompt cache instead of re-prefilling the agent prompt (about 21 s on an RTX Spark), because the Qwen3.8 template writes the effort level at the top of the system prompt.
  • Gateway floor rises to v2026.6.5, since older gateways reject the level-hiding setting. Only native-gateway installs are affected; WSL setup installs the latest gateway.
  • Phones can still switch Effort to Low through voice transcripts and the iOS share extension. That needs an upstream gateway change.

Validation so far

Proven end to end through real gateways 2026.9.8, 2026.10.2 and 2026.10.5 beta on Linux with an RTX 4090, using a small stand-in model loaded with the real Qwen chat templates.

Not yet tested: the real catalog models, Windows, the app's own Effort UI and canvas path, and llama-server router mode under auto.

Lingua principale
C#
Stelle
2.2k
Fork
336
Merge medio
3g 11h
PR unite (30g)
166

Preparare l'ambiente

  • Nessun Dockerfile né file Docker Compose
  • Ha un modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di openclaw/openclaw-windows-node

Tutte le issue di openclaw/openclaw-windows-node

Issue simili

Altre issue su C#

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.