Local AI: Effort Off doesn't stop llama-server thinking, and newer gateways return HTTP 500 for unsupported levels

Open
  • Is something wrong?
#1,976 1 comment 1 reaction 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
18/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
csharp, ollama
Domain
api, backend

Research direction

Start with LlamaServerRouterConfiguration.cs near line 295, where the llama-server preset sets reasoning = on, and LocalAiGatewayProviderDefinition.BuildProviderJson for the missing compat.supportsReasoningEffort fields. Also check GatewayActionTransport.cs near line 122, which sends thinking: "low" on canvas actions. Done means Effort Off reaches llama-server as reasoning_effort none, per-model level hiding is declared, and canvas actions stop changing the session effort, verified against the gateway configuration writes.

Written by the indexing model from the issue text.

Description

bug clawsweeper:needs-live-repro clawsweeper:needs-product-decision clawsweeper:no-new-fix-pr impact:auth-provider impact:session-state issue-rating: 🐚 platinum hermit local-models P2

Summary

Reasoning control is broken for Local AI (the llama-server the app launches and registers as the llamacpp provider):

  1. Effort Off / /think off does not stop thinking. The model keeps reasoning; the gateway only hides the output, so Off still costs the full thinking time.
  2. Newer gateways can fail outright. Gateway 2026.10.5 beta sends reasoning_effort to local OpenAI-compatible providers by default. With the app's current provider entry, /think minimal on Qwen3.8 returns HTTP 500 from llama-server.

Root cause

  • LlamaServerRouterConfiguration.cs:295 writes reasoning = on into the llama-server preset. With --reasoning on, llama.cpp ignores reasoning_effort: "none", so the per-request off switch has no effect. This is an upstream llama.cpp bug (still on master); under the default --reasoning auto, "none" turns thinking off for every template in the Local AI catalog.
  • LocalAiGatewayProviderDefinition.BuildProviderJson declares reasoning = true but no compat.supportsReasoningEffort / supportedReasoningEfforts. The gateway therefore either sends nothing the server honors (older gateways) or sends levels the model template does not support, such as minimal (2026.10.5 beta).
  • GatewayActionTransport.cs:122 sends thinking: "low" with every canvas (A2UI) action. The gateway persists that level on the chat session, so one canvas action silently switches the user's Effort to Low and turns Off back into thinking.

Proposed fix (standard reasoning_effort, not model-specific)

Use OpenAI's reasoning_effort field, which llama-server implements and the gateway already supports per model, with Off sent as "none":

  1. Drop reasoning = on from the llama-server preset (fall back to the default auto).
  2. Declare per-model effort support in the provider entry, sourced from the model catalog for every entry:
    • Qwen3.8: supportedReasoningEfforts: [none, low, medium, high], hide Minimal (its template has low / medium / xhigh, and accepts high as xhigh).
    • Qwen3.6 and Qwen3.5: [none, medium], hide Minimal and High (on/off only).
    • "none" must be listed, or the gateway drops Off.
    • Do not hide Low: the gateway rejects an explicitly requested hidden level rather than clamping it, and paired phones request Low.
  3. Stop canvas actions from sending thinking: "low".

Resulting pickers: Off / Low / Medium / High / Ultra for Qwen3.8, Off / Low / Medium / Ultra for Qwen3.6 (Ultra is a gateway mode and cannot be hidden).

Both app config write paths (config set --batch-file and config.patch) preserve the hidden-level settings and pass the exact readback check. Existing installs need a provider-entry migration so an upgrade does not hit the "changed outside the companion" repair state.

Trade-offs to decide

  • Default effort drops for Qwen3.8 from xhigh to medium (the gateway default). In return, switching between Medium and Off reuses the prompt cache instead of re-prefilling the agent prompt (about 21 s on an RTX Spark), because the Qwen3.8 template writes the effort level at the top of the system prompt.
  • Gateway floor rises to v2026.6.5, since older gateways reject the level-hiding setting. Only native-gateway installs are affected; WSL setup installs the latest gateway.
  • Phones can still switch Effort to Low through voice transcripts and the iOS share extension. That needs an upstream gateway change.

Validation so far

Proven end to end through real gateways 2026.9.8, 2026.10.2 and 2026.10.5 beta on Linux with an RTX 4090, using a small stand-in model loaded with the real Qwen chat templates.

Not yet tested: the real catalog models, Windows, the app's own Effort UI and canvas path, and llama-server router mode under auto.

Dominant language
C#
Stars
2.2k
Forks
336
Avg merge
3d 20h
Merged PRs (30d)
163

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from openclaw/openclaw-windows-node

All issues in openclaw/openclaw-windows-node

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.