llama.cpp backend: allow `--reasoning on|off` in runtime flag allowlist
Nobody has claimed this yet.
Assessment
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Newbie friendliness
- 90/100
Research direction
Start in pkg/inference/runtime_flags_allowlist.go, where the existing llama.cpp reasoning-related flags are defined. Compare the allowlist entries with the documented llama-server --reasoning on|off|auto flag, then verify that docker model configure -- --reasoning off is accepted without the previous rejection.
Written by the indexing model from the issue text.
Description
llama.cpp's llama-server (verified on build 9879 / 72874f559) exposes a first-class server flag:
--reasoning [on|off|auto] Use reasoning/thinking in the chat (default: auto = detect from template)
docker model configure rejects it:
$ docker model configure <model> -- --reasoning off
runtime flag "--reasoning" is not allowed for backend "llama.cpp"
It's missing from pkg/inference/runtime_flags_allowlist.go (which already allows --reasoning-format, --reasoning-budget, --reasoning-preserve, --chat-template-kwargs).
Why it matters: for hybrid thinking models (Qwen 3.5/4B family), the current way to pin a model non-thinking at the server level is --chat-template-kwargs '{"enable_thinking": false}'. On current builds this logs a deprecation warning at every server start:
W Setting 'enable_thinking' via --chat-template-kwargs is deprecated. Use --reasoning on / --reasoning off instead.
The flag is a simple enum, touches no files or external resources, and fits the existing allowlist security model (cf. #1032 for the reasoning-preserve flags).
Related: #1028 (request-body reasoning_effort: none), #1032 (reasoning-preserve flags).
- Dominant language
- Go
- Stars
- 651
- Forks
- 155
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from docker/model-runner
-
Difficulty 4/5 3-5 days Newbie friendliness 68/100
docker/model-runner#1063 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 42/100
docker/model-runner#1059 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
docker/model-runner#1058 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
docker/model-runner#1056 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
docker/model-runner#1054 ·
All issues in docker/model-runner
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 84/100
-
enhancement needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
kind/cleanup
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
kubernetes-sigs/kueue#15947 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
sympozium-ai/sympozium#627 ·
-
priority: p3
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
googleapis/librarian#7636 ·