[Engine] Tool-call arguments ignore the schema type: booleans come back as the strings "True"/"False"
メンテナーはふだん 4 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 2/5
- 見積もり時間
- 1〜3時間
- 初心者へのやさしさ
- 76/100
調査の方向性
serve/frontend.pyを読み、issueで変換経路の候補として挙げられているcoerce_value(value, declared)を調べてください。json.loadsへのフォールバックより前に、宣言されたブール値がどのように処理されるかを確認し、関連するツール呼び出しテストを実行してください。issueにはテストファイルの指定がありません。完了条件は、モデルがTrueまたはFalseと記述した場合、ツール引数がJSONのブール値として出力され、宣言された文字列は文字列のままであることです。
索引モデルが issue の本文から書いたものです。
説明
[Engine] Tool-call arguments ignore the schema type: booleans come back as the strings "True"/"False"
Area: [Engine] (the server and API) · How: Docker · Reproduction status: Reproduced locally
(0.2.7 / 0.2.9 / 0.2.11; the same code path on current main is read, not run — see Evidence)
Problem
For a declared tool whose parameter is {"type": "boolean"}, the OpenAI-compatible response carries the
argument as a string: "formatted": "True" instead of "formatted": true. The model's chosen value is
right; only the JSON type is wrong.
Environment
Docker ghcr.io/niko1221/strata:0.2.7-cuda13 (engine 0.1.52) — still present in 0.2.9 (engine 0.1.55) and
0.2.11 (engine 0.1.57). Our image is our own build of the upstream release tag (buildah, upstream Dockerfile
unchanged, the release's signed cuda13 engine package); same code as upstream's image.
Linux · RTX PRO 5000 48 GB, one card (the 2026-11-10 capture below; the 0.2.7 and 0.2.9 observations were on
two cards: the same 48 GB plus an RTX 5060 Ti 16 GB) · 62.7 GB RAM · Qwen3.8-Flash-Next
FAMILY=nvfp4 MODEL=NVFP4-NVIDIA, 225,280 context, 2 slots. Seen with MODEL=NVFP4 (GPTQ),
NVFP4-QAD and FAMILY=unsloth MODEL=UD-Q4_K_XL too, so it is not pack-specific.
Reproduce
-
Start any Qwen3.8-Flash-Next pack and POST a request whose tools declare a boolean parameter. Keep
"reasoning_effort": "none"— with thinking on the model usually spells ittrue, so the bug shows on a
minority of requests; thinking off, this prompt reproduced 8 times in 8:curl -s http://<host>/v1/chat/completions -H 'content-type: application/json' -d '{ "model": "qwen3.8-flash-next-nvfp4-nvfp4-nvidia", "reasoning_effort": "none", "messages": [{"role": "user", "content": "Get the prime factors of 450, formatted."}], "tools": [{"type": "function", "function": { "name": "get_prime_factors", "description": "Return the prime factors of a number.", "parameters": {"type": "object", "properties": { "number": {"type": "integer"}, "formatted": {"type": "boolean", "description": "Return the factors formatted as a string"}}, "required": ["number", "formatted"]}}}], "tool_choice": "auto"}' -
Read
choices[0].message.tool_calls[0].function.arguments.
What we got (captured live 2026-11-10, engine 0.1.57)
Same request, only reasoning_effort and temperature varied; arguments is
choices[0].message.tool_calls[0].function.arguments verbatim:
| attempts | arguments returned |
formatted type |
|---|---|---|
8 (thinking off via "reasoning_effort": "none", temp 0.7 and 0.0) |
{"number":450,"formatted":"True"} |
string, 8/8 |
| 4 (thinking on, server default, temp 0.7) | 1× {"number":450,"formatted":"True"}, 3× {"number":450,"formatted":true} |
string 1, boolean 3 |
So the failure is the model's spelling, not the sampling settings: when it writes True the server hands the
text through untouched, when it writes true the value arrives correctly typed. Thinking off, this prompt
fails every time; thinking on it is intermittent, which is why a single request can look fine. Full body of the
failing case is pasted under Logs; the correct case for contrast:
"tool_calls": [{"id": "call_fcd6accc1a144cd7913b8250", "type": "function", "function": {
"name": "get_prime_factors", "arguments": "{\"number\":450,\"formatted\":\"True\"}"}}] -- actual
"tool_calls": [{"id": "call_…", "type": "function", "function": {
"name": "get_prime_factors", "arguments": "{\"number\":450,\"formatted\":true}"}}] -- expected
Expected
Each <parameter=NAME>VALUE</parameter> value is converted by its declared JSON-schema type (boolean,
integer, number, array/object) before it is emitted in tool_calls[].function.arguments, so a boolean
parameter is a JSON true/false.
Actual
"formatted": "True" — a string where the schema says boolean.
BFCL v4 simple_python (400 entries, thinking off) counts these as type_error:simple. Entries affected, by
pack, on the same config: 15/400 NVFP4-NVIDIA (90.25% as scored → 94.0% counted correct), 25/400
NVFP4-QAD (88.75% → 95.0%), 20/400 NVFP4 GPTQ, 24/400 Unsloth UD-Q4_K_XL. Example, BFCL simple_python_17:
{"get_prime_factors": {"number": 450, "formatted": "True"}} → Expected type boolean, got str
Why it matters
Clients with strict tool validation (MCP servers, typed agent frameworks) reject the call. Worse, a client
that is lenient treats the non-empty string "False" as truthy, so the behaviour silently inverts —
the opposite of what the model asked for.
Evidence
-
Reproduced locally: the table above, run against our deployment on 2026-11-10, and seen on 0.2.7, 0.2.9 and
0.2.11 before that (BFCL suite plus direct requests). -
Suspected path (read on
main2026-11-05 and again 2026-11-10, not run against a 0.2.13 build):
serve/frontend.pycoerce_value(value, declared)— it does receive the declared type, but for anything
butstringit only triesjson.loads(value); JSON spells booleanstrue/false, so the model's
Python-styleTruefails to parse, and the fallback lenient path only fires when the text starts with
[,{or a backtick, so the raw text is returned unchanged. As ofmainon 2026-11-10 that function is
unchanged (line 1118), and the 0.2.12 and 0.2.13 release notes mention neither the symptom nor a fix.if declared == "string": return value try: v = json.loads(value) except ValueError: if not value.lstrip().startswith(("[", "{", "`")) or ...: return value # <-- "True" leaves here, still a string -
Precedent: SGLang's Qwen3-coder detector lowercases the text first for
boolean/bool
(param_value.lower() == "true"); vLLM's and SGLang'sqwen3_xmlparsers coerce by schema type.
Suggested fix (not tested as a patch — offered as the shape of it)
# in coerce_value, before json.loads(value)
if declared in ("boolean", "bool") and value.strip().lower() in ("true", "false"):
return value.strip().lower() == "true"
Safe to do here because the declared == "string" branch already returns before it, so a string parameter
whose value is literally True is untouched.
Searched first (2026-11-10)
boolean argument, True False string argument, schema type argument, coerce, type_error,
tool call parameter True. Closest hits, all distinct: #4037 (Chat/Messages coerce boolean token
limits — the request side, not tool arguments), #4630 (array/object parameter with a raw newline),
#4751 / #4071 (calls emitted as text or not read as a call of a declared tool), #5281 (streamed
MCP call landing in content), #4622 (undeclared tool-call names). Happy to move this into one of those
threads if a maintainer considers it a duplicate.
Workaround we use
Nothing server-side: our benchmark harness counts those entries separately so the number is not silently
wrong, and clients have to be told not to trust truthiness on tool arguments.
Logs
The failing response, verbatim (nothing but token counts and timings in it — no keys, no private text):
{
"id": "chatcmpl-575bfad3a8c14fb68dad4de7",
"object": "chat.completion",
"created": 1794343744,
"model": "qwen3.8-flash-next-nvfp4-nvfp4-nvidia",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": null,
"reasoning_content": "We need answer user: \"Get the prime factors of 450, formatted.\" Need call get_prime_factors with number 450 formatted true. Need final likely factorization. Let's compute? 450=2*3^2*5^2. Formatted maybe string. Need call tool.\n",
"tool_calls": [
{
"id": "call_fcd6accc1a144cd7913b8250",
"type": "function",
"function": {
"name": "get_prime_factors",
"arguments": "{\"number\":450,\"formatted\":\"True\"}"
}
}
]
},
"finish_reason": "tool_calls"
}
],
"usage": {
"prompt_tokens": 335,
"completion_tokens": 110,
"total_tokens": 445,
"prompt_tokens_details": {
"cached_tokens": 330
}
},
"timings": {
"cache_n": 330,
"prompt_n": 5,
"prompt_ms": 109.0,
"prompt_per_token_ms": 21.8,
"prompt_per_second": 45.9,
"predicted_n": 110,
"predicted_ms": 1210.3,
"predicted_per_token_ms": 11.003,
"predicted_per_second": 90.9,
"draft_n": 85,
"draft_n_accepted": 70
}
}
- 主要言語
- C++
- スター
- 11.6k
- フォーク
- 1k
- PR マージ指標
- 30日以内にマージされた PR はありません
環境構築
このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
Niko1221/Strata のほかの issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
Niko1221/Strata#5730 · コメント 2 件 ·
メンテナーはふだん 4 日以内に返信
-
needs-format
難易度 2/5 1〜3時間 初心者へのやさしさ 85/100
Niko1221/Strata#5720 · コメント 2 件 ·
メンテナーはふだん 4 日以内に返信
-
[Engine] Two prompt block scorers enabled at once: WMMA silently wins and no log line is printedオープン
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
Niko1221/Strata#5693 · コメント 2 件 ·
メンテナーはふだん 4 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
メンテナーはふだん 4 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 76/100
メンテナーはふだん 4 日以内に返信
Niko1221/Strata の issue をすべて見る
似ている issue
-
area:qmc bug
難易度 2/5 1〜3時間 初心者へのやさしさ 69/100
メンテナーはふだん 3 日以内に返信
-
bug good first issue
難易度 2/5 1〜3時間 初心者へのやさしさ 74/100
ASGC-Game-Jam/asgc-gamejam-2026#563 ·
メンテナーはふだん 1 日以内に返信
-
area/ysql kind/bug priority/medium status/awaiting-triage
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
yugabyte/yugabyte-db#37523 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 66/100
-
難易度 2/5 1〜3時間 初心者へのやさしさ 62/100
Cockatrice/Cockatrice#7896 ·
メンテナーはふだん 1 日以内に返信