Truncated tool call is emitted as a complete one (unterminated heredoc reaches the client)
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
Research direction
Start in modeling_qwen3_5vl.cpp around lines 493-500 and trace parse_nstream_content for blocks lacking both closing tags. Use the provided flm serve and curl reproduction to check the emitted tool_calls and finish_reason. Done means incomplete tool-call blocks are not emitted, while complete calls and the fallback still work.
Written by the indexing model from the issue text.
Description
Summary
When generation stops inside a <tool_call> block — normally because it hit the
output limit while writing a large argument — the parser takes the remainder of
the text as the block and emits a complete-looking tool call built from partial
input. The client cannot detect this: the arguments JSON is well formed,
because it is rebuilt from the parsed parameters. A shell heredoc cut mid-body
becomes an unterminated cat <<EOF.
Affects qwen3_5vl, qwen3_6_moe and qwen3_5_omni. Reproduced on main at
92f3f13.
Environment
- OS: Ubuntu 26.04.1 LTS (kernel 7.0.0-31-generic)
- NPU driver: NPU FW 1.1.2.64, amdxdna 0.7
flm version:FLM v1.0.6flm validate:
[Linux] Kernel: 7.0.0-31-generic
[Linux] NPU: /dev/accel/accel0 with 8 columns
[Linux] NPU FW Version: 1.1.2.64
[Linux] amdxdna version: 0.7
[Linux] Memlock Limit: infinity
Reproduce
flm serve qwen3.5:9b
curl -s localhost:52625/v1/chat/completions -H 'Content-Type: application/json' -d '{
"model": "qwen3.5:9b",
"messages": [{"role":"user","content":"Use the bash tool to write a very long 300-line poem to /tmp/p.txt using a cat heredoc."}],
"max_tokens": 400,
"tools": [{"type":"function","function":{"name":"bash","description":"Run a shell command",
"parameters":{"type":"object","properties":{"command":{"type":"string"}},"required":["command"]}}}],
"tool_choice": "auto"
}' | jq '.choices[0]'
Observed on 92f3f13:
finish_reason : length
tool_calls : True
args length : 1580
args tail : '...n a sacred soundless gown.\n\nSo let us walk gently on this precious"}'
The heredoc has no closing EOF. A client that executes this hangs waiting for
the terminator.
Cause
parse_nstream_content, e.g. modeling_qwen3_5vl.cpp:493-500:
} else {
// Unclosed tag — search for </function> fallback
size_t func_end_pos = response_text.find(func_end_tag, block_content_start);
if (func_end_pos != std::string::npos) {
block_end = func_end_pos + func_end_tag.length();
} else {
block_end = response_text.length(); // <-- fabricates a complete call
}
The </function> fallback is reasonable: a block missing only the outer
</tool_call> is still complete. The final else is not — neither closing tag
present means the call was never finished.
Impact
finish_reason is length, which is the only hint, and clients commonly ignore
it when tool_calls is present. The result is a syntactically valid tool call
carrying a truncated command. For file-writing agents that is an unterminated
heredoc; for other tools it is a silently truncated argument.
Fix
PR follows: drop the block when neither closing tag is present, so no tool call
is emitted. The response still carries finish_reason: "length", which is the
honest signal that the client should retry with a larger budget. Complete calls
and the </function> fallback are unaffected.
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 152
- Avg merge
- 4h 14m
- Merged PRs (30d)
- 11
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ROCm/FastFlowLM
-
No valid checkpoint to restore + Max length reached on multi-turn tool calls (Qwen3.6-MoE, NPU) Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
ROCm/FastFlowLM#744 · 2 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
ROCm/FastFlowLM#730 ·
-
Download timeout Open
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
ROCm/FastFlowLM#520 · 7 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
ROCm/FastFlowLM#434 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
ROCm/FastFlowLM#400 · 8 comments · 10 reactions ·
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
AXERA-TECH/ax-llm#77 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
games-on-whales/wolf#509 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
bug-unconfirmed
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
NVIDIA/cuda-samples#453 ·