qwen3_coder: max_tokens truncation leaks raw tool-call markup into content and arguments

Start with python/freetoken/server/function_call_parser.py: Qwen3CoderDetector.detect_and_parse and its function_regex, parameter_regex, and _parse_function_call, plus recover_truncated_call. Then read generation._parse_tool_response and the InvokeParamStreamMixin streaming path. Reproduce with the attached script at full and truncated max_tokens. Done when no cut point leaks <tool_call> or <function markup into content, a fabricated tool name, or a </parameter> fragment into arguments, in both modes.

Before you start

• I have read the FAQ and my problem is not answered there.
• I have read the Roadmap and this is not already planned there.
• I have searched existing issues and found no duplicate.
• I am on the latest release, or on a freshly rebuilt main when building from source.

What happened

On Qwen3.5-4B (the qwen3_coder tool parser and the qwen3 reasoning parser, both auto-selected),
/v1/chat/completions returns tool-call markup to the client when max_tokens cuts a tool call inside one of its tags. For a cut before the > that closes the <function=NAME> header, the two modes fail differently on the same request:

• Non-streaming: tool_calls is absent and message.content is the raw markup generated so far, for example <tool_call>\n<function=get_weather.
• Streaming: once the cut falls after <function= plus at least one character of the name, the stream emits a tool call whose name is the partial name with </tool_call appended, for example get_weather</tool_call or get</tool_call, with arguments {}. No tool by that name was offered. For earlier cuts the stream returns nothing.

A cut inside the parameter's closing tag </parameter> (after the value, before the tag's >) fails in the
non-streaming mode only:

• Non-streaming: the call is returned, but the cut tag is part of the argument value: {"city": "Lima\n</"} or {"city": "Lima\n</parameter"}.
• Streaming: the arguments stop at the value, {"city":"Lima, with no fragment.

With thinking off, the full call is 27 tokens. The cuts at max_tokens 1–6 out of 27 fall in the header and the cuts
at 18–19 fall in </parameter>. With thinking on (the same request with enable_thinking: true), the answer is a short reasoning and then the same call, 68 tokens, and all three faces appear at the same distance from the end: the header cuts at 42–47 of 68 (streamed names get</tool_call / get_weather</tool_call at 46–47), and the cuts at 59–60 of 68 return {"city": "Lima\n</"} / {"city": "Lima\n</parameter"} non-streaming and {"city":"Lima streamed. Cuts inside the reasoning return clean, partial reasoning_content. The same request at the full budget is fine in both modes, and the other cut points (inside <parameter=city>, inside the value, after </parameter>) return a clean call in both modes.

A stop string that ends the answer inside the header has the same non-streaming result, because stop strings are matched on the raw decode: stop: ["function"] gives content='<tool_call>\n<' and stop: ["\n"] gives content='<tool_call>', both with finish_reason: "stop".

Reproduced 3/3 runs on main (377a3bc): 36 affected answers and 0 control failures. These are 18 non-streaming header cuts, 6 streaming header cuts, 6 non-streaming </parameter> cuts and 6 non-streaming stops. With thinking on, 3/3 runs gave 30 affected answers (18 + 6 + 6, the same faces at the same cut points; both stop strings end inside the reasoning, so they do not apply) and 0 control failures. python/freetoken/server/function_call_parser.py is unchanged on current main, and so is generation._parse_tool_response.

Reproduction:

# the full call (thinking off) is 27 tokens; cut it at 6
curl -s localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "qwen35-4b", "temperature": 0, "max_tokens": 6, "chat_template_kwargs": {"enable_thinking": false},
  "messages": [{"role": "user", "content": "What is the weather in Lima right now? Use the get_weather tool."}],
  "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a city.",
             "parameters": {"type": "object", "properties": {"city": {"type": "string", "description": "The city name"}},
                            "required": ["city"]}}}]}'
# -> "content": "<tool_call>\n<function=get_weather", no "tool_calls", "finish_reason": "length"
# the same body with "stream": true -> one tool call: name "get_weather</tool_call", arguments "{}", finish_reason "length"
# the same body at "max_tokens": 19 (the cut falls inside </parameter>)
# -> "tool_calls": [{"function": {"name": "get_weather", "arguments": "{\"city\": \"Lima\\n</parameter\"}"}}], "finish_reason": "length"
#    streamed: arguments '{"city":"Lima' (no fragment)

repro_truncated_function_tag.py (stdlib only) sends the request at the full budget
in both modes, sweeps every cut point 1..N-1 in both modes, then tries the two stop strings; --thinking on runs the same sweep with thinking on. Every answer is checked
for markup in content, in tool-call names and in argument values. [A] is raw markup in content, [B] a
fabricated name, and [C] markup inside an argument value:

run 0 ok   control full budget json   finish=tool_calls tokens=27 calls=[('get_weather', '{"city": "Lima"}')] content=''
run 0 ok   control full budget stream finish=tool_calls tokens=None calls=[('get_weather', '{"city":"Lima"}')] content=''
run 0 FAIL [A] cut max_tokens=1   of 27: json finish=length calls=[] content='<tool_call>' | stream finish=length calls=[] content=''
run 0 FAIL [A] cut max_tokens=2   of 27: json finish=length calls=[] content='<tool_call>\n' | stream finish=length calls=[] content=''
run 0 FAIL [A] cut max_tokens=3   of 27: json finish=length calls=[] content='<tool_call>\n<' | stream finish=length calls=[] content=''
run 0 FAIL [A] cut max_tokens=4   of 27: json finish=length calls=[] content='<tool_call>\n<function' | stream finish=length calls=[] content=''
run 0 FAIL [A,B] cut max_tokens=5   of 27: json finish=length calls=[] content='<tool_call>\n<function=get' | stream finish=length calls=[('get</tool_call', '{}')] content=''
run 0 FAIL [A,B] cut max_tokens=6   of 27: json finish=length calls=[] content='<tool_call>\n<function=get_weather' | stream finish=length calls=[('get_weather</tool_call', '{}')] content=''
run 0 FAIL [C] cut max_tokens=18  of 27: json finish=length calls=[('get_weather', '{"city": "Lima\\n</"}')] content='' | stream finish=length calls=[('get_weather', '{"city":"Lima')] content=''
run 0 FAIL [C] cut max_tokens=19  of 27: json finish=length calls=[('get_weather', '{"city": "Lima\\n</parameter"}')] content='' | stream finish=length calls=[('get_weather', '{"city":"Lima')] content=''
run 0: 8 of 26 cut points carry tool-call markup in a visible field
run 0 FAIL stop=['function']: json finish=stop calls=[] content='<tool_call>\n<' | stream finish=stop calls=[] content=''
run 0 FAIL stop=['\n']: json finish=stop calls=[] content='<tool_call>' | stream finish=stop calls=[] content=''
faces (face, mode): A/json=18, B/stream=6, C/json=6
verdict: BUG PRESENT (36 answer(s) with qwen3_coder markup over 3 run(s); 0 control failure(s))

(Runs 1 and 2 are identical.)

Some possible cause, assited with Claude

1. Non-streaming: generation._parse_tool_response hands the text to FunctionCallParser.parse_non_stream, because <tool_call> and <function= are in TOOLS_TAG_LIST. Qwen3CoderDetector.detect_and_parse finds the block with the unterminated alternative of function_regex (<function=(.*?)</function>|<function=(.*)$), but _parse_function_call returns None when the function string has no > (function_str.index(">") raises). With no calls, _parse_tool_response returns None, so _generate_full_impl keeps the whole text as content. The same happens for a cut before <function= (detect_and_parse returns no call). The text[:idx] normal text the detector computed is never used.
2. Streaming: InvokeParamStreamMixin holds <function=get_weather in its buffer while waiting for >. At the end of the stream, FunctionCallParser.recover_truncated_call appends eot_token (</tool_call>) and re-parses. The unterminated <function=(.*)$ alternative now captures get_weather</tool_call>, and _parse_function_call takes everything up to the first > as the name: get_weather</tool_call.
3. That name is not one of the request's tools. _parse_function_call still keeps it because _should_forward_unknown_tool is true by default (FREETOKEN_FORWARD_UNKNOWN_TOOLS defaults to "True"), so recovered item is emitted as a call.
4. Argument value, non-streaming: Qwen3CoderDetector.parameter_regex is <parameter=(.*?)(?:</parameter>|(?=<parameter=)|(?=</function>)|$). When the text ends with </parameter or </ (the closing tag cut before its >), the lazy group cannot stop at a complete </parameter>, so it runs to $. _parse_function_call then strips only one leading and one trailing \n from the value, which ends in r or /, not in \n. The value keeps \n</parameter. The streaming path (InvokeParamStreamMixin, pstr mode) holds back a suffix that could still grow into </parameter> (_ends_with_partial_token) together with the trailing newline, so the streamed fragments stop at Lima.

How did you install FreeToken

Built from source

FreeToken version

freetoken 0.1.3, git rev-parse --short HEAD: 377a3bc (main, 2026-10-07, "ci: auto-label amd issues and prs by title (#620)")

OS

Arch Linux

OS details

Red Hat Enterprise Linux 9.8, kernel 5.14.0-687.56.1.el9_8.x86_64, Python 3.12.3.

GPU and driver

NVIDIA RTX A4000 16 GB (sm_86), CUDA 13.0.

CPU and system RAM

AMD EPYC 7313 (8 cores allotted), 48 GB allotted.

Checkpoint

Qwen/Qwen3.5-4B (Hugging Face, revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a)

Command

python -m freetoken --model <local snapshot of Qwen/Qwen3.5-4B @ 851bf6e> --served-model-name qwen35-4b --memory-ratio 0.85 \
    --max-seq-len-override 16384 --moe-strategy fused --cache-type radix --num-tokens 16384 --max-running-requests 4 \
    --max-prefill-length 2048 --enable-cache-report

Full log

Resolved config: attention_backend='fi', cache_type='hybrid_radix', page_size=1
Allocating 16384 tokens for KV cache, K + V = 0.50 GiB
API server is ready to serve on 127.0.0.1:35211
# the script's requests: only batches and 200s, no warning or traceback (0 Traceback lines in the whole log)
INFO     Prefill batch, #new-seq: 1, #new-token: 294, #cached-token: 0, token usage: 0.02, #mamba-slot: 4/24, mamba usage: 0.17, #running-req: 1, #queue-req: 0, input throughput (token/s): 296.45
INFO     Decode batch, #running-req: 1, #token: 304, token usage: 0.02, #mamba-slot: 4/24, mamba usage: 0.17, gen throughput (token/s): 15.00, #queue-req: 0
INFO:     127.0.0.1:46422 - "POST /v1/chat/completions HTTP/1.1" 200 OK
INFO:     127.0.0.1:46428 - "POST /v1/chat/completions HTTP/1.1" 200 OK
INFO     Prefill batch, #new-seq: 1, #new-token: 38, #cached-token: 256, token usage: 0.02, #mamba-slot: 4/24, mamba usage: 0.17, #running-req: 1, #queue-req: 0, input throughput (token/s): 33.21

Anything else

No response