Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

qwen3_coder streaming: a literal <tool_call> in prose hides the rest of the answer

オープン
  • 何か問題がありますか？
#716 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
38/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
python
領域
ai-infra-agents

調査の方向性

Start with InvokeParamStreamMixin.parse_streaming_increment and finish_streaming in python/freetoken/server/function_call_parser.py, which handle the idle and block modes. Then check _parse_tool_response in python/freetoken/server/generation.py, which supplies the raw-text fallback for non-streaming. Run the repro script from the issue to see the stream and non-stream outputs diverge. Done when both cases return the full text in streaming and the existing tool-call tests still pass.

索引モデルが issue の本文から書いたものです。

説明

What happened

With tools declared and the qwen3_coder parser, streaming treats any <tool_call> in the content as the start of a call. If no <function= follows, everything after it is dropped: the rest of the answer when the tag has no closer, or the text up to </tool_call> when it does. This happens when the model quotes or explains the tool-call format, for example while an agent works on tool-calling code. Non-streaming returns the full text for the same output.

model text streaming content non-streaming content
The parser looks for <tool_call> tags and splits the output there. The parser looks for full text
Wrap each call in <tool_call>...</tool_call> and keep going. Wrap each call in and keep going. full text

Minimal reproduction (CPU only, from a clean checkout, PYTHONPATH=python python repro.py):

import asyncio
from types import SimpleNamespace
from freetoken.message.frontend import UserReply
from freetoken.server.generation import ContentDelta, GenSpec, generate_events, generate_full

TOOLS = [{"type": "function", "function": {"name": "read", "parameters": {"type": "object",
          "properties": {"path": {"type": "string"}}}}}]

class State:  # stands in for the engine: replays the model's tokens, one per reply
    def __init__(self, pieces):
        self.config = SimpleNamespace(tool_call_parser="qwen3_coder", reasoning_parser=None)
        self.pieces = pieces
    async def wait_for_ack(self, uid):
        for i, p in enumerate(self.pieces):
            yield UserReply(uid=uid, incremental_output=p, finished=i == len(self.pieces) - 1,
                            prompt_tokens_delta=int(i == 0), completion_tokens_delta=1)

spec = lambda: GenSpec(messages=[], sampling_params=None, parser_tools=TOOLS, template_tools=TOOLS)
for pieces in (["The parser looks for ", "<tool_call>", " tags and splits", " the output there."],
               ["Wrap each call in ", "<tool_call>", "...", "</tool_call>", " and keep going."]):
    async def stream():
        return "".join([ev.text async for ev in generate_events(1, spec(), State(pieces))
                        if isinstance(ev, ContentDelta)])
    print("stream    :", repr(asyncio.run(stream())))
    print("non-stream:", repr(asyncio.run(generate_full(1, spec(), State(["".join(pieces)]))).content))

Output on main 9b68e007:

stream    : 'The parser looks for '
non-stream: 'The parser looks for <tool_call> tags and splits the output there.'
stream    : 'Wrap each call in  and keep going.'
non-stream: 'Wrap each call in <tool_call>...</tool_call> and keep going.'
Cause (current main, 9b68e007)

In InvokeParamStreamMixin.parse_streaming_increment (python/freetoken/server/function_call_parser.py; the mixin also backs minimax, which behaves the same with its own opener):

  • In idle mode, a <tool_call> always switches to block mode (:760-761), whatever follows it.
  • In block mode, text that is neither <function= nor </tool_call> is treated as whitespace between calls and discarded (:806-807).
  • At the end of the stream, finish_streaming returns "" for any mode other than idle (:906-907), so an opener with no closer also loses its tail.

Non-streaming keeps the text only because _parse_tool_response (python/freetoken/server/generation.py:463-464) falls back to the raw text when no call was parsed.

Proposed fix

After <tool_call>, look at the next non-whitespace text. If it cannot start <function=, </tool_call> or another <tool_call>, release the opener and that text as content and go back to idle. While that text is too short to decide, hold it, as idle mode already holds a partial opener, and have finish_streaming release a held opener that never started a call. Real calls keep parsing as now, and the two modes then agree on prose that quotes the tag.

Environment

Seen serving nvidia/Qwen3.8-Flash-Next-NVFP4 on an RTX 5080 16 GB; the reproduction above is CPU-only on main 9b68e007.

Related qwen3_coder stream/non-stream mismatches: #713, #714, #715. The empty-block handling proposed in #713 should not drop a quoted <tool_call>…</tool_call> that this issue treats as prose.

主要言語
Python
スター
14.2k
フォーク
1.4k
平均マージ
21時間 39分
マージ済み PR（30日）
100

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

FlashML-org/FreeToken のほかの issue

FlashML-org/FreeToken の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。