qwen3 reasoning parser: streamed content starts with "\n\n" after </think> and streamed reasoning keeps its trailing newline; non-streaming strips both

Abierto
  • ¿Algo no está bien?
#712 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
3/5
Tiempo estimado
1-2 días
Aptitud para principiantes
62/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
python
Área
backend

Línea de trabajo

Start in python/freetoken/server/reasoning_parser.py: compare BaseReasoningParser.detect_and_parse with parse_streaming_increment, where the whitespace around </think> is handled differently. Run the issue's CPU reproduction with PYTHONPATH=python to see the stream and non-stream outputs. Done when both paths return the same reasoning and content for the text-answer and tool-call-only token sequences, with a test that covers both.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

What happened

With thinking on (--reasoning-parser qwen3), the model writes …reasoning\n</think>\n\n<answer or tool call>. Non-streaming strips the whitespace on both sides of </think>. Streaming does not, so the same turn looks different on /v1/chat/completions depending on stream:

stream: false stream: true (joined deltas)
content, text answer "Answer." "\n\nAnswer."
content, tool-call-only turn "" "\n\n", sent as a content delta before the tool_calls delta
reasoning_content "Think." "Think.\n"

So a thinking-on streamed answer starts with a blank line, and a tool-call-only turn sends a whitespace-only content delta. A client that creates a text part when the first content delta arrives therefore shows an empty assistant message before each tool call. The /v1/messages and /v1/responses streams go through the same path (generate_events), so their text blocks get the same "\n\n".

Minimal reproduction (CPU only, PYTHONPATH=python):

from freetoken.server.reasoning_parser import ReasoningParser
from freetoken.server.function_call_parser import FunctionCallParser
from freetoken.server.api_models import Tool

toks = ["Think", ".", "\n", "</think>", "\n\n", "Answer", "."]
p = ReasoningParser("qwen3", force_reasoning=True)
d = [p.parse_stream_chunk(t) for t in toks] + [p.flush()]
print("stream:     reasoning=%r content=%r" % ("".join(r for r, _ in d), "".join(c for _, c in d)))
r, c = ReasoningParser("qwen3", force_reasoning=True).parse_non_stream("".join(toks))
print("non-stream: reasoning=%r content=%r" % (r.strip(), c))   # .strip(): generation.py:823

# tool-call-only turn, streamed through the qwen3_coder detector as generate_events does
tools = [Tool(type="function", function={"name": "ls", "parameters": {"type": "object", "properties": {"path": {"type": "string"}}}})]
toks = ["Plan", "\n", "</think>", "\n\n", "<tool_call>", "\n<function=ls>\n<parameter=path>\n.\n</parameter>\n</function>\n</tool_call>"]
rp, fp, text = ReasoningParser("qwen3", force_reasoning=True), FunctionCallParser(tools, "qwen3_coder"), []
for t in toks:
    _, c = rp.parse_stream_chunk(t)
    if c: text += [e[1] for e in fp.parse_stream_events(c) if e[0] == "text"]
print("stream content deltas before the call:", text)
_, c = ReasoningParser("qwen3", force_reasoning=True).parse_non_stream("".join(toks))
print("non-stream content:", repr(FunctionCallParser(tools, "qwen3_coder").parse_non_stream(c).normal_text))

Output on main 9b68e007:

stream:     reasoning='Think.\n' content='\n\nAnswer.'
non-stream: reasoning='Think.' content='Answer.'
stream content deltas before the call: ['\n\n']
non-stream content: ''
Cause (current main, 9b68e007)
  • python/freetoken/server/reasoning_parser.py:134 and :151-153: BaseReasoningParser.detect_and_parse (non-streaming) strips the whole text and then normal_text.strip().
  • python/freetoken/server/reasoning_parser.py:176-183: parse_streaming_increment returns the text after </think> unstripped. Its rstrip() applies only to reasoning in the same chunk as </think>. When the "\n" arrives in an earlier delta than </think> (the usual case, since </think> is a single added token), it has already been streamed as reasoning by then.
  • python/freetoken/server/generation.py:823: the non-streaming result also .strip()s the reasoning. The streaming loop (:706-723) forwards the deltas as they are. In the tool path, suppress_ws (:667) only drops whitespace after a call has closed, not before the first one. openai_api.py:285 sends each ContentDelta as is.

The code is in BaseReasoningParser, so the qwen3, glm, minimax and deepseekv32 parsers all inherit it.

Proposed fix

Make the streaming path follow the non-streaming one in BaseReasoningParser:

  • After </think>, drop leading whitespace from content until the first non-whitespace character. Flush a whitespace-only remainder at the end as nothing.
  • While reasoning, hold back a trailing run of whitespace instead of streaming it. Emit it if more reasoning follows. Drop it when </think> closes the block.

Both are small, bounded holds, so streaming latency does not change. A stream-vs-non-stream test over the same token sequence (text answer and tool-call-only turn) would keep the two paths in line.

Environment

Seen serving nvidia/Qwen3.8-Flash-Next-NVFP4 on an RTX 5080 16 GB; the reproduction above is CPU-only on main 9b68e007.

Related: #711 (same parser, re-entry on a later <think>), #719 (chat logprobs design, which depends on both modes agreeing).

Lenguaje dominante
Python
Estrellas
14.2k
Forks
1.4k
Merge medio
1 d 1 h
PR fusionados (30 d)
102

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de FlashML-org/FreeToken

Todos los issues de FlashML-org/FreeToken

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.