[Engine] Newline stop at max_tokens returns finish_reason "length" after removing the stop

Inizia in serve/server.py, nel percorso di completamento OpenAI intorno alla riga 5886, poi confrontalo con il record done intorno alle righe 5606-5610 e con la mappatura Anthropic intorno alla riga 6025. Traccia come vengono mappati una sequenza di arresto rilevata e il motivo di completamento del motore, poi controlla i limiti attuali in docs/DETAILS.md. Il lavoro è completato quando /v1/chat/completions restituisce finish_reason: "stop" per il confine di nuova riga riprodotto, mentre il comportamento esistente rimane coerente con /v1/messages.

Area

[Engine] (speed, kernels, models, the server and API)

Desktop app, Terminal or source?

Source (START-HERE.bat, setup.sh or a build)

What happened

On /v1/chat/completions with stop: ["\n"] and max_tokens: 3, a greedy answer whose third token is the newline comes back as "ALPHA" with finish_reason: "length". The server recognised the stop string and removed it from the content, but reports the token cap as the reason.

The same request on /v1/messages (stop_sequences: ["\n"], max_tokens: 3) returns stop_reason: "stop_sequence" with stop_sequence: "\n".

Expected: finish_reason: "stop". docs/DETAILS.md ("Current limits") says the answer ends before the first stop string, which is not sent, with finish_reason "stop" (stop_reason "stop_sequence" on the Anthropic side).

From the code at 7dd5cf18:

• serve/server.py:5606-5610 builds the done record with the engine's finish ("length" here) and adds stop_sequence when stops.hit is set, without changing finish.
• The Anthropic path (serve/server.py:6025) checks stop_sequence first, so it reports stop_sequence.
• The OpenAI path (serve/server.py:5886) maps from finish alone, so length stays length.

Smallest steps or command that reproduce it

Checkout 7dd5cf18e68c639a8773e096c5c79f971ea2a9f5. build/ was configured with CMake; the values from build/CMakeCache.txt are: generator Unix Makefiles, CMAKE_BUILD_TYPE=Release, STRATA_ENABLE_CUDA=ON, CMAKE_CUDA_ARCHITECTURES=120.

Server config I ran (area3_config.json):

{
  "exe": "/content/Strata/build/strata",
  "cwd": "/content/Strata",
  "tokenizer": "/content/Strata-data/packs/q2_0/tokenizer",
  "model_name": "qwen3.8-flash-next-q2_0",
  "log": "/content/area3_engine.log",
  "args": [
    "--pack", "/content/Strata-data/packs/q2_0",
    "--native", "/content/Strata-data/models/Q2_0/Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf",
    "--ple-gguf", "/content/Strata-data/models/Q2_0/Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00002-of-00002.gguf",
    "--expert-profile", "/content/Strata/data/expert-profile.bin",
    "--expert-cache", "auto",
    "--prefill", "512",
    "--spec", "4",
    "--spec-min-p", "0.5",
    "--mtp", "/content/Strata-data/mtp/rt",
    "--max-context", "4096",
    "--kv", "int8",
    "--ple-io", "ram",
    "--vram-reserve-mib", "74000",
    "--adapt-swaps", "0",
    "--pcie-frac", "0"
  ]
}

cd /content/Strata
python3 serve/server.py --engine strata --config /content/area3_config.json --host 127.0.0.1 --port 18091

Client, from another terminal:

import json
import urllib.request

base = "http://127.0.0.1:18091"
messages = [{
    "role": "user",
    "content": "Write exactly these two lines and nothing else:\nALPHA\nBETA"
}]

def post(path, body):
    req = urllib.request.Request(
        base + path,
        data=json.dumps(body).encode(),
        headers={"Content-Type": "application/json"}
    )
    with urllib.request.urlopen(req, timeout=180) as response:
        return json.load(response)

for cap, stop in [(20, False), (20, True), (3, False), (3, True)]:
    body = {
        "model": "strata",
        "messages": messages,
        "temperature": 0,
        "reasoning_effort": "none",
        "max_tokens": cap
    }
    if stop:
        body["stop"] = ["\n"]
    result = post("/v1/chat/completions", body)
    choice = result["choices"][0]
    print(json.dumps({
        "max_tokens": cap,
        "stop_enabled": stop,
        "content": choice["message"]["content"],
        "finish_reason": choice["finish_reason"],
        "completion_tokens": result["usage"]["completion_tokens"]
    }))

result = post("/v1/messages", {
    "model": "strata",
    "messages": messages,
    "temperature": 0,
    "thinking": {"type": "disabled"},
    "max_tokens": 3,
    "stop_sequences": ["\n"]
})
print(json.dumps({
    "content": result["content"],
    "stop_reason": result["stop_reason"],
    "stop_sequence": result.get("stop_sequence"),
    "output_tokens": result["usage"]["output_tokens"]
}))

The max_tokens: 3 control without a stop returns ALPHA\n, so the third token is the newline. I ran only this config (--vram-reserve-mib 74000 leaves a partial expert cache on this 97,887 MiB card); the finish mapping above is in serve/server.py, not in the engine.

Reproduction status

Reproduced locally

Strata version

Commit 7dd5cf18e68c639a8773e096c5c79f971ea2a9f5 (main), engine 0.1.49.2

Graphics card, memory and system

NVIDIA RTX PRO 6000 Blackwell Server Edition 97,887 MiB · 176 GiB RAM · Ubuntu 24.04.4 LTS (Linux 6.6.122+, Google Colab), driver 580.82.07, CUDA 13.0.88, GCC 13.3.0, Python 3.13.16, AMD EPYC 9B45 (48 logical CPUs)

Model, size and settings

Qwen3.8-Flash-Next GSQ-RCO Q2_0 (two GGUF parts, pack, MTP draft layer) · 4096 context · INT8 KV · settings as in the config above

Logs

Client output (my run, 2026-10-25):

{"max_tokens": 20, "stop_enabled": false, "content": "ALPHA\nBETA", "finish_reason": "stop", "completion_tokens": 6}
{"max_tokens": 20, "stop_enabled": true, "content": "ALPHA", "finish_reason": "stop", "completion_tokens": 4}
{"max_tokens": 3, "stop_enabled": false, "content": "ALPHA\n", "finish_reason": "length", "completion_tokens": 3}
{"max_tokens": 3, "stop_enabled": true, "content": "ALPHA", "finish_reason": "length", "completion_tokens": 3}
{"content": [{"type": "text", "text": "ALPHA"}], "stop_reason": "stop_sequence", "stop_sequence": "\n", "output_tokens": 3}

Server log, one done line per request in the same order:

[strata] done: 6 tokens in 0 s (117.0 tok/s) (stop, cancel=False), expert cache 72.4% hit
[strata] done: 4 tokens in 0 s (232.6 tok/s) (stop, cancel=False), expert cache 79.2% hit
[strata] done: 3 tokens in 0 s (148.5 tok/s) (length, cancel=False), expert cache 79.2% hit
[strata] done: 3 tokens in 0 s (149.3 tok/s) (length, cancel=False), expert cache 79.2% hit
[strata] done: 3 tokens in 0 s (149.3 tok/s) (length, cancel=False), expert cache 79.2% hit

Not checked: streaming responses at this boundary, other models, other engine settings.