Structured output: gpt-6-luna (reasoning none) adds text after a valid object — allow recovering it, not failing JSON.parse

Start with adapters/text.js and adapters/responses-text.js, tracing structuredOutputStream and structuredOutput parsing, then read harvestCombinedStructuredOutput in activities/chat/index.js. Compare streaming and non-streaming error paths and the existing schema validation behavior. Done means the selected recovery or raw-error approach works consistently across the named paths without losing valid structured output.

Versions: @tanstack/ai 0.58.0, @tanstack/ai-openrouter 0.19.17

Model and frequency

• Model: openai/gpt-6-luna through OpenRouter (@tanstack/ai-openrouter 0.19.17, @tanstack/ai 0.58.0), with outputSchema set (strict json_schema) and reasoning effort "none".
• Frequency: on a long (~65k-token) system prompt with a four-field schema, about 1–2% of calls stream one complete, schema-valid object and then keep writing. Most often the extra text is a second, near-identical copy of the same object. Sometimes it is stray text, such as *****/ or a sentence like "Need valid JSON…". In our logs the second object repeated the first object's field values in almost every case, so the first object is the model's real answer.
• Reasoning effort matters: the same model with a little reasoning (effort "minimal") produced 0 such replies over a full run of ~1,400 calls.
• Not a provider issue: restricting OpenRouter routing to endpoints that support structured outputs (provider.require_parameters) did not change the rate (12 vs 13 on the same 100 cases), so it is not one provider ignoring response_format.
• Recoverable: extracting the first balanced top-level object recovered every one of these replies in our evals (0 needed a retry).

What happens

We use outputSchema on the OpenRouter adapter with reasoning turned off (effort "none") on a small OpenAI model. About 1–2% of calls stream a reply that holds a complete JSON object that passes the schema, followed by more text. Usually the extra text is a near-copy of the object. Sometimes it is a stray remark. Example (anonymized):

{"query":"t:creature ...","preciseScore":9}

{"query":"t:creature ..."}

Strict JSON.parse rejects the whole reply, so the run fails with code "parse-error". The first object is the model's real answer, and it is lost.

The same thing happens when string values hold regex-like content. They can include backslashes that JSON does not allow (e.g. \{), so parsing fails even when the reply is a single object.

Where this happens:

• @tanstack/ai-openrouter adapters/text.js: structuredOutputStream builds up the streamed text, calls JSON.parse(accumulatedContent), and on failure yields a RUN_ERROR (code: "parse-error"). The message holds only accumulatedContent.slice(0, 200). The non-streaming structuredOutput path in the same file does the same thing.
• adapters/responses-text.js does the same, in both the streaming and non-streaming paths.
• @tanstack/ai activities/chat/index.js: harvestCombinedStructuredOutput also calls JSON.parse strictly and keeps a 200-character preview (code: "structured-output-parse-failed"). The normalize hook runs only after JSON.parse succeeds, so there is no hook that can fix the text before it is parsed.

Why it's hard to work around

The error carries only a 200-character preview, and normalize runs after parsing. To recover, a consumer has to collect the raw text itself from TEXT_MESSAGE_CONTENT chunks in a middleware onChunk, then parse it again inside a catch around chat(). This works: it recovered all 21 of these failures in our eval suite. But it is fragile, depends on the adapter's chunk shape, and has to be duplicated for each adapter path.

Proposal

Any one of these would solve it:

1. An option on chat() / outputSchema that runs before JSON.parse, such as parseStructuredOutput?: (raw: string) => unknown or repair?: (raw: string) => string.
2. Built-in lenient extraction: before giving up, take the first balanced top-level object and parse that.
3. At the very least, attach the full raw text to the error (e.g. error.raw) instead of a 200-character preview, so consumers can recover without capturing chunks.

For reference, our repair is about 30 lines:

1. Take the first balanced {...}, tracking strings and escapes.
2. If that still fails to parse, double every backslash that is not followed by a valid JSON escape character.
3. Validate the result against the schema.

Glad to share the code if it helps.