generate_content_stream fails with UnknownApiResponseError when a streamed text part contains U+2028 / U+2029 / U+0085 (httpx transport)

未关闭
  • 哪里不对？
#3,243 1 条评论 0 个 reaction 已指派 1 人 在 GitHub 查看

维护者通常 1 天内回复

@Venkaiahbabuneelam 已经在做这个了。

开始于 2026年10月29日。

  • #3245 来自 @copybara-service —— 未关闭

评估

这个 Issue 还没有评估数据。

描述

priority: p2 status:awaiting user response type: bug
Environment details
  • Programming language: Python
  • OS: Windows 11 (transport-level, not OS-specific)
  • Language runtime version: 3.14
  • Package version: google-genai 2.33.0 (code unchanged on main)
Symptom

If a streamed chunk's JSON contains a raw U+2028 (LINE SEPARATOR), U+2029 or U+0085 (NEL) inside a string, models.generate_content_stream (sync) and aio.models.generate_content_stream with an httpx client raise:

UnknownApiResponseError: Failed to parse response as JSON. Raw response: {"candidates": [{"content": {"role": "model", "parts": [{"text": "line one

The same bytes parse correctly on the aiohttp async path.

Steps to reproduce

Self-contained script with a local SSE server, no API key (below):

sync  (httpx)   ERR UnknownApiResponseError ... "text": "line one
async (httpx)   ERR UnknownApiResponseError ... "text": "line one
async (aiohttp) ['line one
line two', ' done']

The body is valid JSON (RFC 8259 doesn't require escaping these code points), one data: line per event.

repro_genai_linesep.py 
"""google-genai: streamed text containing U+2028 / U+2029 / U+0085 breaks SSE parsing (httpx path)."""
import asyncio, json, sys, threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import httpx
from google import genai

TEXT = sys.argv[1].encode().decode("unicode_escape") if len(sys.argv) > 1 else "line one\u2028line two"
CHUNKS = [
    {"candidates": [{"content": {"role": "model", "parts": [{"text": TEXT}]}}]},
    {"candidates": [{"content": {"role": "model", "parts": [{"text": " done"}]}, "finishReason": "STOP"}]},
]
# Valid JSON (RFC 8259 does not require escaping these code points); one event per line.
BODY = b"".join(b"data: " + json.dumps(c, ensure_ascii=False).encode() + b"\r\n\r\n" for c in CHUNKS)

class H(BaseHTTPRequestHandler):
    def do_POST(self):
        self.rfile.read(int(self.headers["Content-Length"]))
        self.send_response(200)
        self.send_header("Content-Type", "text/event-stream")
        self.send_header("Content-Length", str(len(BODY)))
        self.end_headers()
        self.wfile.write(BODY)
    def log_message(self, *a): pass

srv = ThreadingHTTPServer(("127.0.0.1", 0), H)
threading.Thread(target=srv.serve_forever, daemon=True).start()
base = f"http://127.0.0.1:{srv.server_port}"
print(f"genai {genai.__version__}, text={TEXT!r}")

def run_sync():
    c = genai.Client(api_key="test", http_options={"base_url": base})
    return [ch.text for ch in c.models.generate_content_stream(model="gemini-2.5-flash", contents="hi")]

async def run_async(httpx_client):
    opts = {"base_url": base}
    if httpx_client:
        opts["httpx_async_client"] = httpx.AsyncClient()
    c = genai.Client(api_key="test", http_options=opts)
    return [ch.text async for ch in await c.aio.models.generate_content_stream(model="gemini-2.5-flash", contents="hi")]

for name, fn in [("sync  (httpx)  ", run_sync),
                 ("async (httpx)  ", lambda: asyncio.run(run_async(True))),
                 ("async (aiohttp)", lambda: asyncio.run(run_async(False)))]:
    try:
        print(name, fn())
    except Exception as e:
        print(name, "ERR", type(e).__name__, str(e)[:160].replace("\n", " "))
Root cause

google/genai/_api_client.py:488 (iter_lines()) and :545 (aiter_lines()) use httpx's line decoder, which follows str.splitlines() and also breaks on U+2028/U+2029/U+0085 (and , , -). The SSE spec treats only CRLF, LF and CR as line terminators. The aiohttp branch (:587, readline() on bytes) gets this right.

Suggested fix

Split the byte stream on `

/
/
only (e.g. iterateiter_bytes()/aiter_bytes()` through a small line splitter, as the aiohttp branch effectively does), then decode each line.

主要语言
Python
星标
4.1k
派生
1.1k
平均合并
1 天 20 小时
30 天内合并 PR
108

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

googleapis/python-genai 的其他 Issue

查看 googleapis/python-genai 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。