openai_harmony.HarmonyError: Unexpected token 12606 while expecting start token 200006
Open
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 30/100
Research direction
Start with the reported failure at sglang/srt/entrypoints/openai/serving_chat.py:520 and the openai_harmony parser process call shown in the traceback. Reproduce it with the provided launch command and openai-harmony==0.0.4, then compare the related vLLM issue; done means the token-stream compatibility failure is explained and covered by a verified fix or regression test.
Written by the indexing model from the issue text.
Description
When using sglang engine for inference and Crush agent there is errors in sglang logs:
[2025-08-12 08:36:55] ERROR: Exception in ASGI application
+ Exception Group Traceback (most recent call last):
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/_utils.py", line 77, in collapse_excgroups
| yield
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/responses.py", line 271, in __call__
| async with anyio.create_task_group() as task_group:
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 772, in __aexit__
| raise BaseExceptionGroup(
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
| result = await app( # type: ignore[func-returns-value]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
| return await self.app(scope, receive, send)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
| await super().__call__(scope, receive, send)
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 186, in __call__
| raise exc
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 164, in __call__
| await self.app(scope, receive, _send)
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, in __call__
| await self.app(scope, receive, send)
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
| await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| raise exc
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
| await app(scope, receive, sender)
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/routing.py", line 716, in __call__
| await self.middleware_stack(scope, receive, send)
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/routing.py", line 736, in app
| await route.handle(scope, receive, send)
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/routing.py", line 290, in handle
| await self.app(scope, receive, send)
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/routing.py", line 78, in app
| await wrap_app_handling_exceptions(app, request)(scope, receive, send)
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
| raise exc
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
| await app(scope, receive, sender)
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/routing.py", line 76, in app
| await response(scope, receive, send)
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/responses.py", line 270, in __call__
| with collapse_excgroups():
| File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__
| self.gen.throw(value)
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/_utils.py", line 83, in collapse_excgroups
| raise exc
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/responses.py", line 274, in wrap
| await func()
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/responses.py", line 254, in stream_response
| async for chunk in self.body_iterator:
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/sglang/srt/entrypoints/openai/serving_chat.py", line 520, in _generate_chat_stream
| harmony_parser.process(token_id)
| File "/home/drros/sglang/.venv/lib/python3.12/site-packages/openai_harmony/__init__.py", line 627, in process
| self._inner.process(token)
| openai_harmony.HarmonyError: Unexpected token 12606 while expecting start token 200006
+------------------------------------
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
await super().__call__(scope, receive, send)
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 186, in __call__
raise exc
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, in __call__
await self.app(scope, receive, send)
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/routing.py", line 716, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/routing.py", line 736, in app
await route.handle(scope, receive, send)
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/routing.py", line 290, in handle
await self.app(scope, receive, send)
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/routing.py", line 78, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/routing.py", line 76, in app
await response(scope, receive, send)
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/responses.py", line 270, in __call__
with collapse_excgroups():
File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__
self.gen.throw(value)
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/_utils.py", line 83, in collapse_excgroups
raise exc
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/responses.py", line 274, in wrap
await func()
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/starlette/responses.py", line 254, in stream_response
async for chunk in self.body_iterator:
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/sglang/srt/entrypoints/openai/serving_chat.py", line 520, in _generate_chat_stream
harmony_parser.process(token_id)
File "/home/drros/sglang/.venv/lib/python3.12/site-packages/openai_harmony/__init__.py", line 627, in process
self._inner.process(token)
openai_harmony.HarmonyError: Unexpected token 12606 while expecting start token 200006
vllm already have similar issue - https://github.com/vllm-project/vllm/issues/22519
running sglang with following command: python3 -m sglang.launch_server --model /mnt/nfs-share/LLM/gpt-oss-20b/ --host 0.0.0.0 --port 30000 --tp 2 --context 65536 --trust-remote-code --dtype float16 --mem-fraction-static 0.8
openai-harmony==0.0.4
- Dominant language
- Rust
- Stars
- 4.5k
- Forks
- 315
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from openai/harmony
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Rendering is quadratic in the length of a single messagePossibly taken @rksharma-owg claimed this 55 days ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 55/100
-
Difficulty 3/5 1-2 days Newbie friendliness 25/100
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
-
Python asyncio eventloop run_in_executor with ThreadPoolExecutor is not workingPossibly taken @Gnoale claimed this 216 days ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 35/100
Similar issues
-
question
Difficulty 1/5 Under an hour Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
autoindex tests: denied loopback bind shows only PermissionDenied (255 failures), not the causeOpenfield-report
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Maintainers usually reply within 1 day
-
area:core
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
julian-hemmer/ypass#4 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 70/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
davidkodar/klipp#3426 ·
Maintainers usually reply within 1 day