Worker panics (FFI Panic: ReadyForRoomEventRequest timeout) when two jobs' ctx.connect() run concurrently, killing all jobs on the worker
メンテナーはふだん 1 日以内に返信
まだ誰も着手していません。
評価
調査の方向性
Start with the minimal JobContext-only entrypoint in the issue and reproduce using two lk dispatch create calls to different rooms about a second apart. Then trace ctx.connect() and the room-connection path around the ReadyForRoomEventRequest timeout. Done means both jobs connect concurrently without a worker panic or a permanent hang, and both remain active.
索引モデルが issue の本文から書いたものです。
説明
Summary
Worker process crashes with a native FFI panic when two jobs' ctx.connect() calls happen within ~1 second of each other in the same worker. This reliably kills every active job on that worker, including ones that were already connected and running fine.
Tested with a non-conversational, JobContext-only entrypoint (no AgentSession, no voice pipeline) — a "programmatic participant" per the docs pattern in Job lifecycle. Repro does not require any real media/webcam — just two lk dispatch create calls to two different rooms a second apart.
Environment
livekit-agents1.8.3livekit(rtc/ffi) 1.1.18- OS: Windows 11
- Python 3.13.3
- Reproduced with both
lk agent devandlk agent start(i.e. not dev-mode-specific)
Minimal repro entrypoint
from livekit.agents import AgentServer, AutoSubscribe, JobContext, cli
server = AgentServer()
@server.rtc_session(agent_name="my-agent")
async def my_agent(ctx: JobContext) -> None:
await ctx.connect(auto_subscribe=AutoSubscribe.SUBSCRIBE_ALL)
if __name__ == "__main__":
cli.run_app(server)
Repro steps
lk agent dev(orlk agent start) to register the worker.- Fire two dispatches to two different, brand-new rooms within ~1 second of each other:
lk dispatch create --room concurrency-test-a --agent-name my-agent & lk dispatch create --room concurrency-test-b --agent-name my-agent & wait - The first job connects fine. ~10-20s later, the second job's connection attempt panics the whole worker process.
Actual behavior
WARNING livekit.agents - The room connection was not established within 10 seconds after calling job_entry. ... {"room": "concurrency-test-a"}
ERROR livekit - livekit_ffi::server::room:261:livekit_ffi::server::room - timed out waiting for ReadyForRoomEventRequest after ConnectCallback (room_handle=8)
FFI Panic: invalid request: timed out waiting for ReadyForRoomEventRequest after ConnectCallback (room_handle=8)
[exited with code 15]
The entire worker process exits — this also takes down the other, already-healthy job that was connected to concurrency-test-a.
Reproduced 3/3 times: twice under lk agent dev, once under lk agent start (production mode). Confirmed not specific to our own agent code, since it reproduces with the near-bare entrypoint above and no real participants/media.
Attempted workaround (partial, still broken)
We tried serializing ctx.connect() calls to avoid the race:
asyncio.Lock()at module scope — fails immediately withRuntimeError: <asyncio.locks.Lock object ...> is bound to a different event loop, because each job clearly runs in a separate OS process (confirmed viatasklist/pid), so an in-memory lock can't coordinate across them.- Cross-process file lock (
filelockpackage) — this avoids the panic, but introduces a new failure: the job holding the lock'sctx.connect()call never returns (no exception, no log line at all afterLock ... acquired), even though the participant shows asACTIVEinlk room participants list. The lock is then held forever, and every subsequent job spins forever retryingacquire()(never timing out, never erroring).
So today there's no way to run multiple concurrent jobs on one worker without either a process-killing panic or a silent permanent hang.
Why this matters
We're building an exam-proctoring agent (candidate video joins a room, agent watches for face-count/etc.) where multiple concurrent exam sessions per worker is the normal case, not an edge case. This bug currently makes that entirely unworkable — one candidate joining while another's session is active can silently take down every other active exam session on the same worker.
Happy to provide the full agent code, logs, or a minimal repo reproducing this if useful.
- 主要言語
- Python
- スター
- 14.6k
- フォーク
- 3.9k
- 平均マージ
- 2日 17時間
- マージ済み PR(30日)
- 207
環境構築
このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
livekit/agents のほかの issue
-
Assistant markup is stripped from chat history when expressive mode is off対応中かも @ShivamSharma43 が 19 日前に担当しました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
livekit/agents#7647 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
docs: openai.LLM vs openai.responses.LLM for HTTP OpenAI-compat hosts再び着手できるかも @AbarnaaSree が 59 日前に担当しましたが、オープン中のプルリクエストはありません。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
livekit/agents#7022 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
enhancement
難易度 5/5 1週間以上 初心者へのやさしさ 30/100
メンテナーはふだん 1 日以内に返信
-
enhancement
難易度 4/5 3〜5日 初心者へのやさしさ 48/100
メンテナーはふだん 1 日以内に返信
-
難易度 5/5 1週間以上 初心者へのやさしさ 35/100
メンテナーはふだん 1 日以内に返信
似ている issue
-
bug P1
難易度 2/5 1〜3時間 初心者へのやさしさ 76/100
Toloka/tolokaforge#1947 ·
メンテナーはふだん 1 日以内に返信
-
SageMaker provider: streaming fails with NoStreamDataError on DJL Serving (LMI) endpoints対応中かも @balajiaru が今日担当しました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 88/100
UKGovernmentBEIS/inspect_ai#6127 ·
メンテナーはふだん 2 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 67/100
メンテナーはふだん 2 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
letsencrypt/cp-cps#361 ·
-
難易度 2/5 1〜3時間 初心者へのやさしさ 65/100