Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

BVCTelephony: AudioStream.from_track blocks the job event loop for ~260 ms at session start, and the filter cannot be prewarmed

Đang mở
  • Có gì không ổn?
#8,189 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
20/100
Loại issue
Lỗi
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
python, rust
Lĩnh vực
audio-video-rtc, performance

Hướng nghiên cứu

Start with the linked livekit-ffi/src/room_apis/server/audio_stream.rs lines 64-70 and 119-137, then follow the Python path through voice/room_io/_input.py:422, rtc/audio_stream.py:268 and rtc/_ffi_client.py:307. Reproduce the event_loop_blocked span on a SIP call and confirm the native new_session() call runs on the caller thread. Done means NewAudioStream no longer blocks the job loop, or a supported preload API lets the model load in setup_fnc, with the monitor no longer reporting the block.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Bug Description

This is a follow-up to #7441. It adds production data for BVCTelephony(), shows where the time goes in livekit-ffi, and describes a second problem: there is no way to prewarm the filter.

In a production telephony voice agent (SIP participant, RoomInputOptions(noise_cancellation=noise_cancellation.BVCTelephony())), the built-in loop monitor reports event_loop_blocked during AgentSession.start() in about 98% of calls:

metric (7 days, 1,593 spans) p50 p95
wall time 263 ms 342 ms
lk.blocking.cpu_time 224 ms –

The loop thread is busy in native code (CPU time is close to wall time). The block is 2–3 times longer than the 106–119 ms that #7441 reports for BVC() on a different machine.

Sampled stack (same in almost all spans):

voice/room_io/room_io.py:384   _init_task
voice/room_io/_input.py:422    _create_stream
rtc/audio_stream.py:268        _create_owned_stream
rtc/_ffi_client.py:307         request

The NewAudioStreamRequest carries audio_filter_options with the BVC telephony model path (inb.bvc.hs.c6.w.s.23cdb3.kef, about 28 MB).

Where the time goes. In livekit-ffi, FfiAudioStream::from_track calls the audio filter plugin's new_session() (plugin create / create_v2) synchronously inside the request handler, before it spawns the stream task:
https://github.com/livekit/rust-sdks/blob/71219b95e402b720b5c2682496c50b6cc80b0888/livekit-ffi/src/room_apis/server/audio_stream.rs#L119-L137
Because NewAudioStream is a synchronous FFI request, this runs on the thread that calls livekit_ffi_request. In an agent job, that is the job's asyncio event loop thread. The application cannot move this call off the loop.

Prewarm is not possible. We tried to create the filter or the stream in setup_fnc (prewarm), so the model loads before the job starts. This fails:

from livekit import rtc
from livekit.plugins import noise_cancellation

src = rtc.AudioSource(48000, 1)
track = rtc.LocalAudioTrack.create_audio_track("probe", src)
rtc.AudioStream.from_track(
    track=track, sample_rate=48000, num_channels=1,
    noise_cancellation=noise_cancellation.BVCTelephony(),
)
# -> InvalidRequest("this track has no room information")

The FFI requires a track with a room handle before it creates the filter session (same file, L64-L70). Thus the model can only load inside a live job, on the event loop.

We do not know if the native plugin caches the model per process. In our setup each job runs in its own process, so a per-process cache alone does not help unless the model can be loaded in setup_fnc.

Expected Behavior
  • Audio stream creation with noise cancellation does not block the job event loop for hundreds of milliseconds.
  • There is a supported way to load the noise cancellation model before the job starts (for example in setup_fnc).
Reproduction Steps
from livekit.agents import Agent, AgentSession, JobContext, RoomInputOptions
from livekit.plugins import noise_cancellation

async def entrypoint(ctx: JobContext):
    await ctx.connect()
    session = AgentSession(...)  # any STT/LLM/TTS or realtime model
    await session.start(
        room=ctx.room,
        agent=Agent(instructions="..."),
        room_input_options=RoomInputOptions(
            noise_cancellation=noise_cancellation.BVCTelephony(),
        ),
    )
  1. Run the worker with the process job executor (forkserver) on a Linux x86_64 container with 2 vCPU.
  2. Join the room as a SIP participant (an inbound call).
  3. Read the event_loop_blocked warnings and spans. The block starts when RoomIO creates the audio stream for the SIP track.

#7441 reports that the same stream creation without noise cancellation does not trip the monitor.

Operating System

Linux x86_64 (container, 2 vCPU)

Models Used

Not relevant. The block happens before any model is used.

Package Versions
python==3.11
livekit-agents==1.8.9
livekit==1.1.21
livekit-plugins-noise-cancellation==0.3.2

I checked livekit 1.2.0 and the current main of agents, python-sdks and rust-sdks. The code path is unchanged.

Session/Room/Call IDs

No response

Proposed Solution

Any one of these removes the block from the call path:

  1. Create the filter session off the caller's thread. In livekit-ffi, call new_session() inside the spawned stream task (or on a blocking thread) and pass frames through until the session is ready. This is the same pattern as the fix for the Krisp VIVA filter in #7924. Alternatively, make NewAudioStream an async FFI request with a callback, so AudioStream can await it.
  2. Add a preload or prewarm API for the noise cancellation plugin, for example noise_cancellation.preload(BVCTelephony()), that loads the model in the process without a room. Agents can then call it in setup_fnc.
  3. Cache the loaded model per process in the native plugin (if it does not do this already), together with option 2.

Impact: the block happens right before the first greeting. It delays the first audio of nearly every call by about 250 ms. It also stops all other coroutines of the job (STT/LLM connection setup, telemetry, timers) during that time.

Additional Context

Related:

  • #7441: same stack with BVC(), 106–119 ms, open without a reply.
  • #7924: Krisp VIVA filter built on the event loop on the first frame (200–800 ms); confirmed and fixed.
  • livekit/rust-sdks#1163: audio filter on_load() blocks FfiRoom::connect with no timeout (another synchronous native filter call).

If you prefer one issue, feel free to close this one as a duplicate of #7441. The data here can move there.

Ngôn ngữ chính
Python
Star
14.6k
Fork
3.9k
Merge trung bình
2 ngày 6 giờ
Pull request đã merge (30 ngày)
227

Chuẩn bị môi trường

Dự án này không cung cấp dev container, Dockerfile hay hướng dẫn đóng góp, nên bạn cần tự thiết lập môi trường: hãy bắt đầu từ README và xem hướng dẫn đóng góp lần đầu của chúng tôi để biết các bước chung.

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của livekit/agents

Tất cả issue của livekit/agents

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.