Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

BVCTelephony: AudioStream.from_track blocks the job event loop for ~260 ms at session start, and the filter cannot be prewarmed

Aperta
  • Qualcosa non va?
#8,189 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
20/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python, rust
Ambito
audio-video-rtc, performance

Direzione di ricerca

Start with the linked livekit-ffi/src/room_apis/server/audio_stream.rs lines 64-70 and 119-137, then follow the Python path through voice/room_io/_input.py:422, rtc/audio_stream.py:268 and rtc/_ffi_client.py:307. Reproduce the event_loop_blocked span on a SIP call and confirm the native new_session() call runs on the caller thread. Done means NewAudioStream no longer blocks the job loop, or a supported preload API lets the model load in setup_fnc, with the monitor no longer reporting the block.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Bug Description

This is a follow-up to #7441. It adds production data for BVCTelephony(), shows where the time goes in livekit-ffi, and describes a second problem: there is no way to prewarm the filter.

In a production telephony voice agent (SIP participant, RoomInputOptions(noise_cancellation=noise_cancellation.BVCTelephony())), the built-in loop monitor reports event_loop_blocked during AgentSession.start() in about 98% of calls:

metric (7 days, 1,593 spans) p50 p95
wall time 263 ms 342 ms
lk.blocking.cpu_time 224 ms –

The loop thread is busy in native code (CPU time is close to wall time). The block is 2–3 times longer than the 106–119 ms that #7441 reports for BVC() on a different machine.

Sampled stack (same in almost all spans):

voice/room_io/room_io.py:384   _init_task
voice/room_io/_input.py:422    _create_stream
rtc/audio_stream.py:268        _create_owned_stream
rtc/_ffi_client.py:307         request

The NewAudioStreamRequest carries audio_filter_options with the BVC telephony model path (inb.bvc.hs.c6.w.s.23cdb3.kef, about 28 MB).

Where the time goes. In livekit-ffi, FfiAudioStream::from_track calls the audio filter plugin's new_session() (plugin create / create_v2) synchronously inside the request handler, before it spawns the stream task:
https://github.com/livekit/rust-sdks/blob/71219b95e402b720b5c2682496c50b6cc80b0888/livekit-ffi/src/room_apis/server/audio_stream.rs#L119-L137
Because NewAudioStream is a synchronous FFI request, this runs on the thread that calls livekit_ffi_request. In an agent job, that is the job's asyncio event loop thread. The application cannot move this call off the loop.

Prewarm is not possible. We tried to create the filter or the stream in setup_fnc (prewarm), so the model loads before the job starts. This fails:

from livekit import rtc
from livekit.plugins import noise_cancellation

src = rtc.AudioSource(48000, 1)
track = rtc.LocalAudioTrack.create_audio_track("probe", src)
rtc.AudioStream.from_track(
    track=track, sample_rate=48000, num_channels=1,
    noise_cancellation=noise_cancellation.BVCTelephony(),
)
# -> InvalidRequest("this track has no room information")

The FFI requires a track with a room handle before it creates the filter session (same file, L64-L70). Thus the model can only load inside a live job, on the event loop.

We do not know if the native plugin caches the model per process. In our setup each job runs in its own process, so a per-process cache alone does not help unless the model can be loaded in setup_fnc.

Expected Behavior
  • Audio stream creation with noise cancellation does not block the job event loop for hundreds of milliseconds.
  • There is a supported way to load the noise cancellation model before the job starts (for example in setup_fnc).
Reproduction Steps
from livekit.agents import Agent, AgentSession, JobContext, RoomInputOptions
from livekit.plugins import noise_cancellation

async def entrypoint(ctx: JobContext):
    await ctx.connect()
    session = AgentSession(...)  # any STT/LLM/TTS or realtime model
    await session.start(
        room=ctx.room,
        agent=Agent(instructions="..."),
        room_input_options=RoomInputOptions(
            noise_cancellation=noise_cancellation.BVCTelephony(),
        ),
    )
  1. Run the worker with the process job executor (forkserver) on a Linux x86_64 container with 2 vCPU.
  2. Join the room as a SIP participant (an inbound call).
  3. Read the event_loop_blocked warnings and spans. The block starts when RoomIO creates the audio stream for the SIP track.

#7441 reports that the same stream creation without noise cancellation does not trip the monitor.

Operating System

Linux x86_64 (container, 2 vCPU)

Models Used

Not relevant. The block happens before any model is used.

Package Versions
python==3.11
livekit-agents==1.8.9
livekit==1.1.21
livekit-plugins-noise-cancellation==0.3.2

I checked livekit 1.2.0 and the current main of agents, python-sdks and rust-sdks. The code path is unchanged.

Session/Room/Call IDs

No response

Proposed Solution

Any one of these removes the block from the call path:

  1. Create the filter session off the caller's thread. In livekit-ffi, call new_session() inside the spawned stream task (or on a blocking thread) and pass frames through until the session is ready. This is the same pattern as the fix for the Krisp VIVA filter in #7924. Alternatively, make NewAudioStream an async FFI request with a callback, so AudioStream can await it.
  2. Add a preload or prewarm API for the noise cancellation plugin, for example noise_cancellation.preload(BVCTelephony()), that loads the model in the process without a room. Agents can then call it in setup_fnc.
  3. Cache the loaded model per process in the native plugin (if it does not do this already), together with option 2.

Impact: the block happens right before the first greeting. It delays the first audio of nearly every call by about 250 ms. It also stops all other coroutines of the job (STT/LLM connection setup, telemetry, timers) during that time.

Additional Context

Related:

  • #7441: same stack with BVC(), 106–119 ms, open without a reply.
  • #7924: Krisp VIVA filter built on the event loop on the first frame (200–800 ms); confirmed and fixed.
  • livekit/rust-sdks#1163: audio filter on_load() blocks FfiRoom::connect with no timeout (another synchronous native filter call).

If you prefer one issue, feel free to close this one as a duplicate of #7441. The data here can move there.

Lingua principale
Python
Stelle
14.6k
Fork
3.9k
Merge medio
2g 7h
PR unite (30g)
228

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di livekit/agents

Tutte le issue di livekit/agents

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.