Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Audio routing freeze mid-call (1.6.9)

Aperta
  • Qualcosa non va?
#6,919 6 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
43/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
audio-video-rtc, backend

Direzione di ricerca

Start by tracing AudioRecognition audio routing and the STT stream lifecycle during multi-turn conversations, as identified in the issue. Compare what happens during successful turns with the freeze, especially around activity handoffs; the report confirms room audio frames continue arriving while STT/VAD receive none. Done means identifying and fixing the shared routing failure so user-state events and STT output continue across turns.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug

Description

Audio routing stops working mid-call. user_state stays stuck on "listening", no user_state_changed events are emitted, STT produces no output, and the call hangs up after inactivity prompts.

User continues speaking throughout (confirmed via separate audio stream subscription), but the audio doesn't reach STT/VAD/turn detection components.

Happens randomly during multi-turn conversations - not always on the same turn. Can't reproduce reliably but occurs frequently enough to block production deployment.

Expected vs Actual

Expected: Turn detection works every turn, user_state transitions normally, STT produces transcripts

Actual: Audio routing freezes, user_state stuck on "listening", no user_state_changed events, no STT output

Environment

  • Version: livekit-agents 1.6.9, livekit 1.1.14
  • Deployment: Self-hosted, outbound SIP
  • Models: Deepgram nova-3 (STT), gpt-4.6-luna (LLM), Deepgram aura-2 (TTS)

Comprehensive Testing

Tested multiple configurations to isolate the issue. All configurations exhibit the same freeze behavior.

Configuration 1: Silero VAD + Turn Detector v1-mini
vad=inference.VAD(model="silero", min_speech_duration=0.05, min_silence_duration=0.25, activation_threshold=0.4)
turn_handling=TurnHandlingOptions(
    turn_detection=inference.TurnDetector(version="v1-mini"),
    endpointing={"min_delay": 0.5, "max_delay": 1},
    interruption={"mode": "vad", "min_words": 3},
)

Result: Freezes mid-call

Configuration 2: No VAD + Turn Detector v1-mini
vad=None  # Explicitly disabled
turn_handling=TurnHandlingOptions(
    turn_detection=inference.TurnDetector(version="v1-mini"),
    endpointing={"min_delay": 0.5, "max_delay": 1},
    interruption={"mode": "vad", "min_words": 3},
)

Result: Freezes mid-call
Evidence: Logs confirm Session VAD: None

Configuration 3: No VAD + STT Endpointing
vad=None
turn_handling=TurnHandlingOptions(
    turn_detection="stt",  # Deepgram handles endpointing
    endpointing={"min_delay": 0.5, "max_delay": 1},
    interruption={"mode": "vad", "min_words": 3},
)

Result: Freezes mid-call

What This Proves

The bug is NOT caused by:

  • Silero VAD specifically (tested with vad=None)
  • Turn Detector v1-mini (tested with turn_detection="stt")
  • Corrupted VAD/STT instance state (tested with fresh instances per turn)
  • Network/SIP issues (confirmed audio frames arrive via separate subscription)

The bug appears to be a framework-level issue in:

  • Audio routing in AudioRecognition
  • Audio frame delivery to STT/VAD/turn detection components
  • Affects all turn detection methods equally

Additional Testing and Workarounds (All Failed)

Configuration changes tested:

  • Removed allow_interruptions=False from all session.say() / generate_reply() calls, replaced with temporary elevation of interruption.min_words threshold - no change
  • Switched from deprecated silero.VAD.load() to inference.VAD(model="silero") - no change
  • Switched from text-based turn_detector.multilingual.MultilingualModel to audio-based inference.TurnDetector(version="v1-mini") - no change
  • Created fresh VAD and STT instances per session (not shared across sessions) - still freezes

Runtime workarounds tested:

  1. Runtime VAD refresh - agent.update_options(vad=fresh_vad) when freeze detected - still freezes
  2. Override _update_vad() - Force fresh VAD instances on every activity - still freezes, logging shows VAD channel exists (closed=False) but emits no events
  3. Disable STT pipeline reuse - Force fresh STT WebSocket each time - still freezes

Evidence During Freeze

When freeze occurs:

  • user_state = "listening" (stuck)
  • agent_state = "listening"
  • No user_state_changed events emitted
  • No STT output/transcripts
  • Audio frames arrive at room (verified via rtc.AudioStream.from_participant())
  • VAD channel exists and closed=False (when VAD is used)

Example Timeline

Successful turns followed by freeze (common pattern)
16:04:51 - Agent speaks (turn 1)
16:04:53 - User response detected successfully
16:04:57 - Agent speaks (turn 2)
16:05:00 - User response detected successfully
16:05:03 - Agent speaks (turn 3)
16:05:06 - User response detected successfully
16:05:09 - Agent speaks (turn 4)
16:05:34 - Agent waits for user response
          [FREEZE OCCURS - audio routing stops]
          User continues speaking (confirmed via separate audio stream)
          No user_state_changed events emitted
          No STT transcription output
16:05:39 - WARNING: No user_state events for 5.0s agent_state=listening user_state=listening
16:05:41 - Agent: inactivity prompt 1
16:05:46 - WARNING: No user_state events for 5.0s
16:05:48 - Agent: inactivity prompt 2
16:05:53 - WARNING: No user_state events for 5.0s
16:05:55 - Agent: inactivity prompt 3
16:06:00 - WARNING: No user_state events for 5.0s
16:06:09 - Agent: farewell message and hangup

Note: The freeze can occur after any number of successful turns (sometimes turn 2, sometimes turn 5+). No consistent pattern to when it occurs.

Impact

Blocking production deployment. High call failure rate, poor user experience. Affects self-hosted deployments with outbound SIP.

Root Cause Analysis Needed

Audio frames reach the room but don't reach STT/VAD/turn detection. Need framework investigation of:

  1. AudioRecognition audio routing - How frames are routed to detection components during multi-turn conversations
  2. STT stream lifecycle - Why STT stops consuming frames mid-call
  3. Activity transitions - Whether audio routing breaks during activity handoffs

The fact that all turn detection methods fail identically suggests the issue is in the shared audio routing infrastructure, not in any specific detection component.

Lingua principale
Python
Stelle
14.6k
Fork
3.9k
Merge medio
3g 18h
PR unite (30g)
193

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di livekit/agents

Tutte le issue di livekit/agents

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.