Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

STT FallbackAdapter can't detect a stream that stays connected but stops transcribing

Abierto
  • ¿Algo no está bien?
#7,693 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

@rrfunde ya está trabajando en esto.

Desde el 14/10/2026.

  • #7826 de @rrfunde — abierto

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
22/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python
Área
ai, audio-video-rtc

Línea de trabajo

The payload names no files, only the candidate homes for the check: stt.FallbackAdapter, StreamAdapter, and AgentActivity, which holds both the VAD and the STT. Read how FallbackAdapter already turns errors into retries and fallback, and how #5648 handled a silently hanging TTS socket. The maintainers have not chosen a location, so the first step is to ask which one they prefer before writing code. Done means an opt-in liveness check that raises a retryable APIConnectionError after speech goes unanswered, and that ignores speech heard while the agent is talking.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Feature Type

Would make my life easier

Feature Description

A streaming STT can keep its WebSocket open and stop returning any events while audio still flows. We saw this with Gnani (livekit-agents 1.8.2): no events for 60 s while 16 kHz audio kept flowing, until the provider closed the session itself ("no speech segment received for 60 seconds"). The first words after the reconnect were transcribed at once. stt.FallbackAdapter only reacts to errors and timeouts, so the user is effectively muted for the whole time.

Proposal: an opt-in liveness check. When the session's VAD detects speech of at least min_speech and the STT emits no event during it or within timeout after it ends, treat the stream as failed (a retryable APIConnectionError). Retry and fallback then work as today. Optionally, replay the buffered audio of that speech into the new stream, so the words are not lost.

It could live in FallbackAdapter, StreamAdapter, or AgentActivity, which already holds both the VAD and the STT. Speech the VAD hears while the agent is talking should not count, since on a speaker line it can be the agent's own voice.

Happy to open a PR if you'd take it. Which place would you prefer?

Workarounds / Alternatives

We currently wrap each STT plugin in our own stt.STT subclass. Its stream watches the session VAD's events (through a VAD wrapper) and raises a retryable APIConnectionError when speech goes unanswered, then replays the buffered audio into the reopened stream. It works, but it duplicates retry plumbing that belongs in the framework.

Additional Context
  • The timeline from one call: last final at T; 60 s of steady audio frames and no STT event of any kind; provider closes the session (status 500, retryable); LiveKit reconnects; the next speech is transcribed within a second.
  • A related case that was handled: #5648 (a TTS WebSocket hanging silently).
Lenguaje dominante
Python
Estrellas
14.6k
Forks
3.9k
Merge medio
3 d 20 h
PR fusionados (30 d)
193

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de livekit/agents

Todos los issues de livekit/agents

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.