Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

STT turn detection: Google STT without word offsets (chirp_3) reports `end_time = start_time_offset`, which freezes the end-of-speech anchor, disables `min_delay` and drops turn metrics

Cerrado
  • ¿Algo no está bien?
#7,684 1 comentario 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Ya se ha fusionado un pull request relacionado.

  • #7687 de @15054538509 — fusionado

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
18/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
google-cloud, python
Área
audio-video-rtc

Línea de trabajo

Start in the plugin's stt.py around lines 1172-1181, where a result with text but no word offsets returns start_time_offset as end_time, then read voice/audio_recognition.py around lines 1198-1218 and 1256-1257 where that value overrides the VAD anchor. Pull request #7687 is already open against this issue, so check it before starting. Done means min_delay applies again and the turn metrics are present for chirp_3 without word offsets.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Summary

With turn_detection="stt" and the Google plugin on chirp_3, every final transcript carries an end_time that is not the end of speech: it is the stream's start_time_offset. Since 1.8.0 (#6557) STT turn detection trusts any end_time > 0 as the provider's speech end and overwrites the VAD anchor with it. The result is that the user's "stopped speaking" time sits at the moment the STT stream opened, for every turn of the session.

Where
  • livekit-plugins-google stt.py (1.8.4 / 1.8.5, lines 1172-1181): when a result has text but no words — always the case on chirp_3, which forces enable_word_time_offsets=False (lines 230-235) — it returns SpeechData(start_time=start_time_offset, end_time=start_time_offset, ...).
  • livekit-agents 1.8.5 voice/audio_recognition.py:
    • lines 1198-1202: has_stt_end_time is true for any end_time > 0;
    • lines 1214-1218: in "stt" mode that is enough for use_stt_speaking_time, even when a VAD anchor exists;
    • lines 1256-1257: _last_speaking_time is overwritten on every FINAL_TRANSCRIPT.

start_time_offset is small but positive (the stream opens a few ms after the input starts), so the check passes.

Effects
  1. min_delay never applies. extra_sleep = endpointing_delay + last_speaking_time - time.time() (lines 1730-1732) is negative by however long the session has been running, so endpointing never waits.
  2. Turn metrics disappear. last_speaking_time < speech_start_time trips the stale-anchor guard (around line 109), so the user message gets no stopped_speaking_at, end_of_turn_delay or transcription_delay, and the assistant message gets no e2e_latency.

In 1.6.2 the same plugin output was harmless: the provider time replaced the anchor only when there was no VAD (or the VAD missed the segment).

Reproduction
  • livekit-agents 1.8.5, livekit-plugins-google 1.8.4, google.STT(model="chirp_3", enable_voice_activity_events=True), a VAD (inference.VAD), turn_detection="stt", SIP caller.
  • Five turns in one call: the DEBUG line user turn committed shows the same last_speaking_time on all five (the session's start), earlier than each turn's speech_start_time.
  • No e2e_latency on any assistant message. On 1.6.2, same setup, every turn has it.
Suggested fix (either)
  • Plugin: when there are no word offsets, leave start_time / end_time at 0 rather than the stream offset, so the SDK knows it has no provider timing.
  • SDK: in STT mode, accept the provider time only when it is plausible (end_time + input_started_at >= speech_start_time of the current turn); otherwise keep the VAD anchor.
Lenguaje dominante
Python
Estrellas
14.6k
Forks
3.9k
Merge medio
2 d 17 h
PR fusionados (30 d)
207

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de livekit/agents

Todos los issues de livekit/agents

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.