Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

google realtime (Gemini Live): manual activity detection — barge-in fires on the model's own generation; unclosed transcript-only generation blocks later generate_reply

Abierto
  • ¿Algo no está bien?
#8,003 1 comentario 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

@akashhguptaa ya está trabajando en esto.

Desde el 21/10/2026.

  • #8004 de @akashhguptaa — abierto

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
30/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Estancado
Stack tecnológico
google-cloud, python
Área
audio-video-rtc, backend-api-design

Línea de trabajo

Start with the cited paths in realtime_api.py around lines 867–872, 926, 1383, 1476, and 1728, then trace how AgentSession handles input_speech_started and generation completion. Reproduce both reported cases with manual activity detection and transcription enabled, checking wire events and whether later generate_reply content is sent. Done means model initiated generations do not trigger user activity, transcript-only generations do not block later replies, and the supported behavior for open-activity transcription is documented or clarified. A linked open pull request (#8004) indicates work is already underway.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Versions / config: livekit-agents 1.8.8, livekit-plugins-google 1.8.8, google-genai 2.31.0, Python 3.13, Google AI API, gemini-3.8-live, realtime_input_config.automatic_activity_detection.disabled=True, input/output transcription on, AgentSession(turn_detection="manual"). Caller audio is pushed as frames; the client calls start_user_activity() before each clip. 20 instrumented runs, 2026-10-21, wire logged at the genai AsyncSession.

1. Barge-in fires on the model's own generation

Repro (client-content variant): frames → start_user_activity() → queue a user LiveClientContent(turn_complete=True) via _send_client_event; never send activity_end.

_start_new_generation with no pending future takes the "agent initiated" branch (realtime_api.py ~1383) and emits input_speech_started. AgentSession calls interrupt() (926), which calls start_user_activity() and sends activity_start. The server treats it as a barge-in and cuts the line.

  • Observed: first line cut in 3/3 runs ("Thank you for", then "Thank you for calling. How can I help" twice). Later lines are intact because an activity is already open.
  • Client-closed variant (client sends activity_end): the transcript arrives with no model turn (_is_new_generation counts input_transcription.text, 1728), a non-user-initiated generation starts, and activity_start goes out 1 ms later. With interrupt() stubbed, the interrupted line was re-spoken in 2 of 5 valid runs.
  • Expected: a model-initiated generation is not a user barge-in; no activity_start.

2. Unclosed transcript-only generation blocks later generate_reply

Repro: frames → start_user_activity() → client activity_end → server returns only an input transcription ("Hello?"), no model turn.

The plugin opens a generation for it. No turn_complete follows, and generations finish only on turn_complete (1476). Two later generate_reply(instructions=...) calls created speech handles, but no client content reached the wire; both ended as 25 s playout timeouts (2 of 3 asks, one run). Closing that generation by hand in later runs let 20/20 asks play. The blocking mechanism is inferred from logs, not traced.

Expected: a transcript-only generation is closed, or does not block later replies.

Question (not a bug)

With automatic detection off and activity_end never sent, input transcription is released only when generate_reply sends activity_end (867-872): 0.3-0.6 s after the ask, 0/44 before it (11 runs). The application cannot read the caller's turn before deciding whether to reply.

Is there a supported way to receive input transcription for an open activity, or to trigger a transcript-only turn, without the model generating?

Lenguaje dominante
Python
Estrellas
14.6k
Forks
3.9k
Merge medio
3 d 20 h
PR fusionados (30 d)
193

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de livekit/agents

Todos los issues de livekit/agents

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.