Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

google realtime (Gemini Live): manual activity detection — barge-in fires on the model's own generation; unclosed transcript-only generation blocks later generate_reply

Aperta
  • Qualcosa non va?
#8,003 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@akashhguptaa ci sta già lavorando.

Dal 21/10/2026.

  • #8004 di @akashhguptaa — aperta

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
30/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Ferma
Stack tecnologico
google-cloud, python
Ambito
audio-video-rtc, backend-api-design

Direzione di ricerca

Start with the cited paths in realtime_api.py around lines 867–872, 926, 1383, 1476, and 1728, then trace how AgentSession handles input_speech_started and generation completion. Reproduce both reported cases with manual activity detection and transcription enabled, checking wire events and whether later generate_reply content is sent. Done means model initiated generations do not trigger user activity, transcript-only generations do not block later replies, and the supported behavior for open-activity transcription is documented or clarified. A linked open pull request (#8004) indicates work is already underway.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Versions / config: livekit-agents 1.8.8, livekit-plugins-google 1.8.8, google-genai 2.31.0, Python 3.13, Google AI API, gemini-3.8-live, realtime_input_config.automatic_activity_detection.disabled=True, input/output transcription on, AgentSession(turn_detection="manual"). Caller audio is pushed as frames; the client calls start_user_activity() before each clip. 20 instrumented runs, 2026-10-21, wire logged at the genai AsyncSession.

1. Barge-in fires on the model's own generation

Repro (client-content variant): frames → start_user_activity() → queue a user LiveClientContent(turn_complete=True) via _send_client_event; never send activity_end.

_start_new_generation with no pending future takes the "agent initiated" branch (realtime_api.py ~1383) and emits input_speech_started. AgentSession calls interrupt() (926), which calls start_user_activity() and sends activity_start. The server treats it as a barge-in and cuts the line.

  • Observed: first line cut in 3/3 runs ("Thank you for", then "Thank you for calling. How can I help" twice). Later lines are intact because an activity is already open.
  • Client-closed variant (client sends activity_end): the transcript arrives with no model turn (_is_new_generation counts input_transcription.text, 1728), a non-user-initiated generation starts, and activity_start goes out 1 ms later. With interrupt() stubbed, the interrupted line was re-spoken in 2 of 5 valid runs.
  • Expected: a model-initiated generation is not a user barge-in; no activity_start.

2. Unclosed transcript-only generation blocks later generate_reply

Repro: frames → start_user_activity() → client activity_end → server returns only an input transcription ("Hello?"), no model turn.

The plugin opens a generation for it. No turn_complete follows, and generations finish only on turn_complete (1476). Two later generate_reply(instructions=...) calls created speech handles, but no client content reached the wire; both ended as 25 s playout timeouts (2 of 3 asks, one run). Closing that generation by hand in later runs let 20/20 asks play. The blocking mechanism is inferred from logs, not traced.

Expected: a transcript-only generation is closed, or does not block later replies.

Question (not a bug)

With automatic detection off and activity_end never sent, input transcription is released only when generate_reply sends activity_end (867-872): 0.3-0.6 s after the ask, 0/44 before it (11 runs). The application cannot read the caller's turn before deciding whether to reply.

Is there a supported way to receive input transcription for an open activity, or to trigger a transcript-only turn, without the model generating?

Lingua principale
Python
Stelle
14.6k
Fork
3.9k
Merge medio
3g 18h
PR unite (30g)
193

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di livekit/agents

Tutte le issue di livekit/agents

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.