Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

google realtime (Gemini Live): manual activity detection — barge-in fires on the model's own generation; unclosed transcript-only generation blocks later generate_reply

Open
  • Is something wrong?
#8,003 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@akashhguptaa is already working on this.

Since Oct 21, 2026.

  • #8004 by @akashhguptaa — open

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
30/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
google-cloud, python
Domain
audio-video-rtc, backend-api-design

Research direction

Start with the cited paths in realtime_api.py around lines 867–872, 926, 1383, 1476, and 1728, then trace how AgentSession handles input_speech_started and generation completion. Reproduce both reported cases with manual activity detection and transcription enabled, checking wire events and whether later generate_reply content is sent. Done means model initiated generations do not trigger user activity, transcript-only generations do not block later replies, and the supported behavior for open-activity transcription is documented or clarified. A linked open pull request (#8004) indicates work is already underway.

Written by the indexing model from the issue text.

Description

Versions / config: livekit-agents 1.8.8, livekit-plugins-google 1.8.8, google-genai 2.31.0, Python 3.13, Google AI API, gemini-3.8-live, realtime_input_config.automatic_activity_detection.disabled=True, input/output transcription on, AgentSession(turn_detection="manual"). Caller audio is pushed as frames; the client calls start_user_activity() before each clip. 20 instrumented runs, 2026-10-21, wire logged at the genai AsyncSession.

1. Barge-in fires on the model's own generation

Repro (client-content variant): frames → start_user_activity() → queue a user LiveClientContent(turn_complete=True) via _send_client_event; never send activity_end.

_start_new_generation with no pending future takes the "agent initiated" branch (realtime_api.py ~1383) and emits input_speech_started. AgentSession calls interrupt() (926), which calls start_user_activity() and sends activity_start. The server treats it as a barge-in and cuts the line.

  • Observed: first line cut in 3/3 runs ("Thank you for", then "Thank you for calling. How can I help" twice). Later lines are intact because an activity is already open.
  • Client-closed variant (client sends activity_end): the transcript arrives with no model turn (_is_new_generation counts input_transcription.text, 1728), a non-user-initiated generation starts, and activity_start goes out 1 ms later. With interrupt() stubbed, the interrupted line was re-spoken in 2 of 5 valid runs.
  • Expected: a model-initiated generation is not a user barge-in; no activity_start.

2. Unclosed transcript-only generation blocks later generate_reply

Repro: frames → start_user_activity() → client activity_end → server returns only an input transcription ("Hello?"), no model turn.

The plugin opens a generation for it. No turn_complete follows, and generations finish only on turn_complete (1476). Two later generate_reply(instructions=...) calls created speech handles, but no client content reached the wire; both ended as 25 s playout timeouts (2 of 3 asks, one run). Closing that generation by hand in later runs let 20/20 asks play. The blocking mechanism is inferred from logs, not traced.

Expected: a transcript-only generation is closed, or does not block later replies.

Question (not a bug)

With automatic detection off and activity_end never sent, input transcription is released only when generate_reply sends activity_end (867-872): 0.3-0.6 s after the ask, 0/44 before it (11 runs). The application cannot read the caller's turn before deciding whether to reply.

Is there a supported way to receive input transcription for an open activity, or to trigger a transcript-only turn, without the model generating?

Dominant language
Python
Stars
14.6k
Forks
3.9k
Avg merge
3d 18h
Merged PRs (30d)
193

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from livekit/agents

All issues in livekit/agents

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.