Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Google plugin: the model does not stay silent after a tool returns `None` on Vertex AI

Aperta
  • Qualcosa non va?
#7,660 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@aayushbaluni ci sta già lavorando.

Dal 8/10/2026.

  • #7664 di @aayushbaluni — aperta

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
25/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Ferma
Stack tecnologico
gcp, python
Ambito
ai

Direzione di ricerca

Start with the Gemini-only scheduling: SILENT condition in livekit-plugins/livekit-plugins-google/livekit/plugins/google/utils.py (around lines 81-83) and realtime/realtime_api.py (around lines 729-745). The goal is to send scheduling on Vertex AI for a NON_BLOCKING tool that returns None. Run the linked repro_vertex_silent.py gist to confirm the model stays silent. A pull request (#7664) is already open for this, so check it before starting.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug
Bug Description

In LiveKit, a tool that returns None needs no reply. On Vertex AI, the model replies to it, even when the tool is NON_BLOCKING.

What I observed

I ran 10 trials for each cell on 2026-10-08, with Vertex AI in us-central1.

Model on Vertex AI BLOCKING NON_BLOCKING
gemini-3.8-live replied 10/10 replied 10/10
gemini-live-2.5-flash-native-audio replied 10/10 replied 10/10

What the plugin sends

The plugin keeps the model silent with scheduling: SILENT, but only on the Gemini API (utils.py, realtime_api.py):

API tool_behavior scheduling for a tool that returns None
Gemini API NON_BLOCKING SILENT
Gemini API BLOCKING, or not set not sent
Vertex AI any not sent

On Vertex AI, the plugin logs "pass tool_behavior=NON_BLOCKING to keep it silent" even for NON_BLOCKING tools.

Vertex AI supports scheduling now

The plugin skips Vertex AI because of #3784 (2025-11). In that issue, gemini-live-2.5-flash-preview-native-audio-09-2025 on Vertex AI rejected scheduling with 1007 Unknown name "scheduling". #3793 fixed it with the comment "vertexai currently doesn't support the scheduling parameter". #6785 then added scheduling: SILENT only on the Gemini API.

The model from #3784 is no longer available on Vertex AI. The Vertex AI documentation now lists scheduling (Asynchronous function calling). When the NON_BLOCKING tool response included scheduling: SILENT, both models above accepted it and stayed silent in 10 of 10 trials, and no session ended with an error.

Expected Behavior

On Vertex AI, the plugin sends scheduling: SILENT for a NON_BLOCKING tool that returns None, as on the Gemini API, and the model stays silent.

Reproduction Steps
1. Create `google.realtime.RealtimeModel(vertexai=True, model="gemini-3.8-live", tool_behavior=types.Behavior.NON_BLOCKING)`.
2. Give the agent a function tool that returns `None`.
3. Send a user message that makes the model call the tool.
4. The tool returns `None`. The plugin logs "a tool result wants no reply, but Gemini will answer it anyway; pass tool_behavior=NON_BLOCKING to keep it silent", and the model replies.

I used this script for the repeated trials in the description: `repro_vertex_silent.py` in https://gist.github.com/pokutuna/ff5a12222f44632d9fd8f5d7c722364f
Run it with: `uv run repro_vertex_silent.py`
Each API needs its own credentials. The script skips an API without them:
  Gemini API: `GOOGLE_API_KEY`
  Vertex AI:  `GOOGLE_CLOUD_PROJECT` and Application Default Credentials
Operating System

macOS 26

Models Used

Google RealtimeModel on Vertex AI: gemini-3.8-live, gemini-live-2.5-flash-native-audio

Package Versions
livekit-agents==1.8.5
livekit-plugins-google==1.8.5
google-genai==2.28.0
Session/Room/Call IDs

(None. I ran the script locally, without LiveKit Cloud. Each session has one user turn.)

Proposed Solution
Send `scheduling` on Vertex AI too, in the same way as on the Gemini API.
I can open a pull request for this.
Additional Context

What the proposed change affects

The proposed change only removes the Vertex AI condition from the plugin. Nothing changes on the Gemini API.

Case on Vertex AI Now With the change
NON_BLOCKING tool returns None no scheduling, the model replies scheduling: SILENT, the model stays silent
NON_BLOCKING tool returns a value no scheduling no change
BLOCKING, or tool_behavior not set no scheduling no change
the user sets tool_response_scheduling dropped, with a warning sent

gemini-3.8-live uses NON_BLOCKING by default, so with the default settings its tools that return None become silent. On Vertex AI, I tested only SILENT, not the other tool_response_scheduling values.

Notes

  • In each trial, the user gives one answer as text, and the model calls a tool. The tool returns None after 5 seconds, and the script watches for speech from the model for 15 seconds. The delay keeps out speech that some models make while a NON_BLOCKING tool runs.
  • Not tested: Vertex AI regions other than us-central1, and audio input.
  • A separate problem: the plugin also sends scheduling: SILENT to gemini-3.8-live-extended-thinking (Gemini API), because 3.8 models use NON_BLOCKING by default. This model does not support scheduling (model page), so the server closed the session with 1007 "Function response scheduling is not supported for this model." in 10 of 10 trials. The same script shows this when GOOGLE_API_KEY is set.
  • Related: #7302
Screenshots and Recordings

(None)

Lingua principale
Python
Stelle
14.6k
Fork
3.9k
Merge medio
2g 23h
PR unite (30g)
177

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di livekit/agents

Tutte le issue di livekit/agents

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.