Gemini Live: updates inside a tool miss its reply

オープン
  • 何か問題がありますか？
#7,917 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

@longcw がすでに取り組んでいます。

2026年10月19日 から。

  • #7944 @longcw による — オープン

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
55/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
活発
技術スタック
google-cloud, python
領域
ai, backend

調査の方向性

The reproduction uses Agent.update_instructions(), Agent.update_tools(), and the Gemini realtime model; start by tracing how tool execution sends session.update and then submits the tool result. Compare the inside-tool and before-turn cases with the repro in the issue. Done means updates made inside a tool affect the reply to its result, or the realtime limitation is documented.

索引モデルが issue の本文から書いたものです。

説明

Bug Description

On Gemini Live through LiveKit Inference, update_instructions() and update_tools() called inside a function tool don't apply to the reply to that tool's result. LiveKit sends the session.update before the tool result, but Gemini answers with the old instructions and tools.

Agents that change phase from a tool hit this. For example, a confirm_ready tool that swaps the welcome instructions for the interview instructions gets a first question written without the interview instructions. A tool that adds a finish tool and asks the model to call it gets no call.

In the repro below, the unlock tool swaps in instructions containing a secret word, adds a finish tool, and returns "Tell the user the secret word from your instructions, then call finish.":

Updates made Knew the secret word Called finish
Inside the tool 0 of 4 0 of 4
Before the turn 4 of 4 4 of 4

When the updates are made inside the tool, the model replies "The session is now unlocked." and nothing more.

Expected Behavior

Instructions and tools updated inside a tool apply to the reply to that tool's result, as they do for chained LLMs. If Gemini Live can't apply them mid-turn, the SDK could hold the tool reply until the session.update has been applied.

Reproduction Steps
  1. Set LIVEKIT_API_KEY / LIVEKIT_API_SECRET.
  2. Run python repro.py inside (updates inside the tool), then python repro.py before (the same updates before the turn).
import asyncio
import sys

from livekit.agents import Agent, AgentSession, RunContext, function_tool, inference

INSIDE_TOOL = sys.argv[1] == "inside"

SECRET = "You are helping with the Alfred's study. The secret word is MARIGOLD."


class Assistant(Agent):
    def __init__(self):
        super().__init__(
            instructions="When the user asks to unlock, call the unlock tool."
        )

    @function_tool()
    async def unlock(self, context: RunContext) -> str:
        """Unlock the session when the user asks."""
        if INSIDE_TOOL:
            await self.update_instructions(SECRET)
            await self.update_tools([*self.tools, finish])
        return "Unlocked. Tell the user the secret word from your instructions, then call finish."


@function_tool()
async def finish() -> None:
    """Finish the session."""
    print("FINISH CALLED")


async def main():
    session = AgentSession(
        llm=inference.RealtimeModel(
            "google/gemini-3.8-live", modalities=["text"], turn_detection=None
        )
    )
    said = []
    session.on(
        "conversation_item_added",
        lambda ev: said.append(ev.item.text_content)
        if getattr(ev.item, "role", None) == "assistant"
        else None,
    )
    agent = Assistant()
    await session.start(agent=agent)
    if not INSIDE_TOOL:
        await agent.update_instructions(SECRET)
        await agent.update_tools([*agent.tools, finish])
    await session.generate_reply(user_input="Please unlock.")
    await asyncio.sleep(8)
    print("AGENT SAID:", said)
    await session.aclose()


asyncio.run(main())

Output:

## inside: AGENT SAID: ['The session is now unlocked.']
## before: FINISH CALLED AGENT SAID: ['The secret word is MARIGOLD.', 'The session is now finished.']
Operating System

macOS 15.2

Models Used

LiveKit Inference google/gemini-3.8-live, text output

Package Versions
livekit-agents==1.8.8
Session/Room/Call IDs

No response

Proposed Solution
Wait for the realtime session to acknowledge a `session.update` sent during tool execution before requesting the tool reply, or document that instruction and tool updates inside a tool only apply from the next turn on realtime models.
Additional Context

Related: #7915, #7916.

Screenshots and Recordings

No response

主要言語
Python
スター
14.6k
フォーク
3.9k
平均マージ
2日 7時間
マージ済み PR（30日）
224

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

livekit/agents のほかの issue

livekit/agents の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。