Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Gemini Live: updates inside a tool miss its reply

Open
  • Is something wrong?
#7,917 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@longcw is already working on this.

Since Oct 19, 2026.

  • #7944 by @longcw — open

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
55/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
google-cloud, python
Domain
ai, backend

Research direction

The reproduction uses Agent.update_instructions(), Agent.update_tools(), and the Gemini realtime model; start by tracing how tool execution sends session.update and then submits the tool result. Compare the inside-tool and before-turn cases with the repro in the issue. Done means updates made inside a tool affect the reply to its result, or the realtime limitation is documented.

Written by the indexing model from the issue text.

Description

Bug Description

On Gemini Live through LiveKit Inference, update_instructions() and update_tools() called inside a function tool don't apply to the reply to that tool's result. LiveKit sends the session.update before the tool result, but Gemini answers with the old instructions and tools.

Agents that change phase from a tool hit this. For example, a confirm_ready tool that swaps the welcome instructions for the interview instructions gets a first question written without the interview instructions. A tool that adds a finish tool and asks the model to call it gets no call.

In the repro below, the unlock tool swaps in instructions containing a secret word, adds a finish tool, and returns "Tell the user the secret word from your instructions, then call finish.":

Updates made Knew the secret word Called finish
Inside the tool 0 of 4 0 of 4
Before the turn 4 of 4 4 of 4

When the updates are made inside the tool, the model replies "The session is now unlocked." and nothing more.

Expected Behavior

Instructions and tools updated inside a tool apply to the reply to that tool's result, as they do for chained LLMs. If Gemini Live can't apply them mid-turn, the SDK could hold the tool reply until the session.update has been applied.

Reproduction Steps
  1. Set LIVEKIT_API_KEY / LIVEKIT_API_SECRET.
  2. Run python repro.py inside (updates inside the tool), then python repro.py before (the same updates before the turn).
import asyncio
import sys

from livekit.agents import Agent, AgentSession, RunContext, function_tool, inference

INSIDE_TOOL = sys.argv[1] == "inside"

SECRET = "You are helping with the Alfred's study. The secret word is MARIGOLD."


class Assistant(Agent):
    def __init__(self):
        super().__init__(
            instructions="When the user asks to unlock, call the unlock tool."
        )

    @function_tool()
    async def unlock(self, context: RunContext) -> str:
        """Unlock the session when the user asks."""
        if INSIDE_TOOL:
            await self.update_instructions(SECRET)
            await self.update_tools([*self.tools, finish])
        return "Unlocked. Tell the user the secret word from your instructions, then call finish."


@function_tool()
async def finish() -> None:
    """Finish the session."""
    print("FINISH CALLED")


async def main():
    session = AgentSession(
        llm=inference.RealtimeModel(
            "google/gemini-3.8-live", modalities=["text"], turn_detection=None
        )
    )
    said = []
    session.on(
        "conversation_item_added",
        lambda ev: said.append(ev.item.text_content)
        if getattr(ev.item, "role", None) == "assistant"
        else None,
    )
    agent = Assistant()
    await session.start(agent=agent)
    if not INSIDE_TOOL:
        await agent.update_instructions(SECRET)
        await agent.update_tools([*agent.tools, finish])
    await session.generate_reply(user_input="Please unlock.")
    await asyncio.sleep(8)
    print("AGENT SAID:", said)
    await session.aclose()


asyncio.run(main())

Output:

## inside: AGENT SAID: ['The session is now unlocked.']
## before: FINISH CALLED AGENT SAID: ['The secret word is MARIGOLD.', 'The session is now finished.']
Operating System

macOS 15.2

Models Used

LiveKit Inference google/gemini-3.8-live, text output

Package Versions
livekit-agents==1.8.8
Session/Room/Call IDs

No response

Proposed Solution
Wait for the realtime session to acknowledge a `session.update` sent during tool execution before requesting the tool reply, or document that instruction and tool updates inside a tool only apply from the next turn on realtime models.
Additional Context

Related: #7915, #7916.

Screenshots and Recordings

No response

Dominant language
Python
Stars
14.6k
Forks
3.9k
Avg merge
3d 14h
Merged PRs (30d)
169

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from livekit/agents

All issues in livekit/agents

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.