Google Realtime (Vertex AI): update_instructions() appends a model turn instead of replacing the system instruction

The bug is in update_instructions in livekit/plugins/google/realtime/realtime_api.py, where the turn is sent with a hard-coded role="model". Read that method and the reproduction script in the issue, which compares prompt-token counts across roles on Vertex AI. Work is already under way in linked PR #7802, so check that before starting. Done means a Vertex-only role change plus a unit test covering vertexai=True and False.

Bug Description

On Vertex AI, RealtimeSession.update_instructions() sends new instructions to a live session as a client_content turn with role="model" (livekit/plugins/google/realtime/realtime_api.py, update_instructions). That does not change the session's system instruction:

• the previous instructions stay in context;
• the new instructions are only appended as a model turn, which the model may ignore;
• context-window compression (SlidingWindow) can later drop that turn, while the old system instruction is always kept.

So after update_instructions() the session runs on the old instructions plus a conversation turn, not on the new instructions.

Google documents a mid-session update that does replace it. From Start and manage live sessions → Update system instructions during a session (https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/live-api/start-manage-session):

To update the system instructions mid-session, you can send text content with the system role. The updated system instruction will remain in effect for the remaining session.

The code comment next to the role says "Both APIs error on role="system"". On Vertex AI I see no error: the server accepts the turn and swaps the instruction. The evidence is below.

Expected Behavior

On Vertex AI, after update_instructions(new) on an active session, the session should run on new as its system instruction, with the previous instructions gone. This is what mutable_instructions=True implies, and what happens when instructions are set before connecting.

Reproduction Steps

The script below uses only `google-genai` (no LiveKit) to show what the server does with each role. It measures the prompt tokens of the same user turn in three Vertex AI sessions:

- **fresh B:** `system_instruction=B`
- **A + system B:** `system_instruction=A`, then a `role="system"` turn with B
- **A + model B:** `system_instruction=A`, then a `role="model"` turn with B (what `update_instructions` sends today)

Instruction A is about 577 tokens and contains a fact (a backup word) that B does not.

Results, identical in two runs:

| Model | fresh B | A + system B | A + model B |
|---|---|---|---|
| `gemini-live-2.5-flash-native-audio` | 34 | **34** | 611 |
| `gemini-live-2.5-flash` | 34 | **34** | 611 |
| `gemini-3.8-live` | 34 | **34** | 611 |

- **`role="system"`:** the prompt is exactly the size of a fresh session on B, so A is gone. The model also no longer knows A's backup word.
- **`role="model"`:** A is still there (+577 tokens). On `gemini-live-2.5-flash` the model kept answering from A in both runs: *"The secret code is APPLE-17 and the backup word is CHERRY."*


"""Does a mid-session role="system" turn replace the system instruction on Vertex AI Live?

    GOOGLE_CLOUD_PROJECT=... GOOGLE_CLOUD_LOCATION=us-central1 python repro_system_role.py [model ...]
"""

import asyncio
import os
import sys

from google import genai
from google.genai import types

FILLER = " ".join(f"Note {i}: this sentence only adds length to instruction A." for i in range(40))
A = (
    "You are assistant A. The secret code is APPLE-17. The backup word is CHERRY. "
    "Answer in one short sentence. " + FILLER
)
B = "You are assistant B. The secret code is BANANA-42. Answer in one short sentence."
Q = "What is the secret code, and what is the backup word?"


async def ask(client: genai.Client, model: str, system: str, update_role: str | None = None):
    config = types.LiveConnectConfig(
        response_modalities=[types.Modality.AUDIO],
        system_instruction=types.Content(parts=[types.Part(text=system)]),
        output_audio_transcription=types.AudioTranscriptionConfig(),
    )
    async with client.aio.live.connect(model=model, config=config) as session:
        if update_role:
            await session.send_client_content(
                turns=types.Content(role=update_role, parts=[types.Part(text=B)]),
                turn_complete=False,
            )
        await session.send_client_content(
            turns=types.Content(role="user", parts=[types.Part(text=Q)]), turn_complete=True
        )
        tokens, said = None, ""
        async for msg in session.receive():  # ends at turn_complete
            if msg.usage_metadata and msg.usage_metadata.prompt_token_count:
                tokens = msg.usage_metadata.prompt_token_count
            sc = msg.server_content
            if sc and sc.output_transcription and sc.output_transcription.text:
                said += sc.output_transcription.text
        return tokens, said.strip()


async def main() -> None:
    client = genai.Client(
        vertexai=True,
        project=os.environ["GOOGLE_CLOUD_PROJECT"],
        location=os.environ.get("GOOGLE_CLOUD_LOCATION", "us-central1"),
    )
    for model in sys.argv[1:] or ["gemini-live-2.5-flash-native-audio", "gemini-3.8-live"]:
        results = await asyncio.gather(
            ask(client, model, B), ask(client, model, A, "system"), ask(client, model, A, "model")
        )
        print(model)
        for name, (tokens, said) in zip(("fresh B", "A + system B", "A + model B"), results):
            print(f"  {name:13} prompt_tokens={tokens!s:<6} reply={said!r}")


if __name__ == "__main__":
    asyncio.run(main())


I also confirmed this inside a LiveKit `AgentSession` on Vertex AI with production-size prompts (about 1k tokens, replaced mid-call by about 17k), on `gemini-live-2.5-flash` and `gemini-3.8-live`:

- With `role="system"`, the prompt-token count after the update equalled a fresh session started on the new prompt.
- The new instruction survived `SlidingWindow` compression. The same text sent as a `user` or `model` turn was dropped by the window.
- There were no errors or connection closes.

Operating System

macOS (agent host); Vertex AI us-central1

Models Used

Vertex AI Live: gemini-live-2.5-flash-native-audio, gemini-live-2.5-flash, gemini-3.8-live

Package Versions

livekit-agents==1.7.1
livekit-plugins-google==1.7.1
google-genai==2.19.0


`update_instructions` still hard-codes `role="model"` in `livekit-plugins-google` 1.8.6

Session/Room/Call IDs

Not applicable: reproduced with the standalone script above

Proposed Solution

Send `role="system"` on Vertex AI and keep `role="model"` for the Gemini API. In `update_instructions` (1.8.6):


                     turns=[
                         types.Content(
                             parts=[types.Part(text=instructions)],
-                            # Both APIs error on role="system". This was role=None on the
-                            # Gemini API, which 2.5 accepted as the system role but 3.1 and
-                            # 3.8 reject with a 1007 close that kills the session. "model"
-                            # is accepted by all three and by Vertex.
-                            role="model",
+                            # On Vertex AI, role="system" replaces the session's system
+                            # instruction (see "Update system instructions during a session"
+                            # in the Vertex AI Live API docs). role="model" is only appended
+                            # as a model turn, so the previous instructions stay in effect.
+                            # The Gemini API errors on role="system" (and 3.1/3.8 close with
+                            # 1007 on role=None), so it keeps role="model".
+                            role="system" if self._opts.vertexai else "model",
                         )
                     ],
                     turn_complete=False,


- **Scope:** only `vertexai=True` with an active session. Instructions set before connecting still go into the setup message.
- **Also covers handoffs:** when `AgentActivity` reuses a realtime session across agents and pushes the new agent's instructions, those would then really take effect on Vertex.
- **Not tested by me:** the Gemini API (unchanged by the proposal), and 3.1 models on Vertex.

Happy to open a PR with this change, plus a unit test asserting the role sent for `vertexai=True` and `False`.

Additional Context

• google-genai's Content.role docstring says "Must be either 'user' or 'model'", but Vertex AI accepts "system" on client_content as documented
• Workaround until this is fixed: subclass RealtimeSession, override update_instructions to send role="system" on Vertex, and subclass RealtimeModel.session() to return it. This is what I use for now

Screenshots and Recordings