[LiteLLM] role='tool_responses' for Gemma 4 breaks tool calling on Vertex AI Model Garden MaaS (400 INVALID_ARGUMENT)

Offen Anfängerfreundlich
  • Stimmt etwas nicht?
#7,977 1 Kommentar 0 Reaktionen 1 zugewiesene Person Auf GitHub ansehen

Maintainer antworten meist innerhalb von 4 Tagen

@surajksharma07 arbeitet bereits daran.

Seit 04.11.2026.

  • #7978 von @rladeira — offen

Bewertung

Schwierigkeit
2/5
Geschätzter Aufwand
1-3 Stunden
Anfängerfreundlichkeit
72/100
Issue-Typ
Bug
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
gcp, python
Bereich
ai

Rechercherichtung

Der Fehler liegt in google/adk/models/lite_llm.py, in _is_gemma4_model und im Set _STANDARD_TOOL_ROLE_PROVIDERS, das entscheidet, ob Tool-Ergebnisse role="tool" oder role="tool_responses" erhalten. Beginne damit, nachzulesen, wie der effektive Provider für einen vertex_ai/-Modellstring abgeleitet wird, und prüfe danach die bestehenden Tests, die die Ausnahmen für openai, azure und lm_studio abdecken. Fertig ist es, wenn Gemma 4 auf vertex_ai role="tool" sendet, ollama und hosted_vllm weiterhin tool_responses behalten und ein Test den Fall vertex_ai abdeckt.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

🔴 Required Information

Describe the Bug:
_is_gemma4_model in google/adk/models/lite_llm.py causes ADK to emit tool results with role="tool_responses" whenever the model string matches gemma-?4, except when the effective provider is in _STANDARD_TOOL_ROLE_PROVIDERS ({"openai", "azure", "lm_studio"}, added in 7d190e6e for #6482).

When calling Gemma 4 on Vertex AI Model Garden MaaS via LiteLLM (model="vertex_ai/google/gemma-4-26b-a4b-it-maas"), LiteLLM routes the request to Vertex AI's OpenAI-compatible chat-completions endpoint (.../endpoints/openapi/chat/completions), which strictly validates OpenAI message roles. Because "vertex_ai" is not in _STANDARD_TOOL_ROLE_PROVIDERS, ADK sends role="tool_responses" and Vertex AI rejects every tool-result turn with HTTP 400 INVALID_ARGUMENT:

Unrecognized 'role' field for a message; found: 'tool_responses'.

Steps to Reproduce:

  1. Configure ADC with a GCP project that has Vertex AI API enabled.
  2. Instantiate LiteLlm(model="vertex_ai/google/gemma-4-26b-a4b-it-maas", vertex_project="<gcp-project>", vertex_location="global").
  3. Send a tool-calling round trip (user prompt -> model FunctionCall -> user FunctionResponse).
  4. Observe litellm.BadRequestError (400 INVALID_ARGUMENT).

Expected Behavior:
ADK sends tool results with role="tool" for vertex_ai Gemma 4 models, and Vertex AI returns a 200 OK completion grounded in the tool result.

Observed Behavior:
ADK sends role="tool_responses" and Vertex AI returns:

{
  "error": {
    "code": 400,
    "message": "Unrecognized 'role' field for a message; found: 'tool_responses'.",
    "status": "INVALID_ARGUMENT"
  }
}

Environment Details:

  • ADK Library Version (pip show google-adk): 2.13.0 and main (76627586)
  • Desktop OS: Linux
  • Python Version (python -V): Python 3.12

Model Information:

  • Are you using LiteLLM: Yes
  • Which model is being used: vertex_ai/google/gemma-4-26b-a4b-it-maas

🟡 Optional Information

Regression:
Introduced when gemma-?4 matching was broadened in 49301558 (#6334 / #6340); 7d190e6e (#6482 / #7042) excluded openai, azure, and lm_studio, but did not include vertex_ai.

Logs:

roles sent: ['user', 'assistant', 'tool_responses']
EXCEPTION: BadRequestError litellm.BadRequestError: Vertex_aiException BadRequestError - [{
  "error": {
    "code": 400,
    "message": "Unrecognized 'role' field for a message; found: 'tool_responses'.",
    "status": "INVALID_ARGUMENT"
  }
}]

Additional Context:
We also verified a self-deployed Vertex AI Model Garden Gemma 4 vLLM endpoint (google/gemma4@gemma-4-e2b-it via vertex_ai/gemma/... :predict with @requestFormat="chatCompletions" and dedicated /chat/completions): it accepts role="tool" with 200 OK and produces the expected final answer without re-calling the tool. Adding "vertex_ai" to _STANDARD_TOOL_ROLE_PROVIDERS therefore fixes Vertex MaaS without regressing self-deployed Vertex Model Garden endpoints, while preserving role="tool_responses" for ollama/... and hosted_vllm/....

Minimal Reproduction Code:

import asyncio
from google.adk.models.lite_llm import LiteLlm
from google.adk.models.llm_request import LlmRequest
from google.genai import types


async def main() -> None:
  model = "vertex_ai/google/gemma-4-26b-a4b-it-maas"
  llm = LiteLlm(
      model=model,
      vertex_project="<gcp-project>",
      vertex_location="global",
  )
  weather = types.FunctionDeclaration(
      name="get_weather",
      description="Returns the weather for a city.",
      parameters=types.Schema(
          type=types.Type.OBJECT,
          properties={"city": types.Schema(type=types.Type.STRING)},
          required=["city"],
      ),
  )
  req = LlmRequest(
      model=model,
      contents=[
          types.Content(
              role="user",
              parts=[types.Part(text="What is the weather in Paris?")],
          ),
          types.Content(
              role="model",
              parts=[
                  types.Part(
                      function_call=types.FunctionCall(
                          id="call_1",
                          name="get_weather",
                          args={"city": "Paris"},
                      )
                  )
              ],
          ),
          types.Content(
              role="user",
              parts=[
                  types.Part(
                      function_response=types.FunctionResponse(
                          id="call_1",
                          name="get_weather",
                          response={"result": "sunny, 21 C"},
                      )
                  )
              ],
          ),
      ],
      config=types.GenerateContentConfig(
          tools=[types.Tool(function_declarations=[weather])],
          max_output_tokens=64,
      ),
  )
  async for r in llm.generate_content_async(req):
    print(r.finish_reason, r.content)


asyncio.run(main())

How often has this issue occurred?:

  • Always (100%)
Vorherrschende Sprache
Python
Sterne
22.1k
Forks
4.3k
Ø Merge
1 T. 10 Std.
Gemergte PRs (30 T.)
5

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus google/adk-python

Alle Issues in google/adk-python

Ähnliche Issues

Weitere Issues zu Python

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.