Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Live `AsyncSession.send_tool_response` fails with `TypeError` when `FunctionResponse.parts` carries inline bytes

Abierto
  • ¿Algo no está bien?
#3,022 1 comentario 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 1 día

@Venkaiahbabuneelam ya está trabajando en esto.

Desde el 30/9/2026.

  • #3024 de @kkkhs — abierto

Evaluación

Dificultad
3/5
Tiempo estimado
1-2 días
Aptitud para principiantes
72/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python
Área
api

Línea de trabajo

Start at AsyncSession.send_tool_response and compare its _common.convert_to_dict/json.dumps path with send_client_content's model_dump(mode='json') serialization. Reproduce the no-key example using t.t_tool_response and a FunctionResponse containing inline bytes; done means nested FunctionResponseBlob.data is base64-encoded and parts remain intact, with the deprecated AsyncSession.send path no longer dropping parts or scheduling.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

priority: p2 type: bug
Environment details
  • Programming language: Python 3.14 (also reproduced on 3.10)
  • OS: Linux
  • Package version: google-genai 2.25.0
Steps to reproduce

The Live API's function responses accept multimodal parts, and the Gemini 3.x Live models read an image sent that way. For example, gemini-3.8-live and gemini-3.1-flash-live-preview correctly describe a PNG, JPEG or WebP returned from a tool. However, AsyncSession.send_tool_response can't send such a response. It serializes with _common.convert_to_dict, which leaves FunctionResponseBlob.data as raw bytes, and then passes the result to json.dumps:

import asyncio

from google import genai
from google.genai import types


async def main():
    client = genai.Client()
    tool = types.Tool(function_declarations=[types.FunctionDeclaration(name='take_photo', description='Take a photo.')])
    config = types.LiveConnectConfig(response_modalities=['AUDIO'], tools=[tool])
    async with client.aio.live.connect(model='gemini-3.8-live', config=config) as session:
        await session.send_client_content(
            turns=types.Content(role='user', parts=[types.Part(text='Take a photo and describe it.')])
        )
        async for message in session.receive():
            if message.tool_call:
                call = message.tool_call.function_calls[0]
                await session.send_tool_response(
                    function_responses=types.FunctionResponse(
                        id=call.id,
                        name=call.name,
                        response={'output': 'Photo taken.'},
                        parts=[
                            types.FunctionResponsePart(
                                inline_data=types.FunctionResponseBlob(data=b'\x89PNG...', mime_type='image/png')
                            )
                        ],
                    )
                )
                break


asyncio.run(main())

TypeError: Object of type bytes is not JSON serializable

A minimal repro that needs no API key:

import json

from google.genai import _common, _transformers as t, types

response = types.FunctionResponse(
    id='call_1',
    name='take_photo',
    response={'output': 'Photo taken.'},
    parts=[types.FunctionResponsePart(inline_data=types.FunctionResponseBlob(data=b'\x89PNG', mime_type='image/png'))],
)
json.dumps({'tool_response': _common.convert_to_dict(t.t_tool_response(response), convert_keys=True)})
# TypeError: Object of type bytes is not JSON serializable
Expected behavior

The bytes are base64-encoded, as send_client_content does for Blob.data: it serializes with model_dump(mode='json'), which honors the models' ser_json_bytes='base64'.

Workarounds

  • Serialize the message with the SDK's own types and send it on the session's socket:

    message = types.LiveClientMessage(tool_response=types.LiveClientToolResponse(function_responses=[response]))
await session._ws.send(message.model_dump_json(by_alias=True, exclude_none=True))

    This works, but it reaches into the private _ws.

  • types.FunctionResponseBlob.model_construct(data=base64.b64encode(data).decode(), mime_type=...) also gets the message sent, but convert_to_dict then emits PydanticSerializationUnexpectedValue (Expected bytes) on every call.

Suggested fix

Serialize the tool response the way send_client_content does, with model_dump(mode='json', exclude_none=True), before json.dumps, so that nested bytes are base64-encoded.

The deprecated AsyncSession.send(input=FunctionResponse(...)) path has a related problem. It rebuilds each function response from name, response and id only, so it silently drops parts (and scheduling).

Lenguaje dominante
Python
Estrellas
4k
Forks
1k
Merge medio
1 d 17 h
PR fusionados (30 d)
52

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de googleapis/python-genai

Todos los issues de googleapis/python-genai

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.