[Bug]: Executor failure right after its Task is enqueued leaves the task SUBMITTED on DatabaseTaskStore
Mantenedores costumam responder em até 2 dias
Avaliação
Esta issue ainda não foi avaliada.
Descrição
Follow-up to #1067, on a path #1106 did not cover.
What happened?
With DatabaseTaskStore, if an executor enqueues its Task and raises before that Task is saved, the failure write that #1106 added to _run_producer races the consumer's save of the Task. Both insert the same row and one of them fails:
- If the producer's insert fails, the error never reaches the consumer.
on_message_sendreturns the task asSUBMITTED, and the row staysSUBMITTED. - If the consumer's insert fails,
on_message_sendraisesIntegrityErrorinstead of the executor's error, and the stored task is the fallback, without the agent's metadata.
The window is about one store write, so it hits fast failures such as a refused connection, missing credentials or input validation, not failures after a model call. InMemoryTaskStore is not affected because its save never yields. Same on v1.2.2.
Reproduction
repro.py
import asyncio
import collections
import tempfile
import uuid
from sqlalchemy.ext.asyncio import create_async_engine
from a2a.helpers.proto_helpers import new_task_from_user_message
from a2a.server.agent_execution import AgentExecutor
from a2a.server.context import ServerCallContext
from a2a.server.request_handlers import DefaultRequestHandler
from a2a.server.tasks.database_task_store import DatabaseTaskStore
from a2a.types.a2a_pb2 import AgentCard, Message, Part, Role, SendMessageRequest, TaskState
class FailsAfterCreatingTask(AgentExecutor):
async def execute(self, context, event_queue):
task = new_task_from_user_message(context.message)
task.metadata.update({'agent_key': 'kept'})
self.task_id = task.id
await event_queue.enqueue_event(task)
raise RuntimeError('model call failed')
async def cancel(self, context, event_queue):
pass
async def main():
engine = create_async_engine(f'sqlite+aiosqlite:///{tempfile.mkdtemp()}/tasks.db')
store = DatabaseTaskStore(engine)
executor = FailsAfterCreatingTask()
handler = DefaultRequestHandler(executor, store, AgentCard(name='repro'))
outcomes = collections.Counter()
for _ in range(50):
message = Message(role=Role.ROLE_USER, message_id=str(uuid.uuid4()), parts=[Part(text='hi')])
context = ServerCallContext()
try:
result = await handler.on_message_send(SendMessageRequest(message=message), context)
client = f'returned {TaskState.Name(result.status.state)}'
except Exception as exc:
client = f'raised {type(exc).__name__}'
await asyncio.sleep(0.1)
stored = await store.get(executor.task_id, context)
kept = 'agent_key' in stored.metadata
outcomes[(client, TaskState.Name(stored.status.state), kept)] += 1
for (client, state, kept), n in outcomes.most_common():
print(f'{n:3} {client:30} stored {state:21} metadata kept: {kept}')
asyncio.run(main())
On main (01da00b), none of the 50 runs ends with the executor's error and a FAILED task:
30 returned TASK_STATE_SUBMITTED stored TASK_STATE_SUBMITTED metadata kept: True
19 raised IntegrityError stored TASK_STATE_FAILED metadata kept: False
1 raised RuntimeError stored TASK_STATE_FAILED metadata kept: False
Proposed fix
Make the consumer the only writer. The producer enqueues the failure instead of writing it, and the consumer writes FAILED after the events the executor enqueued before raising, creating the fallback task only if the executor never made one. An upsert in DatabaseTaskStore.save fixes the state but lets the fallback overwrite the agent's Task, and a lock still drops the agent's Task when the producer gets there first.
I have the change with a regression test that fails on main and passes with it, and can open a PR.
Environment
a2a-sdk main @ 01da00b and v1.2.2, Python 3.12, SQLAlchemy 2.1.5, aiosqlite 0.22.1, Linux.
- Linguagem predominante
- Python
- Estrelas
- 2.2k
- Forks
- 528
- Merge médio
- 3d 15h
- PRs com merge (30d)
- 45
Preparar o ambiente
- Sem Dockerfile nem arquivo Docker Compose
- Tem um modelo de pull request
- Ler o guia de contribuição
Primeiros passos
- Leia a issue inteira e depois o guia de contribuição do projeto.
- Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
- Faça um fork do repositório e trabalhe em uma branch.
- Abra um pull request que referencie o número da issue.
Mais de a2aproject/a2a-python
-
[Bug]: `A2A-Version` validation ignores minor versions and the query parameterTalvez já em andamento @rohityan assumiu há 68 dias. Abertacomponent: server status:awaiting response
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 75/100
a2aproject/a2a-python#1204 · 3 comentários · 1 responsável ·
Mantenedores costumam responder em até 2 dias
-
[Bug]: REST task/request id sanitizationTalvez já em andamento @Linux2010 assumiu há 127 dias. Abertamaintainers-only
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 65/100
a2aproject/a2a-python#805 · 1 comentário ·
Mantenedores costumam responder em até 2 dias
-
[Bug]: taskId-only continuation still fails required contextId inference in SDK 1.3.0 (follow-up to #1206)Talvez já em andamento @rohityan assumiu hoje. Aberta
a2aproject/a2a-python#1411 · 2 responsáveis ·
Mantenedores costumam responder em até 2 dias
-
[Feat]: Add a PermissionDenied server error with HTTP 403 and gRPC PERMISSION_DENIED mappingsTalvez já em andamento @rohityan assumiu hoje. Aberta
a2aproject/a2a-python#1410 · 2 responsáveis ·
Mantenedores costumam responder em até 2 dias
-
[Bug]: Blocking continuation returns stale INPUT_REQUIRED when executor emits current taskTalvez já em andamento @rohityan assumiu hoje. Aberta
a2aproject/a2a-python#1409 · 2 responsáveis ·
Mantenedores costumam responder em até 2 dias
Todas as issues de a2aproject/a2a-python
Issues semelhantes
-
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 82/100
Nixtla/mlforecast#823 ·
Mantenedores costumam responder em até 1 dia
-
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 68/100
Mantenedores costumam responder em até 1 dia
-
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 65/100
microsoft/ML-For-Beginners#1061 ·
-
bot-found documentation priority: P3
Dificuldade 2/5 Menos de uma hora Facilidade para iniciantes 82/100
madenvel/KalinkaPlayer#503 ·
-
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 63/100
EleutherAI/lm-evaluation-harness#4508 ·
Mantenedores costumam responder em até 1 dia