Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Realtime: a context copied while a text turn's push is in flight omits the user message, and the next update deletes it from the server

Chiusa
  • Qualcosa non va?
#7,906 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

  • #7910 di @Rehansanjay — chiusa senza merge

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
35/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Ferma
Stack tecnologico
python
Ambito
ai, backend

Direzione di ricerca

Start at voice/agent_activity.py around _realtime_reply_task and compare the text-turn ordering with the two tool-output paths near lines 4759–4808. Use the delayed-acknowledgement reproduction with openai.realtime.RealtimeModel to confirm that a copied context lacks the user message and later deletes it. Done means the context contains the message while its push is in flight, and the chosen behavior for push failures preserves consistency without leaving an invalid local turn.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Bug Description

With a realtime model, a text user turn adds its user message to the agent's own chat context only after its push to the server returns. While that push is in flight, agent.chat_ctx does not contain the message, so anything that copies the context in that window produces a context that is missing an item the server is about to acknowledge — and the next diff deletes it from the server conversation.

_realtime_reply_task, livekit-agents 1.8.8, voice/agent_activity.py:

msg = chat_ctx.add_message(role="user", content=user_input)   # on a copy of the mirror
try:
    await self._rt_session.update_chat_ctx(chat_ctx)          # :4164  the push
except ...
self._agent._chat_ctx._upsert_item(msg)                       # :4198  only now is it local
self._session._conversation_item_added(msg)

The interleaving:

  1. The text turn pushes hello and waits for the server to acknowledge it.
  2. Application code calls session.update_chat_ctx(agent.chat_ctx.copy()). That copy has no hello in it, because of the ordering above. AgentActivity.update_chat_ctx assigns self._agent._chat_ctx = chat_ctx immediately and then calls RealtimeSession.update_chat_ctx, which blocks on the plugin's _update_chat_ctx_lock.
  3. The text turn's ack lands, the mirror now contains hello, and the turn upserts it locally.
  4. The replacement acquires the plugin lock and diffs its own context object — still without hello — against a mirror that has it. hello appears in to_remove, so a conversation.item.delete goes out for it.

The local context keeps the message and the server conversation loses it, so the reply is generated without the user's turn.

Both tool-output paths already do this the other way round — self._agent._chat_ctx.insert(interrupted_fnc_outputs) at :4759 and the two _upsert_item calls at :4807-4808 run before their pushes, so a context copied mid-push contains those items and no diff can delete them. The text turn at :4198 is the only push that updates the local context after the fact.

This is not the same race as #7880 (overlapping pushes diffing stale snapshots of the server mirror, fixed by serializing snapshot and push in #7904 / #7905). Serializing does not help here: the replacement's context object is captured before it blocks, and no ordering of the pushes refreshes it.

Expected Behavior

A context copied from agent.chat_ctx while a text turn's push is in flight contains that user message, so a replacement push cannot delete an item the server is about to acknowledge. The agent's context and the server conversation stay in agreement.

Reproduction Steps
# Needs a provider whose update_chat_ctx diffs against an acknowledged mirror
# (the OpenAI realtime plugin) and a non-instant ack, so there is a window:
1. AgentSession with openai.realtime.RealtimeModel and a realtime provider that acks after ~50 ms.
2. session.generate_reply(user_input="hello")  — the push at agent_activity.py:4164 is now in flight.
3. Before it returns, from another task: await session.update_chat_ctx(session.current_agent.chat_ctx.copy())
4. Watch the client events: after the conversation.item.create for the user message is acked,
   the replacement sends conversation.item.delete for that same item.
5. The reply is generated from a server conversation with no user turn.
# With an instant ack there is no window and step 4 does not happen.
Possible fix

Hoisting self._agent._chat_ctx._upsert_item(msg) above the push makes the text turn consistent with the two tool-output paths, and is one line. It is not free, so it is worth deciding deliberately rather than just moving it:

  • _conversation_item_added is emitted from the same place, so hoisting changes when that event fires relative to the push.
  • On the unexpected-exception path the turn currently returns without recording the message at all (speech_handle._mark_done(error=e); return). Hoisting would leave it in the local context for a turn that failed, unless it is removed there.

Happy to send a PR once there is a view on which of those two behaviours should win.

Operating System

Linux and Windows (read from the source; the window depends only on ack latency)

Models Used

OpenAI Realtime via openai.realtime.RealtimeModel; any realtime provider that diffs against an acknowledged server mirror

Package Versions
livekit-agents==1.8.8
# the ordering at agent_activity.py:4198 is the same on main at ceb3d598f
Session/Room/Call IDs

None: found while fixing #7880.

Lingua principale
Python
Stelle
14.6k
Fork
3.9k
Merge medio
3g 18h
PR unite (30g)
193

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di livekit/agents

Tutte le issue di livekit/agents

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.