Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

Realtime: a context copied while a text turn's push is in flight omits the user message, and the next update deletes it from the server

Đã đóng
  • Có gì không ổn?
#7,906 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

  • #7910 của @Rehansanjay — đã đóng, không merge

Đánh giá

Độ khó
3/5
Thời gian dự kiến
1-2 ngày
Mức phù hợp với người mới
35/100
Loại issue
Lỗi
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Đình trệ
Công nghệ
python
Lĩnh vực
ai, backend

Hướng nghiên cứu

Start at voice/agent_activity.py around _realtime_reply_task and compare the text-turn ordering with the two tool-output paths near lines 4759–4808. Use the delayed-acknowledgement reproduction with openai.realtime.RealtimeModel to confirm that a copied context lacks the user message and later deletes it. Done means the context contains the message while its push is in flight, and the chosen behavior for push failures preserves consistency without leaving an invalid local turn.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Bug Description

With a realtime model, a text user turn adds its user message to the agent's own chat context only after its push to the server returns. While that push is in flight, agent.chat_ctx does not contain the message, so anything that copies the context in that window produces a context that is missing an item the server is about to acknowledge — and the next diff deletes it from the server conversation.

_realtime_reply_task, livekit-agents 1.8.8, voice/agent_activity.py:

msg = chat_ctx.add_message(role="user", content=user_input)   # on a copy of the mirror
try:
    await self._rt_session.update_chat_ctx(chat_ctx)          # :4164  the push
except ...
self._agent._chat_ctx._upsert_item(msg)                       # :4198  only now is it local
self._session._conversation_item_added(msg)

The interleaving:

  1. The text turn pushes hello and waits for the server to acknowledge it.
  2. Application code calls session.update_chat_ctx(agent.chat_ctx.copy()). That copy has no hello in it, because of the ordering above. AgentActivity.update_chat_ctx assigns self._agent._chat_ctx = chat_ctx immediately and then calls RealtimeSession.update_chat_ctx, which blocks on the plugin's _update_chat_ctx_lock.
  3. The text turn's ack lands, the mirror now contains hello, and the turn upserts it locally.
  4. The replacement acquires the plugin lock and diffs its own context object — still without hello — against a mirror that has it. hello appears in to_remove, so a conversation.item.delete goes out for it.

The local context keeps the message and the server conversation loses it, so the reply is generated without the user's turn.

Both tool-output paths already do this the other way round — self._agent._chat_ctx.insert(interrupted_fnc_outputs) at :4759 and the two _upsert_item calls at :4807-4808 run before their pushes, so a context copied mid-push contains those items and no diff can delete them. The text turn at :4198 is the only push that updates the local context after the fact.

This is not the same race as #7880 (overlapping pushes diffing stale snapshots of the server mirror, fixed by serializing snapshot and push in #7904 / #7905). Serializing does not help here: the replacement's context object is captured before it blocks, and no ordering of the pushes refreshes it.

Expected Behavior

A context copied from agent.chat_ctx while a text turn's push is in flight contains that user message, so a replacement push cannot delete an item the server is about to acknowledge. The agent's context and the server conversation stay in agreement.

Reproduction Steps
# Needs a provider whose update_chat_ctx diffs against an acknowledged mirror
# (the OpenAI realtime plugin) and a non-instant ack, so there is a window:
1. AgentSession with openai.realtime.RealtimeModel and a realtime provider that acks after ~50 ms.
2. session.generate_reply(user_input="hello")  — the push at agent_activity.py:4164 is now in flight.
3. Before it returns, from another task: await session.update_chat_ctx(session.current_agent.chat_ctx.copy())
4. Watch the client events: after the conversation.item.create for the user message is acked,
   the replacement sends conversation.item.delete for that same item.
5. The reply is generated from a server conversation with no user turn.
# With an instant ack there is no window and step 4 does not happen.
Possible fix

Hoisting self._agent._chat_ctx._upsert_item(msg) above the push makes the text turn consistent with the two tool-output paths, and is one line. It is not free, so it is worth deciding deliberately rather than just moving it:

  • _conversation_item_added is emitted from the same place, so hoisting changes when that event fires relative to the push.
  • On the unexpected-exception path the turn currently returns without recording the message at all (speech_handle._mark_done(error=e); return). Hoisting would leave it in the local context for a turn that failed, unless it is removed there.

Happy to send a PR once there is a view on which of those two behaviours should win.

Operating System

Linux and Windows (read from the source; the window depends only on ack latency)

Models Used

OpenAI Realtime via openai.realtime.RealtimeModel; any realtime provider that diffs against an acknowledged server mirror

Package Versions
livekit-agents==1.8.8
# the ordering at agent_activity.py:4198 is the same on main at ceb3d598f
Session/Room/Call IDs

None: found while fixing #7880.

Ngôn ngữ chính
Python
Star
14.6k
Fork
3.9k
Merge trung bình
3 ngày 18 giờ
Pull request đã merge (30 ngày)
193

Chuẩn bị môi trường

Dự án này không cung cấp dev container, Dockerfile hay hướng dẫn đóng góp, nên bạn cần tự thiết lập môi trường: hãy bắt đầu từ README và xem hướng dẫn đóng góp lần đầu của chúng tôi để biết các bước chung.

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của livekit/agents

Tất cả issue của livekit/agents

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.