Manual turn detection: clear_user_turn() after commit_user_turn() resolves discards the committed turn
I maintainer di solito rispondono entro 1 giorno
@chenghao-mou ci sta già lavorando.
Dal 14/10/2026.
Valutazione
Questa issue non è ancora stata valutata.
Descrizione
Bug Description
In turn_detection="manual", the future returned by AgentSession.commit_user_turn() resolves with the correct transcript, but the turn that actually reaches end-of-turn handling can be empty. If clear_user_turn() is called after that future resolves, the turn is delivered with new_transcript == "". This happens whenever endpointing.min_delay > 0, including the default.
Nothing errors or logs. The commit future reports the transcript, and then the turn is silently dropped.
The cause is that the transcript is captured in a local variable but delivered from the instance field, which is read again later (voice/audio_recognition.py, 1.8.7):
_commit_user_turncapturestranscript = self._audio_transcript, calls_run_eou_detection(..., trigger="manual")(L1112), then resolves the future with that local value (L1119)._run_eou_detectiononly schedules_bounce_eou_task(L1839). That task sleepsmin_delay + (last_speaking_time - now)(L1741–1747), and after the sleep builds_EndOfTurnInfo(new_transcript=self._audio_transcript, ...)(L1769) from the live field._clear_user_turn(L1016) blanks_audio_transcriptand does not touch the pending_end_of_turn_task.
So whenever that sleep is positive, a clear_user_turn() issued after the future resolves always lands before the read. It isn't a flaky race: the sign of the sleep decides it every time. With last_speaking_time=None (VAD never fired) the sleep is the full min_delay.
Related to #7010, which hits the same post-commit endpointing window through a different trigger: there VAD START_OF_SPEECH cancels the pending task; here nothing is cancelled — the task completes and reads a transcript that clear_user_turn() has already blanked. The open fixes for #7010 (#7011, #7059) guard the cancellation path and do not change this outcome.
Expected Behavior
Once the commit_user_turn() future has resolved with a transcript, that transcript is the user turn, and a later clear_user_turn() only affects the next turn. The docstring says the future resolves "once STT is complete and end-of-turn detection has been triggered", which reads as "the turn is captured".
Reproduction Steps
- Create an
AudioRecognitionwithturn_detection="manual"and the default-likemin_delay=0.5. - Set up the state at push-to-talk release: a transcript is present and a FINAL just landed, so the commit takes its fast path.
awaitthe future returned bycommit_user_turn, then callclear_user_turn().- Observe that
on_end_of_turnreceives"", while the future resolved with the real transcript. Withmin_delay=0.0the transcript arrives intact.
Self-contained: no room, no audio, no API keys. It drives AudioRecognition directly; AgentSession.commit_user_turn() / clear_user_turn() reach the same two methods through AgentActivity.
import asyncio
import time
from livekit.agents import llm
from livekit.agents.voice import AgentSession
from livekit.agents.voice.audio_recognition import AudioRecognition
from livekit.agents.voice.endpointing import BaseEndpointing
TRANSCRIPT = "what's the weather tomorrow"
class Hooks:
def __init__(self):
self.seen = []
def on_end_of_turn(self, info):
self.seen.append(info.new_transcript)
return True
def retrieve_chat_ctx(self):
return llm.ChatContext.empty()
def __getattr__(self, _name): # remaining RecognitionHooks are unused here
return lambda *a, **k: None
async def idle_stt(audio, _settings):
async for _ in audio:
pass
return
yield
async def release_ptt(min_delay: float) -> tuple[str, list[str]]:
hooks = Hooks()
rec = AudioRecognition(
AgentSession(),
hooks=hooks,
endpointing=BaseEndpointing(min_delay=min_delay, max_delay=3.0),
stt=idle_stt,
vad=None,
interruption_detection=None,
turn_detection="manual",
)
rec._start()
try:
# State at button release: STT already delivered a FINAL, last speech 0.25s ago.
now = time.time()
rec._audio_transcript = TRANSCRIPT
rec._last_final_transcript_time = now
rec._last_speaking_time = now - 0.25
# What AgentSession.commit_user_turn() / clear_user_turn() call underneath.
fut = rec._commit_user_turn(
audio_detached=True, transcript_timeout=1.0, stt_flush_duration=2.0
)
committed = await fut
rec._clear_user_turn() # e.g. reset STT state for the next turn
await asyncio.sleep(min_delay + 0.2)
return committed, hooks.seen
finally:
await rec._aclose()
for min_delay in (0.5, 0.0):
committed, seen = asyncio.run(release_ptt(min_delay))
print(f"min_delay={min_delay}: future resolved with {committed!r}, end-of-turn got {seen}")
Output:
min_delay=0.5: future resolved with "what's the weather tomorrow", end-of-turn got ['']
min_delay=0.0: future resolved with "what's the weather tomorrow", end-of-turn got ["what's the weather tomorrow"]
Operating System
macOS (not OS-specific; pure asyncio ordering)
Models Used
Not model-specific; the repro uses no STT/LLM/TTS.
Package Versions
livekit==1.1.20
livekit-agents==1.8.7
livekit-api==1.2.1
python==3.12
Session/Room/Call IDs
N/A — reproduces locally without LiveKit Cloud.
Proposed Solution
Either of these would make a resolved commit mean a captured turn:
- Snapshot for manual commits. Pass the transcript captured in
_commit_user_turninto_run_eou_detection/_bounce_eou_task, and use it fornew_transcriptwhentrigger == "manual". The re-read after the sleep makes sense for VAD/STT triggers, where more speech can arrive during the delay, but a manual commit has already decided what the turn is. Skippingmin_delayfor manual triggers as well (there is no silence to wait out) removes the window entirely, which would also close the cancellation path in #7010. - Resolve later. Resolve the
commit_user_turn()future only afteron_end_of_turnhas run for that turn, so it really marks the end of end-of-turn handling.
Option 1 seems smaller and keeps the future's timing unchanged.
Additional Context
Why call clear_user_turn() right after the commit: it is the public way to reset STT state between push-to-talk turns, and it also rebuilds the STT pipeline so a late FINAL from the old stream can't leak into the next turn. Rebuilding a streaming STT connection can take a few hundred ms, so doing it on the next button press risks losing the user's first words; doing it right after the commit resolves moves that cost off the user's path. That is safe only if a resolved commit means the turn is captured.
Workaround: set endpointing.min_delay = 0.0 in manual mode, and wait for Agent.on_user_turn_completed before calling clear_user_turn() — even with no sleep, the reset and _bounce_eou_task can wake in the same event-loop iteration.
Screenshots and Recordings
N/A — the repro output above shows the behavior.
- Lingua principale
- Python
- Stelle
- 14.6k
- Fork
- 3.9k
- Merge medio
- 3g 1h
- PR unite (30g)
- 173
Preparare l'ambiente
Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di livekit/agents
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
I maintainer di solito rispondono entro 1 giorno
-
Assistant markup is stripped from chat history when expressive mode is offForse già presa @ShivamSharma43 l’ha presa 8 giorni fa. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
I maintainer di solito rispondono entro 1 giorno
-
docs: openai.LLM vs openai.responses.LLM for HTTP OpenAI-compat hostsForse di nuovo libera @AbarnaaSree l’ha presa 47 giorni fa e non c’è nessuna pull request aperta. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
livekit/agents#7022 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 25/100
I maintainer di solito rispondono entro 1 giorno
-
DuplexRealtimeAdapter: transcript fragment for audio that is never sent anchors the next burst, shifting transcripts by one utteranceForse già presa @dyyfk l’ha presa oggi. Aperta
Difficoltà 4/5 3-5 giorni Idoneità per principianti 45/100
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di livekit/agents
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
huggingface/sentence-transformers#4186 ·
I maintainer di solito rispondono entro 5 giorni
-
documentation good first issue
Difficoltà 2/5 Mezza giornata Idoneità per principianti 72/100
sujithputta02/Esapay#13 ·
-
bot-triaged module: correctness (silent) module: mps module: NaNs and Infs triage review
Difficoltà 2/5 Mezza giornata Idoneità per principianti 62/100
I maintainer di solito rispondono entro 1 giorno
-
README: CI link/badge checkAperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 62/100
SCIInstitute/SCIRun#2842 ·
I maintainer di solito rispondono entro 3 giorni
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100
I maintainer di solito rispondono entro 1 giorno