Manual turn detection: clear_user_turn() after commit_user_turn() resolves discards the committed turn
Los mantenedores suelen responder en 1 día
@chenghao-mou ya está trabajando en esto.
Desde el 14/10/2026.
Evaluación
Este issue todavía no se ha evaluado.
Descripción
Bug Description
In turn_detection="manual", the future returned by AgentSession.commit_user_turn() resolves with the correct transcript, but the turn that actually reaches end-of-turn handling can be empty. If clear_user_turn() is called after that future resolves, the turn is delivered with new_transcript == "". This happens whenever endpointing.min_delay > 0, including the default.
Nothing errors or logs. The commit future reports the transcript, and then the turn is silently dropped.
The cause is that the transcript is captured in a local variable but delivered from the instance field, which is read again later (voice/audio_recognition.py, 1.8.7):
_commit_user_turncapturestranscript = self._audio_transcript, calls_run_eou_detection(..., trigger="manual")(L1112), then resolves the future with that local value (L1119)._run_eou_detectiononly schedules_bounce_eou_task(L1839). That task sleepsmin_delay + (last_speaking_time - now)(L1741–1747), and after the sleep builds_EndOfTurnInfo(new_transcript=self._audio_transcript, ...)(L1769) from the live field._clear_user_turn(L1016) blanks_audio_transcriptand does not touch the pending_end_of_turn_task.
So whenever that sleep is positive, a clear_user_turn() issued after the future resolves always lands before the read. It isn't a flaky race: the sign of the sleep decides it every time. With last_speaking_time=None (VAD never fired) the sleep is the full min_delay.
Related to #7010, which hits the same post-commit endpointing window through a different trigger: there VAD START_OF_SPEECH cancels the pending task; here nothing is cancelled — the task completes and reads a transcript that clear_user_turn() has already blanked. The open fixes for #7010 (#7011, #7059) guard the cancellation path and do not change this outcome.
Expected Behavior
Once the commit_user_turn() future has resolved with a transcript, that transcript is the user turn, and a later clear_user_turn() only affects the next turn. The docstring says the future resolves "once STT is complete and end-of-turn detection has been triggered", which reads as "the turn is captured".
Reproduction Steps
- Create an
AudioRecognitionwithturn_detection="manual"and the default-likemin_delay=0.5. - Set up the state at push-to-talk release: a transcript is present and a FINAL just landed, so the commit takes its fast path.
awaitthe future returned bycommit_user_turn, then callclear_user_turn().- Observe that
on_end_of_turnreceives"", while the future resolved with the real transcript. Withmin_delay=0.0the transcript arrives intact.
Self-contained: no room, no audio, no API keys. It drives AudioRecognition directly; AgentSession.commit_user_turn() / clear_user_turn() reach the same two methods through AgentActivity.
import asyncio
import time
from livekit.agents import llm
from livekit.agents.voice import AgentSession
from livekit.agents.voice.audio_recognition import AudioRecognition
from livekit.agents.voice.endpointing import BaseEndpointing
TRANSCRIPT = "what's the weather tomorrow"
class Hooks:
def __init__(self):
self.seen = []
def on_end_of_turn(self, info):
self.seen.append(info.new_transcript)
return True
def retrieve_chat_ctx(self):
return llm.ChatContext.empty()
def __getattr__(self, _name): # remaining RecognitionHooks are unused here
return lambda *a, **k: None
async def idle_stt(audio, _settings):
async for _ in audio:
pass
return
yield
async def release_ptt(min_delay: float) -> tuple[str, list[str]]:
hooks = Hooks()
rec = AudioRecognition(
AgentSession(),
hooks=hooks,
endpointing=BaseEndpointing(min_delay=min_delay, max_delay=3.0),
stt=idle_stt,
vad=None,
interruption_detection=None,
turn_detection="manual",
)
rec._start()
try:
# State at button release: STT already delivered a FINAL, last speech 0.25s ago.
now = time.time()
rec._audio_transcript = TRANSCRIPT
rec._last_final_transcript_time = now
rec._last_speaking_time = now - 0.25
# What AgentSession.commit_user_turn() / clear_user_turn() call underneath.
fut = rec._commit_user_turn(
audio_detached=True, transcript_timeout=1.0, stt_flush_duration=2.0
)
committed = await fut
rec._clear_user_turn() # e.g. reset STT state for the next turn
await asyncio.sleep(min_delay + 0.2)
return committed, hooks.seen
finally:
await rec._aclose()
for min_delay in (0.5, 0.0):
committed, seen = asyncio.run(release_ptt(min_delay))
print(f"min_delay={min_delay}: future resolved with {committed!r}, end-of-turn got {seen}")
Output:
min_delay=0.5: future resolved with "what's the weather tomorrow", end-of-turn got ['']
min_delay=0.0: future resolved with "what's the weather tomorrow", end-of-turn got ["what's the weather tomorrow"]
Operating System
macOS (not OS-specific; pure asyncio ordering)
Models Used
Not model-specific; the repro uses no STT/LLM/TTS.
Package Versions
livekit==1.1.20
livekit-agents==1.8.7
livekit-api==1.2.1
python==3.12
Session/Room/Call IDs
N/A — reproduces locally without LiveKit Cloud.
Proposed Solution
Either of these would make a resolved commit mean a captured turn:
- Snapshot for manual commits. Pass the transcript captured in
_commit_user_turninto_run_eou_detection/_bounce_eou_task, and use it fornew_transcriptwhentrigger == "manual". The re-read after the sleep makes sense for VAD/STT triggers, where more speech can arrive during the delay, but a manual commit has already decided what the turn is. Skippingmin_delayfor manual triggers as well (there is no silence to wait out) removes the window entirely, which would also close the cancellation path in #7010. - Resolve later. Resolve the
commit_user_turn()future only afteron_end_of_turnhas run for that turn, so it really marks the end of end-of-turn handling.
Option 1 seems smaller and keeps the future's timing unchanged.
Additional Context
Why call clear_user_turn() right after the commit: it is the public way to reset STT state between push-to-talk turns, and it also rebuilds the STT pipeline so a late FINAL from the old stream can't leak into the next turn. Rebuilding a streaming STT connection can take a few hundred ms, so doing it on the next button press risks losing the user's first words; doing it right after the commit resolves moves that cost off the user's path. That is safe only if a resolved commit means the turn is captured.
Workaround: set endpointing.min_delay = 0.0 in manual mode, and wait for Agent.on_user_turn_completed before calling clear_user_turn() — even with no sleep, the reset and _bounce_eou_task can wake in the same event-loop iteration.
Screenshots and Recordings
N/A — the repro output above shows the behavior.
- Lenguaje dominante
- Python
- Estrellas
- 14.6k
- Forks
- 3.9k
- Merge medio
- 3 d 1 h
- PR fusionados (30 d)
- 173
Preparar el entorno
Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de livekit/agents
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
Los mantenedores suelen responder en 1 día
-
Assistant markup is stripped from chat history when expressive mode is offPosiblemente ocupada @ShivamSharma43 la tomó hace 8 días. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
Los mantenedores suelen responder en 1 día
-
docs: openai.LLM vs openai.responses.LLM for HTTP OpenAI-compat hostsQuizá libre de nuevo @AbarnaaSree la tomó hace 47 días y no hay ningún pull request abierto. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
livekit/agents#7022 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
Dificultad 4/5 3-5 días Aptitud para principiantes 25/100
Los mantenedores suelen responder en 1 día
-
DuplexRealtimeAdapter: transcript fragment for audio that is never sent anchors the next burst, shifting transcripts by one utterancePosiblemente ocupada @dyyfk la tomó hoy. Abierto
Dificultad 4/5 3-5 días Aptitud para principiantes 45/100
Los mantenedores suelen responder en 1 día
Todos los issues de livekit/agents
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
huggingface/sentence-transformers#4186 ·
Los mantenedores suelen responder en 5 días
-
documentation
Dificultad 1/5 Menos de una hora Aptitud para principiantes 92/100
QuantEcon/QuantEcon.py#988 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
fabriziosalmi/certmate#1247 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
Los mantenedores suelen responder en 1 día
-
documentation good first issue
Dificultad 2/5 Medio día Aptitud para principiantes 72/100
sujithputta02/Esapay#13 ·