Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Manual turn detection: clear_user_turn() after commit_user_turn() resolves discards the committed turn

Abierto
#7,806 0 comentarios 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 1 día

@chenghao-mou ya está trabajando en esto.

Desde el 14/10/2026.

  • #7808 de @Hugo-DDT — cerrado sin fusionar
  • #7811 de @chenghao-mou — abierto

Evaluación

Este issue todavía no se ha evaluado.

Descripción

bug
Bug Description

In turn_detection="manual", the future returned by AgentSession.commit_user_turn() resolves with the correct transcript, but the turn that actually reaches end-of-turn handling can be empty. If clear_user_turn() is called after that future resolves, the turn is delivered with new_transcript == "". This happens whenever endpointing.min_delay > 0, including the default.

Nothing errors or logs. The commit future reports the transcript, and then the turn is silently dropped.

The cause is that the transcript is captured in a local variable but delivered from the instance field, which is read again later (voice/audio_recognition.py, 1.8.7):

  1. _commit_user_turn captures transcript = self._audio_transcript, calls _run_eou_detection(..., trigger="manual") (L1112), then resolves the future with that local value (L1119).
  2. _run_eou_detection only schedules _bounce_eou_task (L1839). That task sleeps min_delay + (last_speaking_time - now) (L1741–1747), and after the sleep builds _EndOfTurnInfo(new_transcript=self._audio_transcript, ...) (L1769) from the live field.
  3. _clear_user_turn (L1016) blanks _audio_transcript and does not touch the pending _end_of_turn_task.

So whenever that sleep is positive, a clear_user_turn() issued after the future resolves always lands before the read. It isn't a flaky race: the sign of the sleep decides it every time. With last_speaking_time=None (VAD never fired) the sleep is the full min_delay.

Related to #7010, which hits the same post-commit endpointing window through a different trigger: there VAD START_OF_SPEECH cancels the pending task; here nothing is cancelled — the task completes and reads a transcript that clear_user_turn() has already blanked. The open fixes for #7010 (#7011, #7059) guard the cancellation path and do not change this outcome.

Expected Behavior

Once the commit_user_turn() future has resolved with a transcript, that transcript is the user turn, and a later clear_user_turn() only affects the next turn. The docstring says the future resolves "once STT is complete and end-of-turn detection has been triggered", which reads as "the turn is captured".

Reproduction Steps
  1. Create an AudioRecognition with turn_detection="manual" and the default-like min_delay=0.5.
  2. Set up the state at push-to-talk release: a transcript is present and a FINAL just landed, so the commit takes its fast path.
  3. await the future returned by commit_user_turn, then call clear_user_turn().
  4. Observe that on_end_of_turn receives "", while the future resolved with the real transcript. With min_delay=0.0 the transcript arrives intact.

Self-contained: no room, no audio, no API keys. It drives AudioRecognition directly; AgentSession.commit_user_turn() / clear_user_turn() reach the same two methods through AgentActivity.

import asyncio
import time

from livekit.agents import llm
from livekit.agents.voice import AgentSession
from livekit.agents.voice.audio_recognition import AudioRecognition
from livekit.agents.voice.endpointing import BaseEndpointing

TRANSCRIPT = "what's the weather tomorrow"


class Hooks:
    def __init__(self):
        self.seen = []

    def on_end_of_turn(self, info):
        self.seen.append(info.new_transcript)
        return True

    def retrieve_chat_ctx(self):
        return llm.ChatContext.empty()

    def __getattr__(self, _name):  # remaining RecognitionHooks are unused here
        return lambda *a, **k: None


async def idle_stt(audio, _settings):
    async for _ in audio:
        pass
    return
    yield


async def release_ptt(min_delay: float) -> tuple[str, list[str]]:
    hooks = Hooks()
    rec = AudioRecognition(
        AgentSession(),
        hooks=hooks,
        endpointing=BaseEndpointing(min_delay=min_delay, max_delay=3.0),
        stt=idle_stt,
        vad=None,
        interruption_detection=None,
        turn_detection="manual",
    )
    rec._start()
    try:
        # State at button release: STT already delivered a FINAL, last speech 0.25s ago.
        now = time.time()
        rec._audio_transcript = TRANSCRIPT
        rec._last_final_transcript_time = now
        rec._last_speaking_time = now - 0.25

        # What AgentSession.commit_user_turn() / clear_user_turn() call underneath.
        fut = rec._commit_user_turn(
            audio_detached=True, transcript_timeout=1.0, stt_flush_duration=2.0
        )
        committed = await fut
        rec._clear_user_turn()  # e.g. reset STT state for the next turn

        await asyncio.sleep(min_delay + 0.2)
        return committed, hooks.seen
    finally:
        await rec._aclose()


for min_delay in (0.5, 0.0):
    committed, seen = asyncio.run(release_ptt(min_delay))
    print(f"min_delay={min_delay}: future resolved with {committed!r}, end-of-turn got {seen}")
Output:


min_delay=0.5: future resolved with "what's the weather tomorrow", end-of-turn got ['']
min_delay=0.0: future resolved with "what's the weather tomorrow", end-of-turn got ["what's the weather tomorrow"]
Operating System

macOS (not OS-specific; pure asyncio ordering)

Models Used

Not model-specific; the repro uses no STT/LLM/TTS.

Package Versions
livekit==1.1.20
livekit-agents==1.8.7
livekit-api==1.2.1
python==3.12
Session/Room/Call IDs

N/A — reproduces locally without LiveKit Cloud.

Proposed Solution

Either of these would make a resolved commit mean a captured turn:

  1. Snapshot for manual commits. Pass the transcript captured in _commit_user_turn into _run_eou_detection / _bounce_eou_task, and use it for new_transcript when trigger == "manual". The re-read after the sleep makes sense for VAD/STT triggers, where more speech can arrive during the delay, but a manual commit has already decided what the turn is. Skipping min_delay for manual triggers as well (there is no silence to wait out) removes the window entirely, which would also close the cancellation path in #7010.
  2. Resolve later. Resolve the commit_user_turn() future only after on_end_of_turn has run for that turn, so it really marks the end of end-of-turn handling.

Option 1 seems smaller and keeps the future's timing unchanged.

Additional Context

Why call clear_user_turn() right after the commit: it is the public way to reset STT state between push-to-talk turns, and it also rebuilds the STT pipeline so a late FINAL from the old stream can't leak into the next turn. Rebuilding a streaming STT connection can take a few hundred ms, so doing it on the next button press risks losing the user's first words; doing it right after the commit resolves moves that cost off the user's path. That is safe only if a resolved commit means the turn is captured.

Workaround: set endpointing.min_delay = 0.0 in manual mode, and wait for Agent.on_user_turn_completed before calling clear_user_turn() — even with no sleep, the reset and _bounce_eou_task can wake in the same event-loop iteration.

Screenshots and Recordings

N/A — the repro output above shows the behavior.

Lenguaje dominante
Python
Estrellas
14.6k
Forks
3.9k
Merge medio
3 d 1 h
PR fusionados (30 d)
173

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de livekit/agents

Todos los issues de livekit/agents

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.