Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Manual turn detection: clear_user_turn() after commit_user_turn() resolves discards the committed turn

Aperta
#7,806 0 commenti 0 reazioni 1 assegnatario Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@chenghao-mou ci sta già lavorando.

Dal 14/10/2026.

  • #7808 di @Hugo-DDT — chiusa senza merge
  • #7811 di @chenghao-mou — aperta

Valutazione

Questa issue non è ancora stata valutata.

Descrizione

bug
Bug Description

In turn_detection="manual", the future returned by AgentSession.commit_user_turn() resolves with the correct transcript, but the turn that actually reaches end-of-turn handling can be empty. If clear_user_turn() is called after that future resolves, the turn is delivered with new_transcript == "". This happens whenever endpointing.min_delay > 0, including the default.

Nothing errors or logs. The commit future reports the transcript, and then the turn is silently dropped.

The cause is that the transcript is captured in a local variable but delivered from the instance field, which is read again later (voice/audio_recognition.py, 1.8.7):

  1. _commit_user_turn captures transcript = self._audio_transcript, calls _run_eou_detection(..., trigger="manual") (L1112), then resolves the future with that local value (L1119).
  2. _run_eou_detection only schedules _bounce_eou_task (L1839). That task sleeps min_delay + (last_speaking_time - now) (L1741–1747), and after the sleep builds _EndOfTurnInfo(new_transcript=self._audio_transcript, ...) (L1769) from the live field.
  3. _clear_user_turn (L1016) blanks _audio_transcript and does not touch the pending _end_of_turn_task.

So whenever that sleep is positive, a clear_user_turn() issued after the future resolves always lands before the read. It isn't a flaky race: the sign of the sleep decides it every time. With last_speaking_time=None (VAD never fired) the sleep is the full min_delay.

Related to #7010, which hits the same post-commit endpointing window through a different trigger: there VAD START_OF_SPEECH cancels the pending task; here nothing is cancelled — the task completes and reads a transcript that clear_user_turn() has already blanked. The open fixes for #7010 (#7011, #7059) guard the cancellation path and do not change this outcome.

Expected Behavior

Once the commit_user_turn() future has resolved with a transcript, that transcript is the user turn, and a later clear_user_turn() only affects the next turn. The docstring says the future resolves "once STT is complete and end-of-turn detection has been triggered", which reads as "the turn is captured".

Reproduction Steps
  1. Create an AudioRecognition with turn_detection="manual" and the default-like min_delay=0.5.
  2. Set up the state at push-to-talk release: a transcript is present and a FINAL just landed, so the commit takes its fast path.
  3. await the future returned by commit_user_turn, then call clear_user_turn().
  4. Observe that on_end_of_turn receives "", while the future resolved with the real transcript. With min_delay=0.0 the transcript arrives intact.

Self-contained: no room, no audio, no API keys. It drives AudioRecognition directly; AgentSession.commit_user_turn() / clear_user_turn() reach the same two methods through AgentActivity.

import asyncio
import time

from livekit.agents import llm
from livekit.agents.voice import AgentSession
from livekit.agents.voice.audio_recognition import AudioRecognition
from livekit.agents.voice.endpointing import BaseEndpointing

TRANSCRIPT = "what's the weather tomorrow"


class Hooks:
    def __init__(self):
        self.seen = []

    def on_end_of_turn(self, info):
        self.seen.append(info.new_transcript)
        return True

    def retrieve_chat_ctx(self):
        return llm.ChatContext.empty()

    def __getattr__(self, _name):  # remaining RecognitionHooks are unused here
        return lambda *a, **k: None


async def idle_stt(audio, _settings):
    async for _ in audio:
        pass
    return
    yield


async def release_ptt(min_delay: float) -> tuple[str, list[str]]:
    hooks = Hooks()
    rec = AudioRecognition(
        AgentSession(),
        hooks=hooks,
        endpointing=BaseEndpointing(min_delay=min_delay, max_delay=3.0),
        stt=idle_stt,
        vad=None,
        interruption_detection=None,
        turn_detection="manual",
    )
    rec._start()
    try:
        # State at button release: STT already delivered a FINAL, last speech 0.25s ago.
        now = time.time()
        rec._audio_transcript = TRANSCRIPT
        rec._last_final_transcript_time = now
        rec._last_speaking_time = now - 0.25

        # What AgentSession.commit_user_turn() / clear_user_turn() call underneath.
        fut = rec._commit_user_turn(
            audio_detached=True, transcript_timeout=1.0, stt_flush_duration=2.0
        )
        committed = await fut
        rec._clear_user_turn()  # e.g. reset STT state for the next turn

        await asyncio.sleep(min_delay + 0.2)
        return committed, hooks.seen
    finally:
        await rec._aclose()


for min_delay in (0.5, 0.0):
    committed, seen = asyncio.run(release_ptt(min_delay))
    print(f"min_delay={min_delay}: future resolved with {committed!r}, end-of-turn got {seen}")
Output:


min_delay=0.5: future resolved with "what's the weather tomorrow", end-of-turn got ['']
min_delay=0.0: future resolved with "what's the weather tomorrow", end-of-turn got ["what's the weather tomorrow"]
Operating System

macOS (not OS-specific; pure asyncio ordering)

Models Used

Not model-specific; the repro uses no STT/LLM/TTS.

Package Versions
livekit==1.1.20
livekit-agents==1.8.7
livekit-api==1.2.1
python==3.12
Session/Room/Call IDs

N/A — reproduces locally without LiveKit Cloud.

Proposed Solution

Either of these would make a resolved commit mean a captured turn:

  1. Snapshot for manual commits. Pass the transcript captured in _commit_user_turn into _run_eou_detection / _bounce_eou_task, and use it for new_transcript when trigger == "manual". The re-read after the sleep makes sense for VAD/STT triggers, where more speech can arrive during the delay, but a manual commit has already decided what the turn is. Skipping min_delay for manual triggers as well (there is no silence to wait out) removes the window entirely, which would also close the cancellation path in #7010.
  2. Resolve later. Resolve the commit_user_turn() future only after on_end_of_turn has run for that turn, so it really marks the end of end-of-turn handling.

Option 1 seems smaller and keeps the future's timing unchanged.

Additional Context

Why call clear_user_turn() right after the commit: it is the public way to reset STT state between push-to-talk turns, and it also rebuilds the STT pipeline so a late FINAL from the old stream can't leak into the next turn. Rebuilding a streaming STT connection can take a few hundred ms, so doing it on the next button press risks losing the user's first words; doing it right after the commit resolves moves that cost off the user's path. That is safe only if a resolved commit means the turn is captured.

Workaround: set endpointing.min_delay = 0.0 in manual mode, and wait for Agent.on_user_turn_completed before calling clear_user_turn() — even with no sleep, the reset and _bounce_eou_task can wake in the same event-loop iteration.

Screenshots and Recordings

N/A — the repro output above shows the behavior.

Lingua principale
Python
Stelle
14.6k
Fork
3.9k
Merge medio
3g 1h
PR unite (30g)
173

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di livekit/agents

Tutte le issue di livekit/agents

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.