AMD: option to classify in the background without pausing speech playout

Abierto
  • ¿Algo no está bien?
#7,689 2 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

@31groot ya está trabajando en esto.

Desde el 15/10/2026.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
18/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python
Área
ai, audio-video-rtc

Línea de trabajo

The pause happens in AMD._run via session._activity._pause_authorization(), and resumption happens only after execute() returns a verdict, so start by reading that method and the activity authorization logic it calls. Scope is open: the request needs a design decision on whether AMD runs without pausing speech and how amd_prediction is emitted afterwards. Done would mean the agent can speak during classification and the prediction still fires when ready. Ask a maintainer for the intended design before writing code.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Feature Type

Would make my life easier

Feature Description

AMD pauses speech playout authorization as soon as it starts (session._activity._pause_authorization() in AMD._run) and only resumes it when execute() returns a verdict. So the agent cannot say anything until the callee has spoken and the classifier has finished.

For outbound calls where the agent should speak first, this means the human picks up the phone and hears nothing until they say "hello?" themselves (and then still waits for the classifier, see #5616). We tried AMD and removed it for this reason.

What we want is for the agent to be able to speak right away, with AMD analysing in the background, and amd_prediction firing whenever the verdict is ready.

Today it is hard to have both: either the agent speaks first and we cannot tell whether a person or a voicemail picked up, or we know and the agent has to wait for the human to talk.

Workarounds / Alternatives
  • Construct AMD before session.start(): session._activity is still None, so nothing gets paused. This depends on internal ordering, not on a documented behaviour.
  • Call session._activity._resume_authorization() right after starting AMD. This is a private API.
  • Skip AMD and treat consent to recording, or the LLM noticing a voicemail greeting, as the "human answered" signal.
Additional Context
  • livekit-agents 1.8.1, outbound calls via Twilio Elastic SIP Trunking, create_sip_participant(wait_until_answered=True), OpenAI realtime model.
  • Related: #6125 (the need for an AMD signal on SIP calls) and #5616 (the latency execute() adds while playout is paused).
Lenguaje dominante
Python
Estrellas
14.6k
Forks
3.9k
Merge medio
2 d 7 h
PR fusionados (30 d)
224

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de livekit/agents

Todos los issues de livekit/agents

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.