Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

[Plugin] livekit-plugins-sherpa: local TTS on sherpa-onnx (Kokoro, Kitten, Piper)

Abierto
  • ¿Algo no está bien?
#7,747 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

@mahimairaja ya está trabajando en esto.

Desde el 11/10/2026.

  • #7749 de @mahimairaja — abierto

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
25/100
Tipo de issue
Nueva funcionalidad
Claridad
Bien especificado
Estado de actividad
Estancado
Stack tecnológico
python
Área
ai-infra-agents

Línea de trabajo

The issue links a working implementation at mahimairaja/local-livekit under livekit-plugins/livekit-plugins-sherpa and reports 70 tests, including real-model speech and STT round trips through tts.StreamAdapter. Start by reading that implementation and its tests, then compare its package structure and API with the existing LiveKit agents plugins. Done means a sherpa-onnx TTS plugin supports the described models and streaming behavior; linked pull request #7749 indicates work is already underway.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Feature Type

Would make my life easier

Feature Description

Every TTS plugin in livekit/agents calls a hosted API. The common local route, Kokoro behind an OpenAI-compatible server through openai.TTS, breaks streaming (#6184), and it needs a separate service.

Proposal: TTS in the same livekit-plugins-sherpa package (STT side: #7746), on sherpa-onnx. In process, ONNX Runtime only, no torch or GPU, no system espeak (the phonemizer data ships with each model).

  • sherpa.TTS, Kokoro v1.0 by default (82M, int8, 54 voices as typed names). Kokoro's English phonemes come from misaki's dictionaries, with espeak-ng for unknown words.
  • voice takes the model's own speaker names or a speaker id, so other sherpa-onnx voices work too. Built in: Kitten nano (about 15M, 8 voices) and Piper en_US amy. engine= takes any sherpa_onnx.OfflineTts (VITS, Matcha, newer Kokoro).
  • streaming=False: AgentSession wraps it in tts.StreamAdapter. Each sentence's audio is pushed through AudioEmitter chunk by chunk as sherpa-onnx produces it, and an interruption stops generation at the next chunk.
  • Models are pinned to Hugging Face commits and fetched by download-files.

Tested on real models: Kokoro, Kitten nano and Piper amy each speak a sentence, and a local STT must transcribe it back (character error rate check), directly and through tts.StreamAdapter.

In a full local agent on an M-series Mac mini, CPU only: Kokoro's first audio arrives about 2.2 s into a 3.4 s voice-to-voice turn, so TTS is the largest share of latency.

Workarounds / Alternatives
  • openai.TTS(base_url=...) against a local Kokoro server: streaming breaks on the framing mismatch described in #6184, and it is an extra service.
  • A Kokoro-only plugin as asked in #6184 would fix one model; sherpa-onnx covers Kokoro, Kitten, Piper/VITS and Matcha with one dependency.
Additional Context

Working implementation, 70 tests, mypy strict: https://github.com/mahimairaja/local-livekit (livekit-plugins/livekit-plugins-sherpa)
Write-up with measurements: https://mahimai.ca/blog/local-livekit-voice-agent-offline

No word-level alignment (aligned_transcript=False): sherpa-onnx does not expose phoneme durations.

Happy to open a PR adding it with the STT.

Lenguaje dominante
Python
Estrellas
14.6k
Forks
3.9k
Merge medio
2 d 23 h
PR fusionados (30 d)
177

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de livekit/agents

Todos los issues de livekit/agents

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.