Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

[Plugin] livekit-plugins-sherpa: local TTS on sherpa-onnx (Kokoro, Kitten, Piper)

Aperta
  • Qualcosa non va?
#7,747 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@mahimairaja ci sta già lavorando.

Dal 11/10/2026.

  • #7749 di @mahimairaja — aperta

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
25/100
Tipo di issue
Funzionalità
Chiarezza
Specificata chiaramente
Stato di attività
Ferma
Stack tecnologico
python
Ambito
ai-infra-agents

Direzione di ricerca

The issue links a working implementation at mahimairaja/local-livekit under livekit-plugins/livekit-plugins-sherpa and reports 70 tests, including real-model speech and STT round trips through tts.StreamAdapter. Start by reading that implementation and its tests, then compare its package structure and API with the existing LiveKit agents plugins. Done means a sherpa-onnx TTS plugin supports the described models and streaming behavior; linked pull request #7749 indicates work is already underway.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Feature Type

Would make my life easier

Feature Description

Every TTS plugin in livekit/agents calls a hosted API. The common local route, Kokoro behind an OpenAI-compatible server through openai.TTS, breaks streaming (#6184), and it needs a separate service.

Proposal: TTS in the same livekit-plugins-sherpa package (STT side: #7746), on sherpa-onnx. In process, ONNX Runtime only, no torch or GPU, no system espeak (the phonemizer data ships with each model).

  • sherpa.TTS, Kokoro v1.0 by default (82M, int8, 54 voices as typed names). Kokoro's English phonemes come from misaki's dictionaries, with espeak-ng for unknown words.
  • voice takes the model's own speaker names or a speaker id, so other sherpa-onnx voices work too. Built in: Kitten nano (about 15M, 8 voices) and Piper en_US amy. engine= takes any sherpa_onnx.OfflineTts (VITS, Matcha, newer Kokoro).
  • streaming=False: AgentSession wraps it in tts.StreamAdapter. Each sentence's audio is pushed through AudioEmitter chunk by chunk as sherpa-onnx produces it, and an interruption stops generation at the next chunk.
  • Models are pinned to Hugging Face commits and fetched by download-files.

Tested on real models: Kokoro, Kitten nano and Piper amy each speak a sentence, and a local STT must transcribe it back (character error rate check), directly and through tts.StreamAdapter.

In a full local agent on an M-series Mac mini, CPU only: Kokoro's first audio arrives about 2.2 s into a 3.4 s voice-to-voice turn, so TTS is the largest share of latency.

Workarounds / Alternatives
  • openai.TTS(base_url=...) against a local Kokoro server: streaming breaks on the framing mismatch described in #6184, and it is an extra service.
  • A Kokoro-only plugin as asked in #6184 would fix one model; sherpa-onnx covers Kokoro, Kitten, Piper/VITS and Matcha with one dependency.
Additional Context

Working implementation, 70 tests, mypy strict: https://github.com/mahimairaja/local-livekit (livekit-plugins/livekit-plugins-sherpa)
Write-up with measurements: https://mahimai.ca/blog/local-livekit-voice-agent-offline

No word-level alignment (aligned_transcript=False): sherpa-onnx does not expose phoneme durations.

Happy to open a PR adding it with the STT.

Lingua principale
Python
Stelle
14.6k
Fork
3.9k
Merge medio
2g 6h
PR unite (30g)
153

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di livekit/agents

Tutte le issue di livekit/agents

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.