Repository metrics
- Stars
- (785 stars)
- PR merge metrics
- (PR metrics pending)
Description
Hi, as more and more LLM-based TTS systems pave the way, they show some remarkable results and expressiveness. Our next goal is to support such a model and make it run as fast as llama.cpp.
However, using llama.cpp to support a brand new model is cumbersome. So, let Rust help!
We are planning to support SparkTTS and Orpheus in the future!
We have a drafted inference engine based on Candle. On macOS, we see obvious speedup compared with PyTorch or ONNX. We can run a 0.5B LLM with more than 35 tokens/s on Mac M1. I believe once we push it further, these LLM-based TTS could be even faster.
As such, I strongly recommend that you guys join me if you are interested!
Please fork our current repo for massive development and support!
https://github.com/lucasjinreal/Crane
This project is not officially released yet, but please don't hesitate to join us at any time. SparkTTS and Orpheus are our goals!
updates
2025.03.26: I have supported SNAC audio decoder in Crane, Orpheus should have no obstacles.