good first issueperformanceresearchð¬
Repository metrics
- Stars
- Â (49,693 stars)
- PR merge metrics
-  (å¹³åããŒãž 5d 20h) (30d 㧠36 merged PRs)
説æ
Christmas is coming soon, and I want to take some time to research something interesting, such as edge low-power inference. Although current whisper.cpp can run on Raspberry Pi, the inference performance cannot achieve real-time transcription. Fortunately, there are now some development boards that use processors with NPUs, which can be used to achieve real-time transcription of large models. My primary goal is to first support RK3566 and RK3588.
Roadmap:
- MatMul offloading
- Conv-Gelu offloading
- LayerNorm offloading ...