vllm-project/guidellmPython
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
(1,166 stars) (156 forks) (0 件の索引済み issue) (0 件のオープンな good first issue)
リポジトリ
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
A high-throughput and memory-efficient inference and serving engine for LLMs