This repo hosts code for vLLM CI & Performance Benchmark infrastructure.
(42 stars) (68 forks) (0 indexed issues) (0 open good first issues)
Repositories
vllm-project repositories
Fast and memory-efficient exact attention
(124 stars) (148 forks) (0 indexed issues) (0 open good first issues)
vllm-project/guidellmPython
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
(1,166 stars) (156 forks) (0 indexed issues) (0 open good first issues)
vllm-project/recipesJavaScript
Common recipes to run vLLM
(833 stars) (292 forks) (0 indexed issues) (0 open good first issues)
System Level Intelligent Router for Mixture-of-Models at Cloud, Data Center and Edge
(4,293 stars) (699 forks) (145 indexed issues) (129 open good first issues)
TPU inference for vLLM, with unified JAX and PyTorch support.
(412 stars) (292 forks) (5 indexed issues) (5 open good first issues)
vllm-project/vllmPython
A high-throughput and memory-efficient inference and serving engine for LLMs
(80,034 stars) (16,816 forks) (61 indexed issues) (41 open good first issues)
Community maintained hardware plugin for vLLM on Ascend
(2,689 stars) (2,101 forks) (80 indexed issues) (77 open good first issues)
vllm-project/vllm-ncclPython
Manages vllm-nccl dependency
(18 stars) (3 forks) (0 indexed issues) (0 open good first issues)
vllm-project/vllm-omniPython
A framework for efficient model inference with omni-modality models
(4,990 stars) (1,067 forks) (153 indexed issues) (133 open good first issues)
(43 stars) (101 forks) (0 indexed issues) (0 open good first issues)