vllm-project/vllm-omni

Consider using pre-built Flash Attention kernels via `kernels`

Open

#3,922 opened on May 28, 2026

View on GitHub
 (5 comments) (0 reactions) (0 assignees)Python (1,067 forks)github user discovery
help wanted

Repository metrics

Stars
 (4,990 stars)
PR merge metrics
 (PR metrics pending)

Description

Hey,

I am Sayak from the Kernels team at Hugging Face. I noticed that this project uses Flash Attention which includes a long build time. We ship pre-built binaries (which provide bit-exact outputs as the upstream) and thereby, we make it easy to use.

Using FA3 on a supported machine is as easy as:

# make sure `kernels` is installed: `pip install -U kernels`
from kernels import get_kernel

kernel_module = get_kernel("kernels-community/flash-attn3")
flash_attn_func = kernel_module.flash_attn_func

flash_attn_func(...)

Let us know if you'd be interested in this and and we'd be happy to provide a draft of how it would look in your repo.

Contributor guide

Research direction
Investigate the current flash attention build steps in the project and evaluate integrating pre built binaries from the `kernels` package. Provide a draft PR or documentation for the change.
Tech stack
python
Domain
backendbuild systemperformance
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
PythonpipGit
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
50