vllm-project/vllm

[Misc]: CMake Clean-up / Refactor Tasks

Open

#9,129 opened on Oct 7, 2024

View on GitHub
 (14 comments) (0 reactions) (0 assignees)Python (16,816 forks)batch import
good first issuehelp wantedkeep-openunstale

Repository metrics

Stars
 (80,034 stars)
PR merge metrics
 (Avg merge 3d 17h) (993 merged PRs in 30d)

Description

In an effort to make the CMake more readable, stable and easy to use we have a few tasks we'd like to work on, creating a GitHub issue here to track that progress, some planned changes/investigations:

  • Rename define_gpu_extension_target, currently this is used for CPU extensions too so the name is now misleading
  • Add a CI test of local builds, i.e. pip install -e .
  • Warn that PTX builds are not currently supported (post https://github.com/vllm-project/vllm/pull/8845), currently if there is a +PTX in TORCH_CUDA_ARCH_LIST this will be ignored. We should warn when this is the case. Alternatively we can add support for PTX builds although this is generally not desirable since PTX increases the wheel size by quite a bit (PTX is larger than SASS), and we already build for all currently supported arches.
  • Have vllm-flash-attn use ExternalProject currently vllm-flash-attn uses the parent CMake scope which creates many footguns since it is in a separate repo, using ExternalProject will mean that the vllm-flash-attn will be run in a separate CMake scope/process
    • It might be even better to revert to the FlashMLA approach, where CMake code lives in vllm.
  • Look into removing early returns in CMakeLists.txt (potentially move backends into its own files)
  • Potential build both C++ and CUDA extensions when building for CUDA and using torch dispatcher to dispatch between the two, https://github.com/vllm-project/vllm/pull/8424

Contributor guide