kernel-builder: provide NCCL >= 2.32 headers and UCCL as build dependencies (for DeepEP)

Open
  • Is something wrong?
#949 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
backend, build-system

Research direction

Start with the DeepEP port in huggingface/kernels-community#1231 and inspect kernel-builder’s Nix packaging for CUDA and ROCm dependencies. Check how torch currently supplies NCCL headers and whether UCCL can be packaged for ROCm; done means CUDA kernels can use NCCL >= 2.32 headers and the UCCL dependency decision is made, with packaging if appropriate.

Written by the indexing model from the issue text.

Description

Context

huggingface/kernels-community#1231 ports DeepEP 2.5.0 as kernels-community/deep-ep to support the upcoming deepep expert-parallel strategy in Transformers. Right now the port has to vendor some dependencies that kernel-builder should probably provide instead.

NCCL

  • DeepEP 2.5's host code uses GIN and EFA-GDA APIs from NCCL 2.31/2.32.
  • The torch that kernel-builder ships comes with NCCL 2.30.4. That's too old to build against.
  • As a workaround, the port builds headers from an NCCL Git checkout pinned at 12df1a11afad322be5a204a2db890161cbf8131d (2.32.3) and uses them only at build time. The extension does not link or ship its own NCCL; it uses the symbols PyTorch has already loaded.
  • Ask: make NCCL >= 2.32 headers available in kernel-builder, for example by bumping the NCCL that comes with torch or by adding NCCL as a build dependency in the Nix setup. Then the vendored headers can be removed from the deep-ep port.

UCCL (AMD / ROCm)

  • We also want DeepEP on AMD GPUs, using the UCCL EP headers.
  • Ask: check whether UCCL can be added as a kernel-builder dependency for ROCm builds. This needs Nix packaging.

Acceptance

  • kernel-builder exposes NCCL >= 2.32 headers to CUDA kernels, so deep-ep can drop its vendored NCCL headers
  • Decide whether UCCL should be a kernel-builder dependency for ROCm, and package it if so

Requested by Daniël de Kok - Slack thread - Agent trace

Dominant language
Python
Stars
781
Forks
141
Avg merge
1d 8h
Merged PRs (30d)
65

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from huggingface/kernels

All issues in huggingface/kernels

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.