kernel-builder: provide NCCL >= 2.32 headers and UCCL as build dependencies (for DeepEP)

Aperta
  • Qualcosa non va?
#949 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
backend, build-system

Direzione di ricerca

Start with the DeepEP port in huggingface/kernels-community#1231 and inspect kernel-builder’s Nix packaging for CUDA and ROCm dependencies. Check how torch currently supplies NCCL headers and whether UCCL can be packaged for ROCm; done means CUDA kernels can use NCCL >= 2.32 headers and the UCCL dependency decision is made, with packaging if appropriate.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Context

huggingface/kernels-community#1231 ports DeepEP 2.5.0 as kernels-community/deep-ep to support the upcoming deepep expert-parallel strategy in Transformers. Right now the port has to vendor some dependencies that kernel-builder should probably provide instead.

NCCL

  • DeepEP 2.5's host code uses GIN and EFA-GDA APIs from NCCL 2.31/2.32.
  • The torch that kernel-builder ships comes with NCCL 2.30.4. That's too old to build against.
  • As a workaround, the port builds headers from an NCCL Git checkout pinned at 12df1a11afad322be5a204a2db890161cbf8131d (2.32.3) and uses them only at build time. The extension does not link or ship its own NCCL; it uses the symbols PyTorch has already loaded.
  • Ask: make NCCL >= 2.32 headers available in kernel-builder, for example by bumping the NCCL that comes with torch or by adding NCCL as a build dependency in the Nix setup. Then the vendored headers can be removed from the deep-ep port.

UCCL (AMD / ROCm)

  • We also want DeepEP on AMD GPUs, using the UCCL EP headers.
  • Ask: check whether UCCL can be added as a kernel-builder dependency for ROCm builds. This needs Nix packaging.

Acceptance

  • kernel-builder exposes NCCL >= 2.32 headers to CUDA kernels, so deep-ep can drop its vendored NCCL headers
  • Decide whether UCCL should be a kernel-builder dependency for ROCm, and package it if so

Requested by Daniël de Kok - Slack thread - Agent trace

Lingua principale
Python
Stelle
783
Fork
143
Merge medio
1g 10h
PR unite (30g)
57

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di huggingface/kernels

Tutte le issue di huggingface/kernels

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.