Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Multi-tensor swizzle kernels fail with "too many resources requested for launch" (missing __launch_bounds__)

Aperta
  • Qualcosa non va?
#3,621 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

@ravimajeti ci sta già lavorando.

Dal 4/10/2026.

  • #3622 di @ravimajeti — aperta

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
25/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
cpp
Ambito
backend

Direzione di ricerca

The four multi-tensor kernels live in transformer_engine/common/swizzle/swizzle.cu and are launched from launch_multi_tensor_swizzle_scaling_factors (the line in the report's stack trace); #2076 added launch_bounds(TB_DIM * TB_DIM) to the single-tensor kernels, so compare those declarations first. Done means the row, row and col variants stay within 64 registers at 1024 threads and the MultiTensorSwizzleTestSuite cases pass when run via tests/cpp/build/operator/test_operator --gtest_filter='MultiTensorSwizzleTestSuite'. Note PR #3622 is already open against this issue, so coordinate before starting.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Describe the bug

The multi-tensor swizzle kernels in transformer_engine/common/swizzle/swizzle.cu (multi_tensor_swizzle_{row,col}_scaling_kernel and multi_tensor_unswizzle_{row,col}_scaling_kernel) are launched with 1024 threads per block (dim3 block_size(TB_DIM, TB_DIM)), but unlike every other swizzle kernel they have no __launch_bounds__(TB_DIM * TB_DIM). Without it, ptxas may give a thread more than 64 registers. 1024 threads × more than 64 registers exceeds the 64K registers available to a block, so the launch fails with:

CUDA Error: too many resources requested for launch

#2076 added __launch_bounds__ to the single-tensor swizzle kernels; the multi-tensor kernels added a week earlier in #2019 were not included.

Whether it fails depends on the GPU arch and CUDA version, because the register count does. Measured with cuobjdump --dump-resource-usage on swizzle.cu compiled with the TE build flags (registers per thread; > 64 cannot launch with 1024 threads):

Kernel CUDA 12.8 sm_90 CUDA 12.8 sm_100 CUDA 12.8 sm_120 CUDA 13.4 sm_90 CUDA 13.4 sm_100 CUDA 13.4 sm_120
multi_tensor_swizzle_row_scaling_kernel<int4> 89 89 96 61 50 56
multi_tensor_swizzle_row_scaling_kernel<int2> 32 61 71 32 61 70
multi_tensor_swizzle_col_scaling_kernel<int4> 89 99 99 64 56 61
other multi-tensor (un)swizzle variants ≤ 56 ≤ 40 ≤ 48 ≤ 55 ≤ 40 ≤ 48

So on sm_120 it fails with both CUDA versions, and with CUDA 12.8 (the minimum for Blackwell) it should also fail on sm_90 and sm_100.

Steps/Code to reproduce bug

RTX 5090 (CC 12.0), TE built with NVTE_CUDA_ARCHS=120, CUDA 12.8:

tests/cpp/build/operator/test_operator --gtest_filter='*MultiTensorSwizzleTestSuite*'

3 cases fail, all routed to multi_tensor_swizzle_row_scaling_kernel<int4>:

[  FAILED  ] OperatorTest/MultiTensorSwizzleTestSuite.TestMultiTensorSwizzle/n2_M128_K1024_row
[  FAILED  ] OperatorTest/MultiTensorSwizzleTestSuite.TestMultiTensorSwizzle/n3_M256_K4096_row
[  FAILED  ] OperatorTest/MultiTensorSwizzleTestSuite.TestMultiTensorSwizzle/n2_M128_K8192_row
C++ exception with description "transformer_engine/common/swizzle/swizzle.cu:1360 in function launch_multi_tensor_swizzle_scaling_factors: CUDA Error: too many resources requested for launch" thrown in the test body.

(n2_M128_K1024_row is meant to cover the narrow-K kernel, which needs 128 KB of shared memory. On a 99 KiB GPU the dispatcher correctly falls back to the regular multi-tensor kernel, which then fails to launch.)

The row<int2> and col<int4> variants are not reached by the current test shapes, e.g. {2, 128, 4352, true} (row, vec_load_size = 2) and {2, 512, 4096, false} (col, vec_load_size = 4) would cover them.

Expected behavior

The multi-tensor swizzle succeeds for all shapes, like the single-tensor path.

Proposed fix

Add __launch_bounds__(TB_DIM * TB_DIM) to the four multi-tensor kernels, matching the other swizzle kernels, and add test shapes that reach the row<int2> and col<int4> variants. I'm happy to open a PR.

Environment overview

  • Environment location: Docker on vast.ai
  • Method of Transformer Engine install: from source (main at 5759fa0f)
  • Docker image: Ubuntu 24.04 with /venv/main

Environment details

  • OS version: Ubuntu 24.04
  • PyTorch version: 2.11.0+cu128
  • Python version: 3.12
  • Transformer Engine version: 2.21.0.dev0+5759fa0f
  • CUDA version: 12.8 (register table also with 13.4)
  • CUDNN version: 9.19

Device details

  • GPU model: NVIDIA GeForce RTX 5090 (CC 12.0), driver 580.95.05
Lingua principale
Python
Stelle
3.6k
Fork
844
Merge medio
4g 42m
PR unite (30g)
49

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di NVIDIA/TransformerEngine

Tutte le issue di NVIDIA/TransformerEngine

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.