[BUG] Order-dependent wrong rmem values on Windows SM120 with CuTe DSL 4.8 CUDA12 stack; CUDA13 control passes

Open
  • Is something wrong?
#3,673 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 4 days

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Active
Tech stack
cpp, python, pytorch
Domain
backend, compilers

Research direction

Start by running the attached repro.py with the four listed variants and compare report.json or stdout against the expected values. Then trace the failing Windows CUDA12 path across the CuTe DSL legacy and extension pipelines, using the CUDA13 and scalar/snapshot variants as controls. Done means identifying and fixing the responsible compiler or runtime component without regressing the passing controls.

Written by the indexing model from the issue text.

Description

? - Needs Triage bug CuTe DSL inactive-30d
Which component has the problem?

CuTe DSL

Bug Report

Describe the bug

A standalone CuTe DSL reproducer produces deterministic incorrect packed values on an RTX 5090 (SM120) under the tested Windows CUDA12 stack.

The same source and input produce correct results when:

  • the four-element rmem tensor is replaced with independent scalar values;
  • the same Windows RTX 5090 is tested with the CUDA13 stack; or
  • the reproducer is run on the Linux SM121 control system.

Reversing the evaluation order also moves the corruption from the first component to the second component.

This behavior appears consistent with an SSA/liveness/code-generation issue, but I have not identified the exact failing compiler stage.

I am not claiming that this is specifically a register allocator, NVVM, or SM120 hardware bug.

Steps/Code to reproduce bug

The attached repro.py is standalone.

It does not require DS41RT, SparkInfer, a model, model weights, or an external fixture.

Run:

python repro.py --variant rmem --output out-rmem
python repro.py --variant scalar --output out-scalar
python repro.py --variant reverse --output out-reverse
python repro.py --variant snapshot --output out-snapshot

Each variant tests 16 split positions with 3 repeats per position.

The script exits successfully after collecting results even when a numerical mismatch is found. Use the passed value in report.json or stdout to determine correctness.

Windows RTX 5090 / SM120 results:

Stack Compiler pipeline rmem scalar reverse snapshot
Torch 2.11.0+cu128, CuTe CUDA12.9 backend legacy FAIL PASS FAIL PASS
Torch 2.11.0+cu128, CuTe CUDA12.9 backend extension FAIL PASS FAIL PASS
Torch 2.11.0+cu130, CuTe CUDA13.4 backend legacy PASS PASS PASS PASS

Linux GB10 / SM121 / CuTe DSL 4.8.0 / CUDA13.2 passes all four variants.

With the normal rmem variant:

part0 -> part1
corruption: part0 side

With reversed evaluation order:

part1 -> part0
corruption: part1 side

Replacing the rmem tensor with four independent Uint32 scalar variables removes the failure.

Adding snapshot stores also suppresses the failure.

Expected behavior

For the rmem variant at split 0, output index 0:

Expected:          0x3f803f82
Windows CUDA12:    0x3c003c45
Windows CUDA13:    0x3f803f82
Windows scalar:    0x3f803f82
Linux control:     0x3f803f82

The CUDA12 result is deterministic across all 3 repeats.

CUDA12 rmem mismatch counts across split 0..15:

[28, 28, 28, 28, 28, 28, 28, 28, 28, 24, 20, 16, 12, 8, 4, 0]

Reverse-order mismatch counts:

[31, 27, 23, 19, 15, 11, 7, 3, 0, 0, 0, 0, 0, 0, 0, 0]

Environment details (please complete the following information):

  • Environment location: Bare-metal

Failing CUDA12 stack:

OS: Windows 11 x86-64
GPU: NVIDIA GeForce RTX 5090
Compute capability: SM120
Driver: 591.86
Python: 3.13.5
PyTorch: 2.11.0+cu128
torch CUDA: 12.8
nvidia-cutlass-dsl: 4.8.0
cuda-bindings: 12.9.8

CuTe native module:
_cutlass_ir.cu12.cp313-win_amd64.pyd

CuTe runtime:
cu12/lib/cute_dsl_runtime.dll

Passing CUDA13 stack:

OS: Windows 11 x86-64
GPU: NVIDIA GeForce RTX 5090
Compute capability: SM120
Driver: 591.86
Python: 3.13.5
PyTorch: 2.11.0+cu130
torch CUDA: 13.0
nvidia-cutlass-dsl: 4.8.0
cuda-bindings: 13.2.0

CuTe backend reports CUDA 13.4

CuTe native module:
_cutlass_ir.cu13.cp313-win_amd64.pyd

CuTe runtime:
cu13/lib/cute_dsl_runtime.dll

The system CUDA_PATH remains pointed at the existing CUDA 12.8 toolkit in both cases. The loaded CuTe native module/runtime was checked explicitly, so the CUDA13 result is not inferred from CUDA_PATH.

The CUDA12 -> CUDA13 comparison changes several components of the CUDA package stack together. Therefore this comparison does not establish which individual component or compiler pass is responsible.

Linux control:

OS: Linux aarch64
GPU: NVIDIA GB10
Compute capability: SM121
Driver: 580.142
CUDA: 13.2
CuTe DSL: 4.8.0
PyTorch: 2.12.0a0 NVIDIA build
cuda-bindings: 13.2.0

Additional context

  • A simple Uint32 load-only helper does not reproduce the problem.
  • Restoring half2 unpack and BF16 packing makes the failure reproducible again on the CUDA12 stack.
  • FP4 decoding is not required.
  • MMA is not required.
  • Multiple tiles are not required.
  • vote_ballot_sync is not required.
  • One warp is sufficient.
  • The generated kernel source is identical across the compared Windows CUDA12/CUDA13 runs.
  • CUDA12 legacy and extension compiler pipelines both reproduce the failure.
  • CUDA13 legacy passes both with and without the host-side compiler-routing observer.

This reproducer demonstrates a standalone CuTe DSL numerical correctness failure with the tested Windows CUDA12 stack.

It does not establish whether the root cause is in CuTe DSL lowering, shared MLIR/LLVM lowering, the CUDA12 NVVM/backend, the CuTe CUDA12 native library/runtime, or another interaction within the tested CUDA12 stack.

I have not tested Linux SM120, the CUDA13 extension compiler pipeline, or a main-branch-built CuTe DSL package.

The latest published CUTLASS/CuTe DSL package I found at the time of testing is 4.8.0.

I also checked related CUTLASS issues involving rmem/SSA/code-generation problems, but I did not find an existing report matching this deterministic order-dependent behavior.

Source attribution: the in-file helper/kernel template was adapted from SparkInfer commit 4b0954148523b5a2e93813f963d483ffd350b9c9, specifically b12x/_lib/intrinsics.py and b12x/attention/dsa_indexer/_v41_overlay.py. Copyright (c) 2025 by the b12x authors. Those portions are licensed under Apache-2.0.

repro.py

Dominant language
C++
Stars
10.7k
Forks
2.2k
Avg merge
3d 20h
Merged PRs (30d)
4

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA/cutlass

All issues in NVIDIA/cutlass

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.