Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

test_cuda_pack_reconstruction_matches_pytorch_at_scale[iq1_m] fails on Jetson Thor: exact ties exceed the blocks // 1000 bound

Aperta
  • Qualcosa non va?
#2,705 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

@LinCanNerd ci sta già lavorando.

Dal 8/10/2026.

  • #2708 di @LinCanNerd — aperta

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
35/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Ferma
Stack tecnologico
python, pytorch
Ambito
machine-learning, testing

Direzione di ricerca

Start with tests/gpu/torch/quantization/test_iq_formats_cuda.py, especially test_cuda_pack_reconstruction_matches_pytorch_at_scale and the neighboring encoder test docstring. Run the reported pytest command on a CUDA GPU and compare per-block squared errors for differing blocks with the PyTorch reference. Done means tied blocks are accepted within the stated tolerance while genuine quality regressions still fail; linked PR #2708 is already open.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Describe the bug

tests/gpu/torch/quantization/test_iq_formats_cuda.py::test_cuda_pack_reconstruction_matches_pytorch_at_scale[iq1_m] fails on Jetson Thor:

>       assert differing <= blocks // 1000, f"{differing} of {blocks} blocks differ"
E       AssertionError: 2 of 1024 blocks differ
E       assert 2 <= (1024 // 1000)

Both differing blocks are exact ties. The CUDA encoding and the PyTorch encoding of each block decode to different values (one group of 8 weights uses a different grid point), but they reconstruct the block with bit-identical squared error:

format block (row, block) SSE, CUDA SSE, PyTorch bytes that differ
iq1_m (60, 7) 41.883705 41.883705 2
iq1_m (68, 4) 43.345009 43.345009 1
iq2_xs (123, 6) 22.985538 22.985538 2

Total relative error matches exactly for all five formats, so the CUDA encoder loses no quality. iq2_xs passes only because it has 1 such block, which is at the bound.

The docstring of test_cuda_pack_matches_pytorch_encoder_and_is_decodable already explains this behavior. The two encoders evaluate the same squared error with different floating-point fusion, so near-ties can round to different sides "in roughly one block in several thousand". That rate depends on how the compiler fuses the CUDA encoder's arithmetic for the target GPU. On Thor it is 2 in 1024 for IQ1_M. The blocks // 1000 bound therefore encodes one GPU's tie rate rather than the property the test's docstring states ("may disagree on a near-tied scale, but not on quality").

Impact: test-only. A GPU test fails on Jetson Thor.

Steps/Code to reproduce bug

On Jetson Thor:

pytest "tests/gpu/torch/quantization/test_iq_formats_cuda.py::test_cuda_pack_reconstruction_matches_pytorch_at_scale"
Expected behavior

The test passes wherever the CUDA encoder reconstructs each block as well as the reference, which is true on Thor.

Proposed fix: require every differing block to have the same per-block squared error as the reference (rtol=1e-5). Keep the total-error check, and loosen the count bound to 1% of blocks. With that change, flipping a grid-index bit in a single block still fails the test. I'll open a PR.

Who can help?
  • @cjluo-nv (IQ formats)

System information

  • Container used (if applicable): none
  • OS: Ubuntu 24.04.4 LTS (JetPack 7.2.1, L4T R39.2)
  • CPU architecture: aarch64
  • GPU name: NVIDIA Thor (Jetson, sm_110, integrated GPU)
  • GPU memory size: 122.8 GiB, shared with the CPU
  • Number of GPUs: 1
  • Library versions (if applicable):
    • Python: 3.12.3
    • ModelOpt version or commit hash: main at 90ba9fbc
    • CUDA: 13.0 (the modelopt_cuda_ext_ggml extension is built for sm_110)
    • PyTorch: 2.13.0+cu130
    • Transformers: 5.14.1
    • TensorRT-LLM: N/A
    • ONNXRuntime: N/A
    • TensorRT: N/A
  • Any other details that may help: Every other test in test_iq_formats_cuda.py passes on Thor, including byte parity on the small fixed input and the comparison with llama.cpp's decoder.
Lingua principale
Python
Stelle
5.3k
Fork
745
Merge medio
3g 17h
PR unite (30g)
177

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di NVIDIA/Model-Optimizer

Tutte le issue di NVIDIA/Model-Optimizer

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.