Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

test_cuda_pack_reconstruction_matches_pytorch_at_scale[iq1_m] fails on Jetson Thor: exact ties exceed the blocks // 1000 bound

オープン
  • 何か問題がありますか？
#2,705 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 2 日以内に返信

@LinCanNerd がすでに取り組んでいます。

2026年10月8日 から。

  • #2708 @LinCanNerd による — オープン

評価

難易度
2/5
見積もり時間
1〜3時間
初心者へのやさしさ
35/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
停滞
技術スタック
python, pytorch
領域
machine-learning, testing

調査の方向性

Start with tests/gpu/torch/quantization/test_iq_formats_cuda.py, especially test_cuda_pack_reconstruction_matches_pytorch_at_scale and the neighboring encoder test docstring. Run the reported pytest command on a CUDA GPU and compare per-block squared errors for differing blocks with the PyTorch reference. Done means tied blocks are accepted within the stated tolerance while genuine quality regressions still fail; linked PR #2708 is already open.

索引モデルが issue の本文から書いたものです。

説明

Describe the bug

tests/gpu/torch/quantization/test_iq_formats_cuda.py::test_cuda_pack_reconstruction_matches_pytorch_at_scale[iq1_m] fails on Jetson Thor:

>       assert differing <= blocks // 1000, f"{differing} of {blocks} blocks differ"
E       AssertionError: 2 of 1024 blocks differ
E       assert 2 <= (1024 // 1000)

Both differing blocks are exact ties. The CUDA encoding and the PyTorch encoding of each block decode to different values (one group of 8 weights uses a different grid point), but they reconstruct the block with bit-identical squared error:

format block (row, block) SSE, CUDA SSE, PyTorch bytes that differ
iq1_m (60, 7) 41.883705 41.883705 2
iq1_m (68, 4) 43.345009 43.345009 1
iq2_xs (123, 6) 22.985538 22.985538 2

Total relative error matches exactly for all five formats, so the CUDA encoder loses no quality. iq2_xs passes only because it has 1 such block, which is at the bound.

The docstring of test_cuda_pack_matches_pytorch_encoder_and_is_decodable already explains this behavior. The two encoders evaluate the same squared error with different floating-point fusion, so near-ties can round to different sides "in roughly one block in several thousand". That rate depends on how the compiler fuses the CUDA encoder's arithmetic for the target GPU. On Thor it is 2 in 1024 for IQ1_M. The blocks // 1000 bound therefore encodes one GPU's tie rate rather than the property the test's docstring states ("may disagree on a near-tied scale, but not on quality").

Impact: test-only. A GPU test fails on Jetson Thor.

Steps/Code to reproduce bug

On Jetson Thor:

pytest "tests/gpu/torch/quantization/test_iq_formats_cuda.py::test_cuda_pack_reconstruction_matches_pytorch_at_scale"
Expected behavior

The test passes wherever the CUDA encoder reconstructs each block as well as the reference, which is true on Thor.

Proposed fix: require every differing block to have the same per-block squared error as the reference (rtol=1e-5). Keep the total-error check, and loosen the count bound to 1% of blocks. With that change, flipping a grid-index bit in a single block still fails the test. I'll open a PR.

Who can help?
  • @cjluo-nv (IQ formats)

System information

  • Container used (if applicable): none
  • OS: Ubuntu 24.04.4 LTS (JetPack 7.2.1, L4T R39.2)
  • CPU architecture: aarch64
  • GPU name: NVIDIA Thor (Jetson, sm_110, integrated GPU)
  • GPU memory size: 122.8 GiB, shared with the CPU
  • Number of GPUs: 1
  • Library versions (if applicable):
    • Python: 3.12.3
    • ModelOpt version or commit hash: main at 90ba9fbc
    • CUDA: 13.0 (the modelopt_cuda_ext_ggml extension is built for sm_110)
    • PyTorch: 2.13.0+cu130
    • Transformers: 5.14.1
    • TensorRT-LLM: N/A
    • ONNXRuntime: N/A
    • TensorRT: N/A
  • Any other details that may help: Every other test in test_iq_formats_cuda.py passes on Thor, including byte parity on the small fixed input and the comparison with llama.cpp's decoder.
主要言語
Python
スター
5.3k
フォーク
745
平均マージ
4日 2時間
マージ済み PR（30日）
160

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

NVIDIA/Model-Optimizer のほかの issue

NVIDIA/Model-Optimizer の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。