Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

test_cast_fp4_impl_gpu_mem is flaky on Jetson Thor: mem_get_info reports system-wide memory on integrated GPUs

Aperta
  • Qualcosa non va?
#2,706 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

@LinCanNerd ci sta già lavorando.

Dal 8/10/2026.

  • #2707 di @LinCanNerd — aperta

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
25/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Ferma
Stack tecnologico
python, pytorch
Ambito
testing

Direzione di ricerca

Read tests/gpu/torch/quantization/test_qtensor_cuda.py, especially TestQTensor::test_cast_fp4_impl_gpu_mem and test_mxfp8_quantize_gpu_mem; the issue also points to test_qtensor in the same file as an allocator-measurement example. Run the supplied pytest command on an integrated GPU and compare the measurements. Done means both affected tests measure allocations independently of system-wide memory and pass reliably; linked PR #2707 is already open.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Describe the bug

tests/gpu/torch/quantization/test_qtensor_cuda.py::TestQTensor::test_cast_fp4_impl_gpu_mem fails intermittently on Jetson Thor:

>       assert (after_quantize - before_quantize) < input_size * 2.1
E       assert (107994492928 - 107802701824) < (10240000 * 2.1)

The test measures memory as total - free from torch.cuda.mem_get_info(). Thor's GPU is integrated (torch.cuda.get_device_properties(0).is_integrated == 1) and shares memory with the CPU, so mem_get_info() reports system-wide memory and changes with every other allocation on the machine. In the failure above, another process was allocating memory during the cast (191 MB were counted).

Even on an otherwise idle machine, the measurement sits at the bound. Over 5 runs of the 1600×1600 FP32 cast, the mem_get_info() delta was 2.05×–2.15× the input size, against a 2.1× bound. The caching allocator's peak (torch.cuda.max_memory_allocated()) was 1.50× every time.

test_mxfp8_quantize_gpu_mem uses the same measurement. It passes on Thor (2.05× against a 3× bound), but it is exposed to the same noise.

Impact: test-only. A GPU test is flaky on Jetson Thor. I have only run it on Thor.

Steps/Code to reproduce bug

Run the test a few times on Thor:

pytest tests/gpu/torch/quantization/test_qtensor_cuda.py -k test_cast_fp4_impl_gpu_mem

Or compare the two measurements directly:

import torch
from modelopt.torch.quantization.qtensor import NVFP4QTensor

NVFP4QTensor._cast_fp4(torch.rand((8, 8), device="cuda"))
for _ in range(5):
    x = torch.rand((1600, 1600), device="cuda")
    size = x.element_size() * x.numel()
    torch.cuda.synchronize()
    torch.cuda.empty_cache()
    free_before, _ = torch.cuda.mem_get_info()
    torch.cuda.reset_peak_memory_stats()
    base = torch.cuda.memory_allocated()
    NVFP4QTensor._cast_fp4(x)
    torch.cuda.synchronize()
    free_after, _ = torch.cuda.mem_get_info()
    print(
        f"mem_get_info: {(free_before - free_after) / size:.2f}x  "
        f"allocator peak: {(torch.cuda.max_memory_allocated() - base) / size:.2f}x"
    )

On Thor, mem_get_info gives 2.08×, 2.15×, 2.05×, 2.05×, 2.05×. The allocator peak is 1.50× each time.

Expected behavior

The test measures only the memory _cast_fp4 allocates, independent of other activity on the device, and passes reliably on integrated GPUs.

Proposed fix: measure with the caching allocator's peak (torch.cuda.reset_peak_memory_stats() and torch.cuda.max_memory_allocated()). test_qtensor in the same file already checks memory through the allocator. I'll open a PR.

Who can help?
  • ?

System information

  • Container used (if applicable): none
  • OS: Ubuntu 24.04.4 LTS (JetPack 7.2.1, L4T R39.2)
  • CPU architecture: aarch64
  • GPU name: NVIDIA Thor (Jetson, sm_110, integrated GPU)
  • GPU memory size: 122.8 GiB, shared with the CPU
  • Number of GPUs: 1
  • Library versions (if applicable):
    • Python: 3.12.3
    • ModelOpt version or commit hash: main at 90ba9fbc
    • CUDA: 13.0
    • PyTorch: 2.13.0+cu130
    • Transformers: 5.14.1
    • TensorRT-LLM: N/A
    • ONNXRuntime: N/A
    • TensorRT: N/A
  • Any other details that may help: The rest of tests/gpu/torch/quantization and tests/gpu/torch/kernels pass on this machine, apart from #2705 and tests skipped for missing optional packages.
Lingua principale
Python
Stelle
5.4k
Fork
767
Merge medio
3g 17h
PR unite (30g)
177

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di NVIDIA/Model-Optimizer

Tutte le issue di NVIDIA/Model-Optimizer

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.