INT4 packing extension never loads from a read-only install

Aperta
  • Qualcosa non va?
#3,025 0 commenti 0 reazioni 1 assegnatario Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

@ajrasane ci sta già lavorando.

Dal 29/10/2026.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
35/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python, pytorch
Ambito
backend, performance

Direzione di ricerca

Read modelopt/onnx/quantization/extensions.py and trace how cppimport builds and loads modelopt_round_and_pack_ext when the installed package is read-only. Reproduce with an unwritable package directory and check whether the extension loads or the fallback is used. Done means INT4 packing works from a read-only install, with tests covering the chosen behavior; the issue lists per-user cache builds, prebuilt wheels, and quiet fallback as possible directions.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Describe the bug

modelopt/onnx/quantization/extensions.py builds modelopt_round_and_pack_ext with cppimport the first time it is imported, next to its source in the installed package. Where the user can't write there, for example in a system-wide install, a read-only container layer, or an image built as root and run as another user, cppimport can't take its build lock, so every process logs

WARNING - [Errno 13] Permission denied: '.../modelopt/onnx/quantization/src/modelopt_round_and_pack_ext.cpython-314-x86_64-linux-gnu.so.lock'
Unable to load `modelopt_round_and_pack_ext', falling back to python based optimized version.

([Errno 30] Read-only file system on a read-only mount), and INT4 packing always uses the NumPy fallback. PyPI has only a py3-none-any wheel, so the extension is always built on the user's machine, and this applies to every such install.

extensions.py is the same in 0.47.0, release/0.48.0 and main.

Steps/Code to reproduce bug

  1. Install ModelOpt with the onnx extra, and don't import modelopt.onnx.quantization from it while it is writable: that builds the extension next to its source, and an up-to-date build then loads without taking the lock.

  2. Make the installed modelopt package unwritable for the user who will run it, for example with chmod -R a-w .../site-packages/modelopt, and use a non-root user (root ignores the permission bits). Or mount it read-only, which gives [Errno 30] even for root.

  3. As that user:

    import modelopt.onnx.quantization.extensions as ext
print(ext.round_and_pack_ext)  # None, with the warning above
Expected behavior

The extension is built somewhere the user can write and then loaded, or the fallback is quiet if that is intended.

Possible fixes
  1. Build in a per-user cache when the package is read-only, as torch.utils.cpp_extension does (~/.cache/torch_extensions). I have a tested change that builds a copy of the source in $XDG_CACHE_HOME/modelopt/onnx_extensions/<hash of the source>/, compares that copy with the package's source before each use, keeps the in-place build for writable installs, still loads a prebuilt extension next to a read-only source (also where no temporary directory can be created), and comes with unit tests. I can open it as a PR if this is the direction you want.
  2. Ship the extension prebuilt in the wheels.
  3. Replace cppimport, as the TODO in extensions.py suggests.

Two related points:

  • cppimport removes its temporary build directory only when the build succeeds (build_module() calls shutil.rmtree after setuptools.setup(), not in a finally). Today a failed build ends the process instead, because setuptools raises SystemExit, which extensions.py doesn't catch; I'm opening a pull request for that, which will reference this issue. Once it doesn't, every process whose build fails leaves a directory in $TMPDIR. A fix belongs in cppimport; ModelOpt could also point tempfile.tempdir at a directory it removes while cppimport runs.
  • I haven't measured how much faster packing is with the extension than with the fallback. If the difference is small, a quiet fallback may be the better fix.
Who can help?
  • ?

System information

  • Container used (if applicable): none; the Nixpkgs build sandbox
  • OS (e.g., Ubuntu 22.04, CentOS 7, Windows 10): NixOS (Linux)
  • CPU architecture (x86_64, aarch64): x86_64
  • GPU name (e.g. H100, A100, L40S): none needed
  • Library versions (if applicable):
    • Python: 3.14.8
    • ModelOpt version or commit hash: 0.47.0, and main at efda42cd
    • PyTorch: 2.14.1
    • ONNXRuntime: 1.27.1
  • Any other details that may help: seen while packaging ModelOpt for Nixpkgs (NixOS/nixpkgs#577198), where the package is installed read-only and the extension is now prebuilt at package build time. cppimport 26.04.17.

AI disclosure: this report and the tested change it mentions were produced with Claude Code (Claude Opus 5.5).

🤖 Generated with Claude Code

Lingua principale
Python
Stelle
5.5k
Fork
785
Merge medio
3g 22h
PR unite (30g)
277

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di NVIDIA/Model-Optimizer

Tutte le issue di NVIDIA/Model-Optimizer

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.