INT4 packing extension never loads from a read-only install

Abierto
  • ¿Algo no está bien?
#3,025 0 comentarios 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 2 días

@ajrasane ya está trabajando en esto.

Desde el 29/10/2026.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
35/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python, pytorch
Área
backend, performance

Línea de trabajo

Read modelopt/onnx/quantization/extensions.py and trace how cppimport builds and loads modelopt_round_and_pack_ext when the installed package is read-only. Reproduce with an unwritable package directory and check whether the extension loads or the fallback is used. Done means INT4 packing works from a read-only install, with tests covering the chosen behavior; the issue lists per-user cache builds, prebuilt wheels, and quiet fallback as possible directions.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Describe the bug

modelopt/onnx/quantization/extensions.py builds modelopt_round_and_pack_ext with cppimport the first time it is imported, next to its source in the installed package. Where the user can't write there, for example in a system-wide install, a read-only container layer, or an image built as root and run as another user, cppimport can't take its build lock, so every process logs

WARNING - [Errno 13] Permission denied: '.../modelopt/onnx/quantization/src/modelopt_round_and_pack_ext.cpython-314-x86_64-linux-gnu.so.lock'
Unable to load `modelopt_round_and_pack_ext', falling back to python based optimized version.

([Errno 30] Read-only file system on a read-only mount), and INT4 packing always uses the NumPy fallback. PyPI has only a py3-none-any wheel, so the extension is always built on the user's machine, and this applies to every such install.

extensions.py is the same in 0.47.0, release/0.48.0 and main.

Steps/Code to reproduce bug

  1. Install ModelOpt with the onnx extra, and don't import modelopt.onnx.quantization from it while it is writable: that builds the extension next to its source, and an up-to-date build then loads without taking the lock.

  2. Make the installed modelopt package unwritable for the user who will run it, for example with chmod -R a-w .../site-packages/modelopt, and use a non-root user (root ignores the permission bits). Or mount it read-only, which gives [Errno 30] even for root.

  3. As that user:

    import modelopt.onnx.quantization.extensions as ext
print(ext.round_and_pack_ext)  # None, with the warning above
Expected behavior

The extension is built somewhere the user can write and then loaded, or the fallback is quiet if that is intended.

Possible fixes
  1. Build in a per-user cache when the package is read-only, as torch.utils.cpp_extension does (~/.cache/torch_extensions). I have a tested change that builds a copy of the source in $XDG_CACHE_HOME/modelopt/onnx_extensions/<hash of the source>/, compares that copy with the package's source before each use, keeps the in-place build for writable installs, still loads a prebuilt extension next to a read-only source (also where no temporary directory can be created), and comes with unit tests. I can open it as a PR if this is the direction you want.
  2. Ship the extension prebuilt in the wheels.
  3. Replace cppimport, as the TODO in extensions.py suggests.

Two related points:

  • cppimport removes its temporary build directory only when the build succeeds (build_module() calls shutil.rmtree after setuptools.setup(), not in a finally). Today a failed build ends the process instead, because setuptools raises SystemExit, which extensions.py doesn't catch; I'm opening a pull request for that, which will reference this issue. Once it doesn't, every process whose build fails leaves a directory in $TMPDIR. A fix belongs in cppimport; ModelOpt could also point tempfile.tempdir at a directory it removes while cppimport runs.
  • I haven't measured how much faster packing is with the extension than with the fallback. If the difference is small, a quiet fallback may be the better fix.
Who can help?
  • ?

System information

  • Container used (if applicable): none; the Nixpkgs build sandbox
  • OS (e.g., Ubuntu 22.04, CentOS 7, Windows 10): NixOS (Linux)
  • CPU architecture (x86_64, aarch64): x86_64
  • GPU name (e.g. H100, A100, L40S): none needed
  • Library versions (if applicable):
    • Python: 3.14.8
    • ModelOpt version or commit hash: 0.47.0, and main at efda42cd
    • PyTorch: 2.14.1
    • ONNXRuntime: 1.27.1
  • Any other details that may help: seen while packaging ModelOpt for Nixpkgs (NixOS/nixpkgs#577198), where the package is installed read-only and the extension is now prebuilt at package build time. cppimport 26.04.17.

AI disclosure: this report and the tested change it mentions were produced with Claude Code (Claude Opus 5.5).

🤖 Generated with Claude Code

Lenguaje dominante
Python
Estrellas
5.5k
Forks
785
Merge medio
3 d 22 h
PR fusionados (30 d)
277

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de NVIDIA/Model-Optimizer

Todos los issues de NVIDIA/Model-Optimizer

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.