INT4 packing extension never loads from a read-only install

Đang mở
  • Có gì không ổn?
#3,025 0 bình luận 0 reaction 1 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 2 ngày

@ajrasane đang làm issue này rồi.

Từ ngày 29/10/2026.

Đánh giá

Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức phù hợp với người mới
35/100
Loại issue
Lỗi
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
python, pytorch
Lĩnh vực
backend, performance

Hướng nghiên cứu

Read modelopt/onnx/quantization/extensions.py and trace how cppimport builds and loads modelopt_round_and_pack_ext when the installed package is read-only. Reproduce with an unwritable package directory and check whether the extension loads or the fallback is used. Done means INT4 packing works from a read-only install, with tests covering the chosen behavior; the issue lists per-user cache builds, prebuilt wheels, and quiet fallback as possible directions.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Describe the bug

modelopt/onnx/quantization/extensions.py builds modelopt_round_and_pack_ext with cppimport the first time it is imported, next to its source in the installed package. Where the user can't write there, for example in a system-wide install, a read-only container layer, or an image built as root and run as another user, cppimport can't take its build lock, so every process logs

WARNING - [Errno 13] Permission denied: '.../modelopt/onnx/quantization/src/modelopt_round_and_pack_ext.cpython-314-x86_64-linux-gnu.so.lock'
Unable to load `modelopt_round_and_pack_ext', falling back to python based optimized version.

([Errno 30] Read-only file system on a read-only mount), and INT4 packing always uses the NumPy fallback. PyPI has only a py3-none-any wheel, so the extension is always built on the user's machine, and this applies to every such install.

extensions.py is the same in 0.47.0, release/0.48.0 and main.

Steps/Code to reproduce bug

  1. Install ModelOpt with the onnx extra, and don't import modelopt.onnx.quantization from it while it is writable: that builds the extension next to its source, and an up-to-date build then loads without taking the lock.

  2. Make the installed modelopt package unwritable for the user who will run it, for example with chmod -R a-w .../site-packages/modelopt, and use a non-root user (root ignores the permission bits). Or mount it read-only, which gives [Errno 30] even for root.

  3. As that user:

    import modelopt.onnx.quantization.extensions as ext
print(ext.round_and_pack_ext)  # None, with the warning above
Expected behavior

The extension is built somewhere the user can write and then loaded, or the fallback is quiet if that is intended.

Possible fixes
  1. Build in a per-user cache when the package is read-only, as torch.utils.cpp_extension does (~/.cache/torch_extensions). I have a tested change that builds a copy of the source in $XDG_CACHE_HOME/modelopt/onnx_extensions/<hash of the source>/, compares that copy with the package's source before each use, keeps the in-place build for writable installs, still loads a prebuilt extension next to a read-only source (also where no temporary directory can be created), and comes with unit tests. I can open it as a PR if this is the direction you want.
  2. Ship the extension prebuilt in the wheels.
  3. Replace cppimport, as the TODO in extensions.py suggests.

Two related points:

  • cppimport removes its temporary build directory only when the build succeeds (build_module() calls shutil.rmtree after setuptools.setup(), not in a finally). Today a failed build ends the process instead, because setuptools raises SystemExit, which extensions.py doesn't catch; I'm opening a pull request for that, which will reference this issue. Once it doesn't, every process whose build fails leaves a directory in $TMPDIR. A fix belongs in cppimport; ModelOpt could also point tempfile.tempdir at a directory it removes while cppimport runs.
  • I haven't measured how much faster packing is with the extension than with the fallback. If the difference is small, a quiet fallback may be the better fix.
Who can help?
  • ?

System information

  • Container used (if applicable): none; the Nixpkgs build sandbox
  • OS (e.g., Ubuntu 22.04, CentOS 7, Windows 10): NixOS (Linux)
  • CPU architecture (x86_64, aarch64): x86_64
  • GPU name (e.g. H100, A100, L40S): none needed
  • Library versions (if applicable):
    • Python: 3.14.8
    • ModelOpt version or commit hash: 0.47.0, and main at efda42cd
    • PyTorch: 2.14.1
    • ONNXRuntime: 1.27.1
  • Any other details that may help: seen while packaging ModelOpt for Nixpkgs (NixOS/nixpkgs#577198), where the package is installed read-only and the extension is now prebuilt at package build time. cppimport 26.04.17.

AI disclosure: this report and the tested change it mentions were produced with Claude Code (Claude Opus 5.5).

🤖 Generated with Claude Code

Ngôn ngữ chính
Python
Star
5.5k
Fork
785
Merge trung bình
3 ngày 22 giờ
Pull request đã merge (30 ngày)
277

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của NVIDIA/Model-Optimizer

Tất cả issue của NVIDIA/Model-Optimizer

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.