[Bug]: offloder cleaning failed
Maintainers usually reply within 1 day
Assessment
This issue has not been assessed yet.
Description
Problem Description
CUDA_VISIBLE_DEVICES=7 python3 -m auto_round ../Qwen3.8-27B/ --iters 0 --disable_opt_rtn --scheme mxfp4 --tasks mmlu --eval_bs 16 --format fake
uantizing model.language_model.layers.61: 95%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ | 61/64 [04:35<00:12, 4.30s/it]2026-10-09 20:01:26 INFO device.py L1553: 'peak_ram': 3.02GB, 'peak_vram': 6.01GB
Quantizing model.language_model.layers.62: 97%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ | 62/64 [04:39<00:08, 4.31s/it]2026-10-09 20:01:30 INFO device.py L1553: 'peak_ram': 3.04GB, 'peak_vram': 6.01GB
Quantizing model.language_model.layers.63: 98%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ | 63/64 [04:44<00:04, 4.32s/it]2026-10-09 20:01:34 INFO device.py L1553: 'peak_ram': 3.08GB, 'peak_vram': 6.01GB
Quantizing model.language_model.layers.63: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 64/64 [04:48<00:00, 4.31s/it]2026-10-09 20:01:37 WARNING offload.py L938: OffloadManager: cleanup failed for /home/wenhuach/auto-round/ar_work_space/offload/compressor_ny51qx4s: [Errno 16] Device or resource busy: '/home/wenhuach/auto-round/ar_work_space/offload/compressor_ny51qx4s'
Quantizing model.language_model.layers.63: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 64/64 [04:51<00:00, 4.55s/it]
2026-10-09 20:01:37 WARNING base.py L1949: Support for exporting activation quantization is limited. Please ensure that your configuration is supported.
2026-10-09 20:01:37 WARNING fake.py L139: Saving fake-quantized model to disk. Linear replacement is deferred to load-time (via auto_round:fake backend); save-time now keeps the in-memory quantized model structure unchanged.
Writing model shards: 0%| | 0/2 [00:00<?,
Reproduction Steps
~
Environment Information
~
Error Logs
Additional Context
No response
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 193
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 50
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from intel/auto-round
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
intel/auto-round#2525 ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 4/5 3-5 days Newbie friendliness 44/100
intel/auto-round#2532 ·
Maintainers usually reply within 1 day
-
[Bug]: FP8 static KV cache accuracy issuePossibly taken @xin3he claimed this 6 days ago. Openbug
intel/auto-round#2518 · 1 reaction · 1 assignee ·
Maintainers usually reply within 1 day
-
[Feature]: split special_model_handler to a folderPossibly taken @Cross2pro claimed this 3 days ago. Openenhancement
Difficulty 3/5 Half a day Newbie friendliness 42/100
intel/auto-round#2514 · 2 comments ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 3/5 1-2 days Newbie friendliness 20/100
intel/auto-round#2507 ·
Maintainers usually reply within 1 day
All issues in intel/auto-round
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
scikit-learn-contrib/category_encoders#553 ·
Maintainers usually reply within 1 day
-
tests/strategies_test.py:174 goes uncovered on about 1 run in 100, failing the coverage ratchetOpenflaky-watch
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
btclib-org/btclib#2839 ·
Maintainers usually reply within 1 day
-
BarrelList constructor drops false-valued iterable contentsPossibly taken @Gambit-Checkmate claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 19 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
vllm-project/vllm#63863 ·
Maintainers usually reply within 1 day