RuntimeError: torch.cuda.MemPool doesn't currently support expandable_segments during vLLM model initialization
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- python, pytorch
- Domain
- machine-learning
Research direction
Start by reproducing the failure with the reported Python 3.12, torch 2.7.1, vLLM 0.10.0, CUDA 12.4, and L4 environment, then inspect art.model.register() and the dependency pins in uv.lock. Compare compatible torch, vLLM, and xformers versions. Done means confirming a tested pinset or documenting the required versions and handling for this initialization error.
Written by the indexing model from the issue text.
Description
Description
Summary
When training a LangGraph agent with openpipe-art[backend,langgraph], the process fails at model initialization with the following error:
RuntimeError: torch.cuda.MemPool doesn't currently support expandable_segments.
The error occurs inside vLLM when allocating CUDA parameters during model initialization.
Environment
- OS: Linux
- GPUs: 2x NVIDIA L4 (23 GB each)
- CUDA: 12.4 (
nvcc --versionshows Cuda compilation tools, release 12.4, V12.4.131) - NVIDIA driver: 550.90.07
- Python: 3.12.x (venv with
uv) - Installed via:
pip install openpipe-art[backend,langgraph] - Dependency versions (from uv.lock):
- torch==2.7.1
- vllm==0.10.0
Steps to reproduce
- Create a new Python 3.12 virtual environment.
uv add openpipe-art[backend,langgraph]>=0.4.11- Run training (which calls
art.model.register()). - Observe the crash at model initialization.
Logs
File ".../vllm/model_executor/layers/vocab_parallel_embedding.py", line 34, in init
weight = Parameter(torch.empty(sum(output_partition_sizes), ...))
RuntimeError: torch.cuda.MemPool doesn't currently support expandable_segments.
Request
- Please confirm if the current pinned torch (2.7.1) + vllm (0.10.0) combination is expected to work with CUDA 12.4 / L4 GPUs.
- If not, could you provide a tested torch/vllm/xformers pinset for CUDA 12.4?
- Alternatively, handle this error in vLLM (or document required versions) so users don’t hit this blocker.
Happy to provide full logs (pip freeze, nvcc, etc.) if needed.
- Dominant language
- Python
- Stars
- 10.8k
- Forks
- 989
- Avg merge
- 6h 25m
- Merged PRs (30d)
- 86
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from OpenPipe/ART
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 5/5 Over a week Newbie friendliness 42/100
-
enhancement
Difficulty 4/5 3-5 days Newbie friendliness 48/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
syfoud/Simulated_Scepter#172 ·
-
A cancelled tests run makes the coverage comment workflow fail and reports it as a red check on main Openarea: ci bug perceived difficulty: 3
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Nitjsefnie-Harness-Commons/daedalus#921 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
EleutherAI/lm-evaluation-harness#4207 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
ClickHouse/clickhouse-connect#1057 ·