TrainerRank oversized admission reaches CUDA OOM during Qwen3.6-35B-A3B forward
Maintainers usually reply within 3 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 10/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Active
- Domain
- ai-infra-agents, performance
Research direction
The issue is a deep memory allocation failure in the TrainerRank oversized admission path for a Qwen3.6-35B-A3B model. Start by examining the planner logic in trainer_rank/_impl.py, specifically _recover_admission_impl. Look at the memory budget calculations and the conditions that lead to admitted_oversized. The failure occurs during a forward pass, likely in MoE expert FC2 or ART's expert LoRA forward. To understand 'done', you need to determine if a smaller execution plan is possible or what additional allocation evidence is required, but the issue lacks a clear, reproducible test case.
Written by the indexing model from the issue text.
Description
A production TrainerRank forward failed with a CUDA OOM after the planner explicitly selected admitted_oversized. This records a confirmed failure of the opt-in oversized path for investigation. The selected estimate already exceeded the available admission budget; the evidence does not establish an underestimate or a failure of normal fit admission.
Event b02499988c7c40e98659be0a31376eb0, September 24, 2026, 2:43:31 AM MDT (08:43:31 UTC): operation=forward_micro_batches, phase=forward, decision.outcome=admitted_oversized. Actual rank fields were allow_oversized_batches=True and planner report threshold 10. The launch explicitly enabled:
ART_TRAINER_RANK_ALLOW_OVERSIZED_BATCHES=1
ART_TRAINER_RANK_PLANNER_MISS_THRESHOLD_PCT=10
The original failure explicitly says CUDA ran out of memory while allocating 2.39 GiB on a 139.81 GiB device with 2.15 GiB free. PyTorch had 133.58 GiB allocated and 243.59 MiB reserved but unallocated. These observations do not establish fragmentation.
| Selected admission / report field | Bytes |
|---|---|
| Predicted peak | 121,280,725,363 |
| Admission required, safety factor 1.1 | 133,408,797,899 |
| Available admission budget | 19,621,678,285 |
| Physical free at selected admission sample | 24,125,308,928 |
| PyTorch allocated at selected admission sample | 72,457,731,584 |
| PyTorch reserved at selected admission sample | 121,867,599,872 |
| OOM partial peak above observation baseline | 73,220,957,696 |
The selected sample uses allocator native, inactive_split_bytes=0, a 3% reserve and no test memory cap. Its required estimate was carried from check ordinal 7 into selected sample ordinal 18. Planner required/peak values describe the planner observation and admission calculation; they are not absolute CUDA allocated-memory measurements. The partial peak is relative to baseline 72,455,432,192 bytes. Completed observed_peak_bytes and error_pct are both null, as intended for an interrupted OOM observation; no underestimate percentage can be calculated from them.
Runtime/model: Qwen/Qwen3.6-35B-A3B, revision 995ad96eacd98c81ed38be0c5b274b04031597b0, BF16, LoRA rank 1; Torch 2.11.0+cu128, CUDA 12.8, rank 0 / cuda:0, compute capability 9.0, device capacity 150121021440 bytes. Observed planner topology is (DP, TP, CP, PP)=(1,1,1,1), sequence parallelism false. Model geometry has 40 layers, 30 GDN layers, hidden size 2048, 256 MoE experts and top-k 8. The event identifies the policy checkpoint slot but does not record its revision. The run owner reports continuation from durable step 225 and four completed resumed callbacks; failing logical batch 230 is a FIFO inference, not a direct event field or optimizer proof.
The entire failing forward plan contains 19 requests, 1,073,093 logical tokens and 359,965 packed tokens, one slot group and four subforwards. All input/target descriptors are 1-D int64 tensors with matching lengths; gradients are enabled, logits/hidden-state outputs are disabled.
| Subforward in plan order | Input lengths | Logical tokens | Packed tokens | GDN segments |
|---|---|---|---|---|
| 0 | 76618, 80763, 84906, 89049 | 331336 | 100410 | 7 |
| 1 | 93189, 94017 | 187206 | 97683 | 3 |
| 2 | 55505, 64202, 68338, 72480 | 260525 | 83557 | 7 |
| 3 | 12430, 16565, 23973, 28111, 32251, 38974, 43111, 47238, 51373 | 294026 | 78315 | 17 |
The exact failing subforward/request and allocation tensor shape are unavailable. The retained stack passes through MoE expert FC2, ART's expert LoRA forward and a TorchInductor compiled call; this locates the failure path without establishing a cause. Request descriptors are flattened in plan-group order; retained original request indices must not be applied directly to that flattened list.
The frozen run uses ART base 8ff205e543473d96557fe4b0ffbeaea19d13b420 plus a feature backport and operational instrumentation, and Caladan base 257be1824d2dd7495e6e3509fbcad9af9ee74e9a. Exact runtime source manifest SHA-256: 8ba82252431f8ce841dcbe4739d8272c925d949c20657331021a503d2eb347d0; executed ART trainer_rank/_impl.py: 6c0e9e68d32eb0067fa5206f0ff419c2840fd60922f0e90941c07b4486b1f313. In that source, _recover_admission_impl assigns admitted_oversized after exhausted memory-refusal recovery; forward tracking catches the CUDA OOM and emits the report. This is not a clean-base reproduction or a claim about current main.
Expected operational outcome was continued training with oversized admission enabled; actual outcome was a terminal CUDA OOM. The flag deliberately allows an over-budget candidate, so this issue asks whether a supported smaller execution plan can preserve the same work, and what additional allocation evidence is needed if it cannot.
Sanitized retained evidence is available: original event SHA-256 b580a842afbdbd58dab0c8f5a8bb17a455897ea62f01940ba45e98ed53623b10; safe projection d7b93efc01290300a6597894e07c954ffe0f5d27ed39e313656dadf2b769f26e. No token values, private payloads, checkpoint weights or credentials are included here. replay_complete=false (device_or_modified_input, layout_inputs_unavailable), so the retained metadata is not a complete executable GPU reproducer. No rerun or proposed fix has been tested.
- Dominant language
- Python
- Stars
- 10.8k
- Forks
- 989
- Avg merge
- 11h 38m
- Merged PRs (30d)
- 104
Getting set up
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from OpenPipe/ART
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 3 days
-
Difficulty 4/5 3-5 days Newbie friendliness 54/100
OpenPipe/ART#961 · 3 comments ·
Maintainers usually reply within 3 days
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
OpenPipe/ART#949 · 5 comments ·
Maintainers usually reply within 3 days
-
Difficulty 5/5 Over a week Newbie friendliness 42/100
Maintainers usually reply within 3 days
-
enhancement
Difficulty 4/5 3-5 days Newbie friendliness 48/100
Maintainers usually reply within 3 days
Similar issues
-
good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
vllm-project/vllm-metal#822 ·
Maintainers usually reply within 1 day
-
vector-store
Difficulty 1/5 1-3 hours Newbie friendliness 90/100
mem0ai/mem0#7461 · 1 comment ·
Maintainers usually reply within 1 day
-
[Bug]: chunk_span_bounds and _validated_chunk_spans reject Pydantic models ChunkSpan and AudioFileOpen
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
BasedHardware/omi#19047 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day