dgx-spark: the threads pool's P/E split has no term on the Spark's default path (P3 row)
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 42/100
- Issue type
- Documentation
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- rust
- Domain
- performance
Research direction
Read docs/contrib/dgx-spark.md:141 and trace the default path through crates/gpu/src/upload.rs:239 to confirm where the thread pool runs. The issue reports that the P/E split does not affect the default Spark path and leaves the documentation unchanged. Done means updating the P3 row to reflect the reported evidence; the proposed topology change is conditional on future work that runs the pool in a compute step.
Written by the indexing model from the issue text.
Description
docs/contrib/dgx-spark.md lists a crates/threads P/E split under P3, and the row at line 141 says the default
thread count is the primary-core count (20 on GB10) with equal chunks, so the 10 E-cores gate every barrier. I looked
at where that barrier sits on a DGX Spark before writing any code, and on the default path I could not find a term for
the split to move. I am not opening a pull request for it now.
Where the pool runs. On the default Spark path (Qwen3.6-35B-A3B Q4_K_M, all of it on the card, through
generate_qwen3moe), the only call into the pool is memcpy_pool in crates/gpu/src/upload.rs:239, the host side of
the weight upload at load. Its pieces are 1 to 32 MiB, so a 21.5 GB file is about 650 dispatches, once per load. The
decode and prefill steps make no dispatch. Every other threads:: use in crates/gpu/src is under host/ (the host
tier and the NVMe tier), which this placement does not start, and crates/gpu/src/arch/qwen3moe/, crates/app/src and
generate_qwen3moe.rs have none.
The cores. Both Sparks I have read the same from sysfs: cpu_capacity 718 on cpu0-4 and 731 on cpu10-14
(Cortex-A725, cpuinfo_max_freq 2808000), 997 on cpu5-9, 1017 on cpu15-18 and 1024 on cpu19 (Cortex-X925, 3900000);
L3 groups 0-9 and 10-19. Each group lists its E-cores first, so by my reading of ccd_slot and cpu_for,
BLOOMERY_THREADS=10 puts all nine workers on A725 cores.
A check of the one place it runs. I ran the load on two Sparks with BLOOMERY_THREADS=20, 10 (the workers all
on E-cores, as above) and 1, alternating the order over three rounds, with the model file already in the page cache.
The load line read the same at 20 and at 10 on both machines, and 1 was clearly slower, so the setting did take
effect. The upload is not bound by the cores the pool runs on. I ran this directly rather than through a lease runner,
so I give no table; it only says there is no term here to measure.
If you want the pool to know core capacity later, the smallest form I see is in detect_topology: order each L3
group's primaries by cpu_capacity (then cpuinfo_max_freq, then id) so the first workers land on the large cores.
ccd_slot, CcdMap::span and chunk_bounds stay as they are, and a host whose cores are all the same sees no change.
It would only matter once the host tier or a P4 path runs on aarch64 with a barrier bound by its cores, and main has no
such path on a Spark today.
Docs. I did not change them. The row at docs/contrib/dgx-spark.md:141 ("the 10 E-cores gate every barrier",
"M in P3") does not hold for the default Spark path.
I am keeping this open as the note for that P3 item. If later work on the Spark changes any of this (a host tier or
a P4 path that does run the pool in a step, or a measurement through a lease runner), I will update this issue and
leave a comment here.
Machine: DGX Spark, NVIDIA GB10, driver 595.84, CPU 10 Cortex-X925 and 10 Cortex-A725, 128 GB unified memory, Ubuntu
24.04.5 LTS, kernel 6.17.0-1032-nvidia; main at 2ce989d7 with the local-mode and single-card timing commits (#38,
#39). git log 2ce989d7..0c8397db -- crates/threads crates/gpu/src/upload.rs crates/gpu/src/weights.rs is empty.
- Dominant language
- Rust
- Stars
- 21
- Forks
- 4
- Avg merge
- 2h 32m
- Merged PRs (30d)
- 530
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from midagedev/bloomery
-
review-bot
Difficulty 3/5 1-2 days Newbie friendliness 74/100
Maintainers usually reply within 1 day
-
review-bot
Difficulty 3/5 1-2 days Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
model
Difficulty 5/5 Over a week Newbie friendliness 25/100
midagedev/bloomery#698 · 3 comments ·
Maintainers usually reply within 1 day
-
model
Difficulty 5/5 Over a week Newbie friendliness 25/100
Maintainers usually reply within 1 day
-
model
Difficulty 5/5 Over a week Newbie friendliness 25/100
midagedev/bloomery#632 · 9 comments ·
Maintainers usually reply within 1 day
All issues in midagedev/bloomery
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 75/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
manuscript-lang/manuscript#1364 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
equinor/septic-config-generator#504 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 72/100
GiganticMinecraft/seichi-portal-backend#1633 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
fable-compiler/Fable#5157 ·
Maintainers usually reply within 1 day