NVIDIA/spark-local-inference

View on GitHub

Is NVIDIA/spark-local-inference beginner-friendly?

In the last 30 days, NVIDIA/spark-local-inference merged 18 of 56 pull requests from outside contributors, and its maintainers usually reply within 1 day. 1 beginner-friendly issue is open now.

Stars: 695
Forks: 76
Open beginner issues: 1
Indexed issues: 51
Avg merge: 9h 33m
Merged PRs (30d): 50

Dominant language: Rust
License: Apache-2.0
Last GitHub push: Oct 29, 2026
Latest indexed: Oct 22, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed

How to contribute to NVIDIA/spark-local-inference

There is no contributing guide, so start from the README and a few recently merged pull requests to see how changes are proposed.
It ships a Dockerfile or Docker Compose file, so you can run it without installing its whole toolchain.
Your contributions will be published under the project's Apache-2.0 license.
Pick the open beginner-friendly issue below and comment that you want to work on it before you start.

Beginner friendly Everything

Open All

Issues somebody may already be working on are listed last. List everything by date

Roadmap question: prefill/decode disaggregation roles, now that the prefix cache pool exists?
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 12/100

NVIDIA/spark-local-inference#160 · Opened Oct 31, 2026

Maintainers usually reply within 1 day
Multi-host export: when rank zero rejects a peer's checkpoint completion, the reason is never reported
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA/spark-local-inference#156 · Opened Oct 30, 2026

Maintainers usually reply within 1 day
DeepSeek V4.1 Flash TP3 on GB10: a speculative round costs 1.5-2x a width-one round, mostly the routed MoE at 4-6 rows
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA/spark-local-inference#153 · Opened Oct 30, 2026

Maintainers usually reply within 1 day
DeepSeek V4.1 Flash TP3 on GB10: with four requests, the balanced policy verifies most rounds at width one; throughput is 8% faster on prose
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 38/100

NVIDIA/spark-local-inference#152 · Opened Oct 30, 2026

Maintainers usually reply within 1 day
GB10 TP2 serving: first request times out on RoCE collective (HCA 0, sequence 1) when both CX-7 rails are active; single rail works
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA/spark-local-inference#151 · Opened Oct 30, 2026

Maintainers usually reply within 1 day
GB10 TP2 export: b12x RoCE prep hard-defaults to GID index 3 and fails when the IPv4 RoCE v2 GID is elsewhere (Helios already scans)
Open
- Is something wrong?
Difficulty 4/5 1-2 days Newbie friendliness 54/100

NVIDIA/spark-local-inference#143 · Opened Oct 29, 2026

Maintainers usually reply within 1 day
DeepSeek V4.1 TP4 export on GB10: Xid 13 "Illegal Instruction Encoding" while warming compiled forwards at eb71702b (bce032c works)
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 36/100

NVIDIA/spark-local-inference#137 · 4 comments · Opened Oct 29, 2026

Maintainers usually reply within 1 day
Storage lowering lets Inductor recompute an in-place indexed copy (`x.copy_(x[idx])`), mixing rows across requests in batched speculative verification
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 50/100

NVIDIA/spark-local-inference#136 · Opened Oct 28, 2026

Maintainers usually reply within 1 day
GLM-5.3 TP2 live serving: [gMASK] reasoning leak, budget starvation, and tool-call rejections trace to one sampled degenerate draw
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/spark-local-inference#133 · 2 comments · Opened Oct 28, 2026

Maintainers usually reply within 1 day
Let preparation peers read their own checkpoint copy (multi-host load 252 s -> 45-50 s on GB10)
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/spark-local-inference#126 · 2 comments · Opened Oct 28, 2026

Maintainers usually reply within 1 day
DeepSeek V4.1 Flash: positions before token 128 return context-free logits when the reasoning-effort prefix is present
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA/spark-local-inference#118 · 2 comments · Opened Oct 28, 2026

Maintainers usually reply within 1 day
Released snapshot aliases hand KV pages back with stale row bytes; masked attention turns them into NaN (device error 6 under KV pressure)
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA/spark-local-inference#116 · Opened Oct 27, 2026

Maintainers usually reply within 1 day
Qwen3.8 `--mixed-prefill` exports fail on main: two regressions (317c1ecc, bce032c0)
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/spark-local-inference#115 · Opened Oct 27, 2026

Maintainers usually reply within 1 day
Loader ignores Control C to cancel loading
Open
- Is something wrong?
Difficulty 2/5 1-3 hours Newbie friendliness 62/100

NVIDIA/spark-local-inference#113 · Opened Oct 27, 2026

Maintainers usually reply within 1 day
b12x CompilePool: compiling in a worker that already described the job publishes nothing (lru_cache in mla_compress escapes evict_planning_artifacts)
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA/spark-local-inference#112 · Opened Oct 27, 2026

Maintainers usually reply within 1 day
Warm multi-host start re-hashes each peer's cached rank on one thread (7 min for a 280 GB DeepSeek V4.1 TP4 rank)
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 45/100

NVIDIA/spark-local-inference#93 · Opened Oct 26, 2026

Maintainers usually reply within 1 day
DS4.1 Flash disk Engram: 8–32 GiB cache leaves long first-touch prefill unchanged; warmed controls recover throughput
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA/spark-local-inference#78 · Opened Oct 25, 2026

Maintainers usually reply within 1 day
qwen38: support compressed-tensors NVFP4 expert-only checkpoints such as ThinkingCap-Qwen3.8-Flash-Next-NVFP4
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 20/100

NVIDIA/spark-local-inference#74 · Opened Oct 24, 2026

Maintainers usually reply within 1 day
Gemma 4 E4B: adaptive MTP verification fails with device error 6 when a width-4 round crosses into a new KV page
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 49/100

NVIDIA/spark-local-inference#61 · Opened Oct 23, 2026

Maintainers usually reply within 1 day
Nemotron 3.5 Bolt on GB10: intermittent "prepared model reported device error 6" mid-stream at concurrency 1
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA/spark-local-inference#52 · Opened Oct 23, 2026

Maintainers usually reply within 1 day
128 shared snapshot slots cap multi-user prefix reuse on a 4-Spark DeepSeek TP4 ring
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA/spark-local-inference#47 · 1 comment · Opened Oct 23, 2026

Maintainers usually reply within 1 day
Dense Qwen retains duplicate attention and FP8 input staging across graph capacities
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/spark-local-inference#46 · Opened Oct 23, 2026

Maintainers usually reply within 1 day
Dense Qwen retains duplicate MLP intermediates across serialized graph capacities
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/spark-local-inference#45 · Opened Oct 23, 2026

Maintainers usually reply within 1 day
Proposal: dense Qwen3.8-27B DFlash2 with BF16 and calibrated NVFP4 drafters
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 18/100

NVIDIA/spark-local-inference#44 · Opened Oct 23, 2026

Maintainers usually reply within 1 day
GLM argument-tags parser aborts the whole stream when a generated tool argument doesn't match its declared JSON type ("tool parameter value does not match its declared JSON type"); ChatML/Qwen parser coerces the same value instead
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/spark-local-inference#39 · Opened Oct 23, 2026

Maintainers usually reply within 1 day
RoCE ring: distant ranks have no fabric path — forward 2-hop transfers via flow label 16383 instead of the CPU relay
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/spark-local-inference#34 · Opened Oct 23, 2026

Maintainers usually reply within 1 day
OpenAI tool-call compatibility issue in flare that kills agentic loops
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA/spark-local-inference#32 · 1 comment · Opened Oct 22, 2026

Maintainers usually reply within 1 day
Support decision endpoint
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 38/100

NVIDIA/spark-local-inference#26 · Opened Oct 22, 2026

Maintainers usually reply within 1 day
Translate xhigh reasoning for deepseek v4.1 flash
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA/spark-local-inference#25 · 2 comments · Opened Oct 22, 2026

Maintainers usually reply within 1 day
DeepSeek V4.1 Flash (gb10): draft acceptance 24.5%/32.8% and draft depth locked to 5 — widths 2-5 reported by inspect but not servable
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/spark-local-inference#24 · Opened Oct 22, 2026

Maintainers usually reply within 1 day
Proposal: native BF16 MTP speculative decoding for dense Qwen3.8-27B (qwen38_dense), with results on SM120
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 15/100

NVIDIA/spark-local-inference#20 · Opened Oct 22, 2026

Maintainers usually reply within 1 day
Add an embeddable Rust SDK for in-process Flare inference
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/spark-local-inference#6 · 1 comment · 1 reaction · Opened Oct 21, 2026

Maintainers usually reply within 1 day
`flare run` fails at "Prepare execution": `cuGraphAddKernelNode_v2` CUDA_ERROR_INVALID_VALUE (qwen3.8-flash TP2, RTX PRO 5000 Blackwell 48 GB)
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA/spark-local-inference#2 · 4 comments · Opened Oct 21, 2026

Maintainers usually reply within 1 day
Congratulations!
Open
- Is something wrong?
Difficulty 1/5 Under an hour Newbie friendliness 1/100

NVIDIA/spark-local-inference#1 · 2 comments · 11 reactions · Opened Oct 20, 2026

Maintainers usually reply within 1 day
Serve the native engine load report at GET /v1/loads for cache-aware gateways
Possibly taken @JCTaz9 claimed this 2 days ago. Open
- Is something wrong?
Difficulty 3/5 1-2 days Newbie friendliness 35/100

NVIDIA/spark-local-inference#148 · Opened Oct 30, 2026

Maintainers usually reply within 1 day
Orbit agent eviction removes `flare download`'s default model store in `~/.cache/flare/packages`
Possibly taken @efegokdemir claimed this 2 days ago. Open
- Is something wrong?
Difficulty 4/5 1-2 days Newbie friendliness 45/100

NVIDIA/spark-local-inference#119 · Opened Oct 28, 2026

Maintainers usually reply within 1 day
GB10 TP2: kernel tuning MemoryError with MTP (glm-5.3-flash); vm.min_free_kbytes hides ~6.6 GB from the headroom check
Possibly taken A pull request linked to this issue is open or already merged. Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 45/100

NVIDIA/spark-local-inference#73 · 1 comment · Opened Oct 24, 2026

Maintainers usually reply within 1 day
Export fails with EINVAL when the package cache is on mergerfs (renameat2 RENAME_NOREPLACE unsupported)
Possibly taken @efegokdemir claimed this 7 days ago. Open
- Is something wrong?
Difficulty 3/5 1-2 days Newbie friendliness 54/100

NVIDIA/spark-local-inference#54 · Opened Oct 23, 2026

Maintainers usually reply within 1 day
Final prefill can starve behind a two-frame ordinary-decode pipeline
Possibly taken @Maxrio9 claimed this 9 days ago. Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/spark-local-inference#42 · Opened Oct 23, 2026

Maintainers usually reply within 1 day
Paged attention decode propagates NaNs from unused V-cache tails into live outputs (intermittent invalid-distribution errors on main)
Possibly taken A pull request linked to this issue is open or already merged. Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/spark-local-inference#14 · 1 comment · Opened Oct 22, 2026

Maintainers usually reply within 1 day

NVIDIA/spark-local-inference

Is NVIDIA/spark-local-inference beginner-friendly?

How to contribute to NVIDIA/spark-local-inference

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Get new issues in your inbox