NVIDIA/flare-local-inference

View on GitHub

Is NVIDIA/flare-local-inference beginner-friendly?

In the last 30 days, NVIDIA/flare-local-inference merged 12 of 31 pull requests from outside contributors, and its maintainers usually reply within 1 day. No beginner-friendly issue is open right now; the list below has the other issues we hold for it.

Stars: 566
Forks: 56
Open beginner issues: 0
Indexed issues: 33
Avg merge: 6h 18m
Merged PRs (30d): 22

Dominant language: Python
License: Apache-2.0
Last GitHub push: Oct 21, 2026
Latest indexed: Oct 22, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed

How to contribute to NVIDIA/flare-local-inference

There is no contributing guide, so start from the README and a few recently merged pull requests to see how changes are proposed.
It ships a Dockerfile or Docker Compose file, so you can run it without installing its whole toolchain.
Your contributions will be published under the project's Apache-2.0 license.
Follow the repository to hear when a new beginner-friendly issue opens, or look through the other issues below.

Beginner friendly Everything

Open All

Issues somebody may already be working on are listed last. List everything by date

uv.lock pins a flashinfer revision whose AutotuneEffort enum cannot express the tuning efforts ignition accepts
Open
- Is something wrong?
Difficulty 2/5 1-3 hours Newbie friendliness 65/100

NVIDIA/flare-local-inference#90 · Opened Oct 26, 2026

Maintainers usually reply within 1 day
DS4.1 Flash disk Engram: 8–32 GiB cache leaves long first-touch prefill unchanged; warmed controls recover throughput
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA/flare-local-inference#78 · Opened Oct 25, 2026

Maintainers usually reply within 1 day
qwen38: support compressed-tensors NVFP4 expert-only checkpoints such as ThinkingCap-Qwen3.8-Flash-Next-NVFP4
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 20/100

NVIDIA/flare-local-inference#74 · Opened Oct 24, 2026

Maintainers usually reply within 1 day
GB10 TP2: kernel tuning MemoryError with MTP (glm-5.3-flash); vm.min_free_kbytes hides ~6.6 GB from the headroom check
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 45/100

NVIDIA/flare-local-inference#73 · Opened Oct 24, 2026

Maintainers usually reply within 1 day
Gemma 4 E4B: adaptive MTP verification fails with device error 6 when a width-4 round crosses into a new KV page
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 49/100

NVIDIA/flare-local-inference#61 · Opened Oct 23, 2026

Maintainers usually reply within 1 day
Nemotron 3.5 Bolt on GB10: intermittent "prepared model reported device error 6" mid-stream at concurrency 1
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA/flare-local-inference#52 · Opened Oct 23, 2026

Maintainers usually reply within 1 day
Literal <think>/</think> text in model output is parsed as reasoning delimiters
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA/flare-local-inference#48 · 1 comment · Opened Oct 23, 2026

Maintainers usually reply within 1 day
128 shared snapshot slots cap multi-user prefix reuse on a 4-Spark DeepSeek TP4 ring
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA/flare-local-inference#47 · 1 comment · Opened Oct 23, 2026

Maintainers usually reply within 1 day
Dense Qwen retains duplicate attention and FP8 input staging across graph capacities
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/flare-local-inference#46 · Opened Oct 23, 2026

Maintainers usually reply within 1 day
Dense Qwen retains duplicate MLP intermediates across serialized graph capacities
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/flare-local-inference#45 · Opened Oct 23, 2026

Maintainers usually reply within 1 day
Proposal: dense Qwen3.8-27B DFlash2 with BF16 and calibrated NVFP4 drafters
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 18/100

NVIDIA/flare-local-inference#44 · Opened Oct 23, 2026

Maintainers usually reply within 1 day
GLM argument-tags parser aborts the whole stream when a generated tool argument doesn't match its declared JSON type ("tool parameter value does not match its declared JSON type"); ChatML/Qwen parser coerces the same value instead
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/flare-local-inference#39 · Opened Oct 23, 2026

Maintainers usually reply within 1 day
RoCE ring: distant ranks have no fabric path — forward 2-hop transfers via flow label 16383 instead of the CPU relay
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/flare-local-inference#34 · Opened Oct 23, 2026

Maintainers usually reply within 1 day
OpenAI tool-call compatibility issue in flare that kills agentic loops
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA/flare-local-inference#32 · 1 comment · Opened Oct 22, 2026

Maintainers usually reply within 1 day
Support decision endpoint
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 38/100

NVIDIA/flare-local-inference#26 · Opened Oct 22, 2026

Maintainers usually reply within 1 day
Translate xhigh reasoning for deepseek v4.1 flash
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA/flare-local-inference#25 · 2 comments · Opened Oct 22, 2026

Maintainers usually reply within 1 day
DeepSeek V4.1 Flash (gb10): draft acceptance 24.5%/32.8% and draft depth locked to 5 — widths 2-5 reported by inspect but not servable
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/flare-local-inference#24 · Opened Oct 22, 2026

Maintainers usually reply within 1 day
Proposal: native BF16 MTP speculative decoding for dense Qwen3.8-27B (qwen38_dense), with results on SM120
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 15/100

NVIDIA/flare-local-inference#20 · Opened Oct 22, 2026

Maintainers usually reply within 1 day
Paged attention decode propagates NaNs from unused V-cache tails into live outputs (intermittent invalid-distribution errors on main)
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/flare-local-inference#14 · 1 comment · Opened Oct 22, 2026

Maintainers usually reply within 1 day
Add an embeddable Rust SDK for in-process Flare inference
Open
- Is something wrong?
Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/flare-local-inference#6 · 1 comment · 1 reaction · Opened Oct 21, 2026

Maintainers usually reply within 1 day
`flare run` fails at "Prepare execution": `cuGraphAddKernelNode_v2` CUDA_ERROR_INVALID_VALUE (qwen3.8-flash TP2, RTX PRO 5000 Blackwell 48 GB)
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA/flare-local-inference#2 · 4 comments · Opened Oct 21, 2026

Maintainers usually reply within 1 day
Congratulations!
Open
- Is something wrong?
Difficulty 1/5 Under an hour Newbie friendliness 1/100

NVIDIA/flare-local-inference#1 · 2 comments · 11 reactions · Opened Oct 20, 2026

Maintainers usually reply within 1 day
`ignition sign` writes an empty license into the signed package
Possibly taken @mark-yong claimed this today. Open
- Is something wrong?
Difficulty 3/5 1-2 days Newbie friendliness 35/100

NVIDIA/flare-local-inference#81 · Opened Oct 25, 2026

Maintainers usually reply within 1 day
docker build at main fails: helios include_str!s ignition/recipes templates the image never ships
Possibly taken @JohannesBertens claimed this 1 day ago. Open
- Is something wrong?
Difficulty 2/5 1-3 hours Newbie friendliness 30/100

NVIDIA/flare-local-inference#76 · Opened Oct 25, 2026

Maintainers usually reply within 1 day
Export fails with EINVAL when the package cache is on mergerfs (renameat2 RENAME_NOREPLACE unsupported)
Possibly taken @efegokdemir claimed this 1 day ago. Open
- Is something wrong?
Difficulty 3/5 1-2 days Newbie friendliness 54/100

NVIDIA/flare-local-inference#54 · Opened Oct 23, 2026

Maintainers usually reply within 1 day
Final prefill can starve behind a two-frame ordinary-decode pipeline
Possibly taken @Maxrio9 claimed this 3 days ago. Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/flare-local-inference#42 · Opened Oct 23, 2026

Maintainers usually reply within 1 day

NVIDIA/flare-local-inference

Is NVIDIA/flare-local-inference beginner-friendly?

How to contribute to NVIDIA/flare-local-inference

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Get new issues in your inbox