Landing next

This is a release-tracking list, not a task with a change to make. Read CONTRIBUTING.md and check the referenced issues and pull requests before considering any listed work; the issue says to comment on the relevant pull request if work overlaps. Done means avoiding duplicate work, not implementing a change from this issue.

TensorFold is the native engine

TensorFold's engine is written in Zig and drives Metal and CUDA directly, with no Python in the serving path. The Python engine is frozen at 0.6.6 and is being retired: nothing more merges into it. This issue says what that means for pull requests and issues, and what lands next. We update it as work lands.

New pull requests

• Target main (zig-flashnext follows it). CONTRIBUTING.md has the rules and ZIG-PREVIEW.md how to build, run and test.
• House rules: a drafted reply equals "draft": false, a resumed prompt equals a fresh one, and concurrent streams equal their solo runs; no precision traded for speed; no file over about 600 lines; one-line comments; zig build test passes.
• A new family, a new backend, or anything that changes a model's arithmetic starts as an issue.
• We rebase for you. Commits land under your GitHub noreply address; main's history keeps no personal addresses and no AI-tool trailer lines, so a pull request whose commits carry either is landed as re-authored commits and shows as closed rather than merged.

The Python engine

Pull requests that change the Python engine close with thanks and a note on where the idea lives on the native engine. Python-engine bug reports close the same way, pointing at the native issue that tracks the feature.

On the native engine today (1.0.5)

• Nemotron 3.5 Lightning on Metal (M1 to M5) and CUDA (GB10, and Ampere 8.6 cards such as the RTX 3090).
• Qwen3.8 Flash Next on an M5 Ultra. Speed-up mode runs it on two Macs from its guide's recorded setup; the release binary's dump-free two-Mac path connects on a user's pair (#567) and still needs our gate.
• GLM-5.3-Flash on one 256 GB M5 Ultra or two M5 Ultras, with concurrency, prompt reuse and --learn.
• Qwen3.8-27B with its DFlash2 drafter (M5 Max, M3 Ultra), and Qwen3.5-2B.
• Living Weights (--slide, experimental): a Nemotron server on a Mac learns facts into its own weights.

Landing next

1.0.5 is out. On main for 1.0.6 so far: Living Weights on CUDA (#591, #599, #632), the GLM fixes (two-Mac prose drafting, #600, #622), NVIDIA's NVFP4 Nemotron checkpoint on CUDA (#595, #629, #633 and our staged prompt kernel), the CUDA core work (#602, #605), the lane core's chunked prompt hook (#623), serve flags (#624), the AMD/HIP pieces (#607, #609, #611, #612, #613, #625), and fixes #604, #616 and #619. Next: structured output and tool_choice=required for Nemotron (#596, in progress), #578 (the 27B serves concurrent requests, in its gate), image and video input (#598, #614, #618), #606, #608, #519, #615 (waiting on a fix), #630 (waiting on a rebase), sm_120 in the Linux archives (#610) and the GB10 glue-kernel tuning (#627). Next up: an A100 (compute capability 8.0) build and the decisions noted on the open pull requests.

Where help is most useful

Flash Next on native CUDA (#472), GLM-5.3-Flash and full GLM-5.3 on native CUDA (#593), image input (#565), structured output (#564), logprobs (#449), an expert pool for cards smaller than the model (the gate is in #554), and AMD through the HIP runtime (#426).

On main for 1.0.6

• #591: Living Weights learns on a GB10 (--slide on the native CUDA server), with a fix for repeated saves and the trainer's memory counted in the CUDA memory budget.
• #604: a NaN or infinite value written the way Python's str() does no longer crashes the server.
• #607 and #609: the HIP build accepts RDNA2 and RDNA3 cards, with a capabilities table from RDNA2 to RDNA4.
• #616: main passes zig fmt --check again.
• #619: nested completion prompt ids outside the vocabulary, or booleans, get a 400 as top-level ids do.
• #602: the CUDA engine's 4-bit repack lives beside its format (cuda/qlinear.zig, affine4_pack.cu), every reply and kernel byte-equal.
• GLM-5.3-Flash on two Macs times its draft windows at load: prose decode back above 1.0.4 (story 116.5 tok/s against 106.1 in 1.0.5).
• #600: GLM loads drop the checkpoint's cached pages first, so a load right after a download no longer waits on free memory.
• #622: a new GLM stream no longer overwrites another conversation's kept prefix.
• #599: Living Weights keeps a lesson only when every fact came back; any other lesson is taken out whole before the save (fixes #589).
• #605, #595, #629, #633: grouped CUDA experts, NVFP4 projections and experts, and float64 checks of every FP8/NVFP4 product.
• NVIDIA's NVFP4 Nemotron 3.5 Lightning serves on CUDA (GB10): prompts as fast as the MLX 4-bit checkpoint, every reply exact.
• #632: Living Weights lessons on CUDA repeat bit for bit.
• #623 and #624: a prompt pass a chunk a round between other streams' rounds, and serve flags a backend opts into.
• #611, #612, #613, #625: HIP runtime pieces and shared core modules for the coming HIP engine.

Landed in 1.0.5

• #542: a CUDA request without top_k samples with 0.6.6's 20 (fixes #515).
• #541 and #531: /metrics reports pinned host memory on CUDA and prompt tokens restored from kept prompt states.
• #560 and #513: a request keeps its own API-key label across a key-file reload, and the server frees what it holds when it stops.
• #543: zig build test runs the HTTP server's tests on Linux too.
• #539: logprob.cu compiles with nvcc 13.3 (fixes #535).
• #493: the native Qwen3.5-2B serves untied heads, and qwen3_5 checkpoints open their engine by geometry.
• #551: tensorfold serve --chat-template FILE.
• #534: the CUDA and GLM recipes separate Python 0.6.6 features from native support.
• #561: the --dashboard page's dial, tiles and decode-rate chart.
• #455: the lane core takes an external drafter in front of any target backend.
• #463: an opt-in HIP runtime with one exact affine matmul, the start of AMD support.
• #496, #532, #533: an EXL3 reference decoder with its oracle fixture, exact token-local tokenizer bytes, and rank-five safetensors shapes.
• #512: Flash Next on two Macs no longer waits forever for a handoff that never arrives.
• RUNBOOK, recipes and refusal messages match the release.
• #497, #499, #518, #502: Nemotron on Metal keeps prompt states between turns (a 19.8k-token chat's second turn prefills in
  0.16 s instead of 8.1 s), keeps shared prefixes on disk with --learn, shares rounds of up to 128 rows, and runs its
  routed experts 10-15% faster at 64 and 128 rows, every reply equal to the cache-off one.
• #467: Nemotron on CUDA keeps prompt states between turns on the 2,048-row prompt grid, every reply equal to the cache-off one.
• #558 and #482: a shared CUDA expert plan, and block-FP8 lane projections for the Kolibri-1 port (#481).
• #487: GB10 KV caches in the display carveout, opt-in and experimental.
• Heat bands (TF_HEAT_HIGH, TF_HEAT_LOW) pause Nemotron CUDA prompt chunks while the GPU runs hot (#339).
• #572: logprobs and top_logprobs on the native CUDA server for Nemotron, from the verified target logits.
• #571: editing, deleting or regenerating the last turn resumes from a state kept before the latest user message.
• #570: Living Weights learns from Japanese and other text written without spaces.
• GLM-5.3-Flash on one M5 Ultra (the MTP head stays on disk when only it is over the load limit), faster GLM prompts and decode on M5 Ultras, and lighter resident prompt states.
• A two-Mac server whose other rank never answers names macOS Local Network access (#567).
• Living Weights' leak check ignores everyday words such as "like" (#590).
• #587: one quantized-weight interface for the CUDA engine (cuda/qlinear.zig); Nemotron's 4-bit projections run through it unchanged.
• #573: the Qwen3.8-27B compiles on macOS 27 again (fixes #579).
• #582: a greedy-only engine refuses a sampled request with a 400 before admission, Flash Next included.
• #576: the CUDA tensorfold runs models, info and pull without opening the driver, and pull keeps hub paths inside the snapshot.
• #588: the native reply's seconds is its decode time.
• top_k has no default filter on CUDA, as on Metal (#515).
• #594: a completion request with nested prompt ids past 2^32 no longer crashes the server.
• The repository checks out on Windows again (eight golden file names), and zig build test refuses names Windows can't check out.
• #584: GLM-5.3-Flash keeps prompt states for prompts under 4,096 tokens.
• GLM_LOAD_LIMIT_GB loads past 70% of RAM only when asked, with the risk logged (#498, #521, #580).
• #508: --learn keeps a free-disk floor and pauses instead of filling the disk.