Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Track cross-process JIT compilation stampede on cold cache misses

Open
  • Is something wrong?
#1,408 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
rust
Domain
performance

Research direction

Start by reading the JIT cache implementation and cutile-book/guide/jit-compilation.md; the issue gives no implementation file or test to begin from. The goal is to coordinate compilation across processes for a missing entry where supported, document network-filesystem behavior, and fall back to redundant compilation without blocking indefinitely when coordination is unavailable. Revisit this only if cold-start stampedes become a significant real-world cost.

Written by the indexing model from the issue text.

Description

cutile-rs

Migrated from NVlabs/cutile-rs#205, opened by @goog00 on 2026-08-05. 3 comments remain on the original.

Context

Follow-up to #193.

The in-process JIT cache uses OnceCell to compile each kernel only once per process. The persistent disk cache can be shared across processes, but compilation is not deduplicated across process boundaries.

When multiple processes sharing the same cache directory cold-start concurrently — commonly with one process per GPU — they can all miss the same key, spawn tileiras, and compile the same kernel.

This is correct: atomic temporary-file + rename writes ensure that the cache converges on one identical entry. However, the first parallel launch wastes N−1 compiles. Subsequent starts hit the populated cache.

Decision for #193

Merge without a per-key cross-process compilation lock and track the limitation here.

A blocking flock was left out because:

  • A slow or hung lock holder could block every waiter.
  • Waiters cannot reliably distinguish active compilation from a stuck compiler.
  • File-lock semantics and reliability can vary across NFS deployments, where shared caches are likely to live.
  • The stampede affects only a cold cache and converges correctly afterward.

Users can avoid the initial stampede by warming the shared cache with one process before launching parallel workers. This workaround is documented in cutile-book/guide/jit-compilation.md.

Follow-up

Revisit cross-process compilation coordination if cold-start stampedes become a significant real-world cost.

Any future solution should:

  • Where cross-process coordination is supported, allow only one process to compile a missing cache entry while the others reuse its result.
  • Document whether this coordination is supported on network filesystems such as NFS.
  • Where coordination is unavailable, fall back to redundant compilation rather than fail a kernel launch or block indefinitely.
Dominant language
Rust
Stars
3.8k
Forks
325
Avg merge
6d 8h
Merged PRs (30d)
66

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA/cuda-rust

All issues in NVIDIA/cuda-rust

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.