bench: N=100 hugepage spawn numbers (deferred from #230)
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 38/100
- Issue type
- Feature
- Clarity
- Clearly specified
- Activity status
- Quiet
- Domain
- infrastructure, performance
Research direction
Start by reading bench/live-fork-pause-window/bench-hugepages.py and the N=4 results in PR #230; the issue says the script is correct and needs a clean host with patched Firecracker v1.16.0-dev and a fresh snapshot. Follow the listed setup steps, including scripts/netns-setup.sh, then run the specified N=100 command. Post the CSV and summary table here and in bench/live-fork-pause-window/RESULTS-v0.5.md, documenting whether the acceptance criteria are met or whether larger-parent runs are needed.
Written by the indexing model from the issue text.
Description
Context. #230 (merged in v0.5.2) shipped the hugepage-backed memfd path end-to-end + bench/live-fork-pause-window/bench-hugepages.py. @theflashwin ran the bench at N=4 on his hardware (numbers in the PR README) and noted he didn't have compute to push to N=100.
I committed in PR review to running N=100 on my dev box — that didn't happen because of two stacked issues that surfaced during the attempt:
What went wrong
- Dev-box Firecracker version drift.
/usr/local/bin/firecrackerwas v1.12.0 (unpatched upstream). TheMemfdSharedbackend requires the deeplethe-patched FC fork (mem_backend.shared = truefield is forkd-specific) — the patched binary was sitting at~/firecracker/build/.../firecrackerv1.16.0-dev but wasn't on PATH. - Snapshot bitcode incompat. Re-snapping with the patched FC works, but old snapshots (taken with v1.12) can't load on v1.16 (
bitcode errorfrom FC's snapshot deserializer). Need a fresh snapshot for the bench.
I got partway through both fixes (installed patched FC, rebuilt a benchsrc-v116 parent, started bench) before hitting an unrelated host issue and rolling back. The bench script itself is correct — the failures were environmental.
To-do
- Provision a clean host (or container) with patched FC v1.16.0-dev installed by default
- Build a small benchsrc snapshot fresh under that FC
-
sudo bash scripts/netns-setup.sh 100to provision per-child netns (the bench needs N netns up front; the script is idempotent) - Bump
/proc/sys/vm/nr_hugepagesto ≥(parent_mem_mib / 2) + n * 2— for a 512 MiB parent + N=100 that's 257 + 200 = ~460 pages; round up to 1024 for headroom - Run
sudo python3 bench/live-fork-pause-window/bench-hugepages.py --source-tag <fresh-tag> --n 100 --iterations 5 --branch-mode diff - Post the CSV + summary table (interleaved p50/p99 spawn_ms, ms/child, pause_ms for baseline vs hugepages) here and in
bench/live-fork-pause-window/RESULTS-v0.5.md
Acceptance
A reproducible N=100 number that either:
- Validates the hugepages path (≥1.3× spawn-time speedup at p50 or measurable pause-window reduction), OR
- Surfaces that the win only shows for memory-heavy parents (in which case the bench should grow a
--mem-mibknob and we re-run at 2 GiB + 4 GiB parents to characterize where it pays off)
Either outcome is shippable as a follow-up patch with a one-paragraph note in the README.
- Dominant language
- Rust
- Stars
- 2.9k
- Forks
- 227
- Avg merge
- 5h 12m
- Merged PRs (30d)
- 12
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from deeplethe/forkd
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
deeplethe/forkd#313 · 2 comments ·
Maintainers usually reply within 1 day
-
backend:firecracker
Difficulty 5/5 Over a week Newbie friendliness 25/100
deeplethe/forkd#317 · 6 comments ·
Maintainers usually reply within 1 day
-
backend:firecracker bug
Difficulty 5/5 Over a week Newbie friendliness 28/100
deeplethe/forkd#301 · 4 comments ·
Maintainers usually reply within 1 day
-
backend:gvisor enhancement help wanted
Difficulty 5/5 Over a week Newbie friendliness 25/100
deeplethe/forkd#21 · 2 comments ·
Maintainers usually reply within 1 day
-
backend:firecracker enhancement help wanted needs-verification
Difficulty 4/5 3-5 days Newbie friendliness 48/100
deeplethe/forkd#3 · 2 comments ·
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
memory_patch_page drops the `#` of headings like `## Notes on C#`Possibly taken @viniciusdsandrade claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
akitaonrails/ai-memory#1859 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
wealthfolio/wealthfolio#2376 ·
Maintainers usually reply within 1 day
-
kind/bug kind/security
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
spiceai/spiceai#15826 · 1 comment ·
Maintainers usually reply within 1 day
-
area:cli bug dotnet good first issue priority:medium
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day