Page-size check prevents mmap for blob-aligned checkpoints
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 75/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- cpp
- Domain
- machine-learning, performance
Research direction
Start at ChooseMode and inspect the checkpoint length check that compares against the system page size, then trace the mmap and ordinary-read fallback paths. Done means blob-aligned lengths such as the 4B checkpoint can use file-backed mappings, while lengths that are not divisible by kBlobAlign still use ordinary reads.
Written by the indexing model from the issue text.
Description
Problem
ChooseMode disables mmap when the checkpoint size is not divisible by the system page size, even if the file is padded to the blob format's 256-byte alignment.
For example, the 4B checkpoint is 5,401,208,832 bytes: divisible by 256, but with a 2,048-byte remainder on a 4,096-byte page system. It therefore falls back to copying the weights into allocated memory instead of using file-backed mappings.
Proposed fix
Relax the file-length alignment requirement from the base page size to kBlobAlign (256 bytes).
The OS accepts partial-page mappings, but inference kernels can read multiple vectors ahead. Retain the blob-alignment guard and fall back to ordinary reads for lengths that do not meet it. Arbitrary-length mmap support would require separate handling of those reads.
Covered by #1032.
- Dominant language
- C++
- Stars
- 7k
- Forks
- 660
- Avg merge
- 20h 43m
- Merged PRs (30d)
- 33
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from google/gemma.cpp
-
Difficulty 4/5 3-5 days Newbie friendliness 68/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
Difficulty 4/5 3-5 days Newbie friendliness 64/100
All issues in google/gemma.cpp
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
AXERA-TECH/ax-llm#77 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
games-on-whales/wolf#509 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
bug-unconfirmed
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
NVIDIA/cuda-samples#453 ·