Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

sys_mmap: eager zero-fill on MAP_ANONYMOUS makes sparse mappings ~5x slower than Linux

Open
  • Is something wrong?
#165 0 comments 0 reactions 1 assignee View on GitHub

Maintainers usually reply within 2 days

@Max042004 is already working on this.

Since Jul 7, 2026.

  • #203 by @Max042004 — closed without merging
  • #307 by @maxliu0 — closed without merging
  • #372 by @Max042004 — open

Assessment

This issue has not been assessed yet.

Description

Summary

sys_mmap zeroes the entire region synchronously for every MAP_ANONYMOUS mapping (src/syscall/mem.c:2439):

/* Zero the mapped region */
memset((uint8_t *) g->host_base + result_off, 0, length);

This is functionally correct, but pays the full zero-fill cost up front. Linux backs new anonymous pages with the shared zero page and zeroes lazily on first touch, so workloads that map a region and touch only a fraction of it (e.g. CPython pymalloc 256 KiB arenas) pay for pages they never use.

Measurements (main @ e6b3455, Apple M-series)

Static aarch64 binary looping mmap(256 KiB, MAP_ANON|MAP_PRIVATE) → touch → munmap, 5000 iterations:

Workload elfuse OrbStack (Linux)
1-byte touch 7,041 ns/cycle 1,392 ns/cycle
touch every 4 KiB page (64 pages) 6,934 ns/cycle 8,904 ns/cycle

On elfuse the 1-byte-touch cost equals the full-touch cost — the whole region is paid for at mmap time. On Linux the cost scales with pages actually touched.

Downstream effect: CPython for _ in range(5M): pass runs ~11.5 ns/iter on elfuse vs ~7.3 on OrbStack with the same python3.12 binary; the gap mostly disappears with PYTHONMALLOC=malloc (bypassing pymalloc's mmap-backed arenas), confirming it is mmap-bound rather than interpreter-bound.

Notes toward a fix

  • The lazy machinery already exists: guest_materialize_lazy() (src/core/guest.c:3265) plus the translation-fault path in src/syscall/proc.c already perform fault-time PTE creation, per-2MiB-block zeroing, and TLBI — but only for regions carrying the noreserve flag. Extending this to all anonymous mappings is the natural fix. Open questions: stale-PTE invalidation cost on address reuse (currently guest_invalidate_ptes per mmap on the NORESERVE path), and concurrent faults on the same range from multiple vCPUs.
  • Host madvise(MADV_DONTNEED) is not a shortcut: on macOS it behaves like MADV_FREE, and replacing the memset with it regressed the benchmark to 14–30 µs/cycle in a local experiment.
  • The memset itself is only ~25% of the gap (measured by unsoundly disabling it: ~5.6 µs/cycle remains). The rest is the per-syscall region-tracking / guest_extend_page_tables / guest_update_perms / TLBI path — lazy materialization also skips most of that work for pages never touched.

Reproduction

#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <time.h>

int main(int argc, char **argv) {
    int N = (argc > 1) ? atoi(argv[1]) : 5000;
    size_t SZ = 256 * 1024;
    struct timespec t0, t1;
    clock_gettime(CLOCK_MONOTONIC, &t0);
    for (int i = 0; i < N; i++) {
        char *p = mmap(NULL, SZ, PROT_READ | PROT_WRITE,
                       MAP_ANON | MAP_PRIVATE, -1, 0);
        p[0] = 1; /* or: for (size_t j = 0; j < SZ; j += 4096) p[j] = 1; */
        munmap(p, SZ);
    }
    clock_gettime(CLOCK_MONOTONIC, &t1);
    double ns = (t1.tv_sec - t0.tv_sec) * 1e9 + (t1.tv_nsec - t0.tv_nsec);
    printf("%.1f ns/cycle\n", ns / N);
    return 0;
}

aarch64-linux-musl-gcc -O2 -static -o bench_mmap bench_mmap.c
build/elfuse bench_mmap 5000
Dominant language
C
Stars
271
Forks
28
Avg merge
2d 18h
Merged PRs (30d)
19

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from sysprog21/elfuse

All issues in sysprog21/elfuse

Similar issues

More C issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.