Not able to prevent additional mmap-s beside reserved memory

Open
#1,228 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
28/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
c

Research direction

Start by reproducing the issue with the listed MIMALLOC_* settings, disallow_os_alloc=1, and the mmap/munmap logging described in the report. Compare the 64 KiB initialization mappings with the reserved huge-page mappings and determine whether the observed behavior is expected or indicates a bug. Done means the mmap behavior and early ENOMEM result are explained, with a focused regression test or a clearly documented limitation if applicable.

Written by the indexing model from the issue text.

Description

I'm using mimalloc as drop-in malloc replacement. For my usecase I want to preallocate big area of 2M huge-pages and make mimalloc use that area for all allocations unless it's fully used in which case I want to fallback to additional memory (regular mmap).

The preallocated area should be enough for 99.9% of runtime and only in extreme cases (e.g. bugs and leaks) I expect additional mmaps.

I'm setting these environment options:

MIMALLOC_RESERVE_HUGE_OS_PAGES=14
MIMALLOC_PURGE_DELAY=-1
MIMALLOC_VERBOSE=1
MIMALLOC_SHOW_STATS=1
MIMALLOC_SHOW_ERRORS=1

I've also added logging around mmap/munmap calls. I expected not to see any additional calls besides initial 14 calls to mmap with 1 GiB size. However I still see numerous mmap calls with size 64kb at init time.

When I tried to setting disallow_os_alloc=1, I've got ENOMEM early in application init, definitely before 14GB were exhausted. I looks like mimalloc tries to allocate from OS even if there is plenty of free memory in preallocated area.

Am I misunderstanding something?

There's similar recent question https://github.com/microsoft/mimalloc/issues/1218, but AFAIU it only applies to arenas larger than 16GB while in my case I have 14GB.

Dominant language
C
Stars
13.6k
Forks
1.2k
Avg merge
18h 12m
Merged PRs (30d)
20

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from microsoft/mimalloc

All issues in microsoft/mimalloc

Similar issues

More C issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.