Analysing allocator CPU & memory issues

Open
#1,140 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
c
Domain
performance

Research direction

Start with the production CPU and memory graph and compare the same workload under mimalloc and glibc, since no minimal reproducible example or source location is provided. Gather allocator-internal measurements during an occurrence and document the application conditions; done means identifying evidence for the spike or narrowing it to a reproducible allocator issue.

Written by the indexing model from the issue text.

Description

I have a problem with mimalloc and I'm a bit unsure how to approach it. My application consumes a lot of memory using glibc allocator, so I switched to mimalloc. It works wonders, except when it doesn't. Sporadically I can see following pattern in my application:

Image

I know for a fact that there is no user code in the application that can spike into 100% CPU and 5x memory usage. I am pretty sure it's an allocator (and this graph looks very different on glibc allocator). This only happens in production environment on real data, and I haven't been able to create a MRE to showcase the isssue.

Is there any way I can tell why/how this is happening, if I am unable to shrink the issue to the MRE? Maybe I can create a worker to log some of the internal mimalloc structs, so when this happens we have some numbers to work from? At this point I'm a bit lost because I haven't had such problems before and I am not sure how to approach it.

Dominant language
C
Stars
13.6k
Forks
1.2k
Avg merge
18h 12m
Merged PRs (30d)
20

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from microsoft/mimalloc

All issues in microsoft/mimalloc

Similar issues

More C issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.