`MI_HUGE_PAGE_ABANDON` cannot be enabled (compile error on v2.5.x, SIGSEGV on dev), and cross-thread huge block free bypasses `MIMALLOC_PURGE_DECOMMITS` on v2 (unconditional MADV_FREE → silent data zeroing on reuse)
メンテナーはふだん 1 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 初心者へのやさしさ
- 35/100
- issue の種類
- バグ
- 明瞭さ
- 明確に書かれている
- 活発さ
- 活発
- 技術スタック
- c, cmake, linux
- 領域
- backend, performance
調査の方向性
Start with _mi_segment_huge_page_reset and _mi_os_reset in the v2 cross-thread huge block free path, then trace how delayed frees are processed and how purge options are applied. Reproduce the supplied 32 MiB cross-thread free scenario on v2.5.x and dev, including a debug build; done when the default configuration no longer allows reused live data to be zeroed and MIMALLOC_PURGE_DECOMMITS=0 retains its intended behavior.
索引モデルが issue の本文から書いたものです。
説明
Summary
Three related problems found while investigating pytorch/pytorch#200905 (silent data zeroing on aarch64, triggered via PyTorch's DataLoader multi-threaded frees). All three reproduce with plain mimalloc + the documented MI_EXTRA_CPPDEFS CMake option — no PyTorch involved.
Verified on: aarch64, Linux 6.6, gcc 12.2, cmake. Versions checked: v2.5.2 (2406cc5, pinned by PyTorch), v2.5.3 (9593e865, latest v2 release), v2 dev tip (9b6a46d3), v3 main3 tip (1d8fd85aa).
1) MI_HUGE_PAGE_ABANDON=1 does not compile on v2.5.2 / v2.5.3
$ cmake -S . -B build -DMI_EXTRA_CPPDEFS="MI_HUGE_PAGE_ABANDON=1" -DMI_BUILD_SHARED=OFF -DMI_BUILD_OBJECT=OFF -DMI_BUILD_TESTS=OFF
$ cmake --build build
...
src/free.c:303:14: error: 'mi_segment_t' {aka 'struct mi_segment_s'} has no member named 'page_kind'
free.c:303 sits inside #if MI_HUGE_PAGE_ABANDON and references segment->page_kind, which does not exist in mi_segment_t on the v2.5.x line (the field is kind, the constant MI_SEGMENT_HUGE). Since the option is commented out by default in types.h, this branch has evidently never been compiled. (The v2 dev branch reworked mi_segment_t and now has a page_kind field, so it compiles there — see next point.)
2) MI_HUGE_PAGE_ABANDON=1 SIGSEGVs on every allocation > 16 MiB (v2 dev)
On the dev tip the compile error is gone, but the mode still cannot be used. Minimal repro:
#include <mimalloc.h>
int main(void) {
void* p = mi_malloc(32u * 1024 * 1024); /* > MI_LARGE_OBJ_SIZE_MAX (16MiB) => huge block */
mi_free(p);
return 0;
}
$ ./test_huge
Segmentation fault
#0 mi_page_queue_enqueue_from_ex
#1 _mi_malloc_generic
#2 main
fault address: 0xbe0 (= &NULL->pages[bin])
Root cause: with the mode on, mi_large_huge_page_alloc sets mi_page_set_heap(page, NULL) (page.c:973 on v2.5.x / :945 on dev) — huge pages have no owning heap and are in no queue — but _mi_malloc_generic still unconditionally does:
// move singleton pages to the full queue
if (page->reserved == page->used) {
mi_page_to_full(page, mi_page_queue_of(page)); // mi_page_queue_of derefs the NULL heap
}
The crash boundary is exactly 16 MiB (MI_LARGE_OBJ_SIZE_MAX). A one-line guard fixes it:
if (page->reserved == page->used
#if MI_HUGE_PAGE_ABANDON
&& !mi_page_is_huge(page)
#endif
) {
mi_page_to_full(page, mi_page_queue_of(page));
}
(plus the page_kind → kind compile fix on v2.5.x). With those two fixes I verified the mode works and eliminates the cross-thread MADV_FREE below — but given the comment in types.h (huge pages in this mode are invisible to heap walks / mi_heap_destroy), maybe the mode is intentionally dormant; in that case point 3 below still needs a fix on its own.
3) Cross-thread huge block free bypasses MIMALLOC_PURGE_DECOMMITS (unconditional MADV_FREE) — silent data zeroing after reuse (v2.5.2 / v2.5.3 / dev, default options)
This is the root cause of pytorch/pytorch#200905 and reproduces with default options, without the abandon mode.
To be clear, MADV_FREE itself is not the problem: it is an intentional optimization when MIMALLOC_PURGE_DECOMMITS=0, and every regular purge path (_mi_os_purge_ex) already respects that option — default 1 = MADV_DONTNEED. This is also mimalloc's own stated design intent, from the comment in prim/unix/prim.c:
With the default
MIMALLOC_PURGE_DECOMMITS=1we ensure that by defaultMADV_DONTNEEDis used.
The bug is solely that the cross-thread huge block free path bypasses this control: _mi_segment_huge_page_reset calls _mi_os_reset (i.e. MADV_FREE) unconditionally, so with default options a cross-thread mi_free of a >16 MiB block tags the pages lazyfree. The marks survive arena reuse (_mi_prim_reuse is a no-op on Linux), and a later MADV_PAGEOUT — or kswapd reclaim under memory pressure — silently zeroes the pages of the live allocation.
strace of the cross-thread mi_free with default options confirms it:
madvise(0x5356b401000, 33812480, MADV_FREE) = 0
Full repro (default options, no env vars):
#include <pthread.h>
#include <stdio.h>
#include <string.h>
#include <sys/mman.h>
#include <mimalloc.h>
#ifndef MADV_PAGEOUT
#define MADV_PAGEOUT 21
#endif
#define BLOCK (32u * 1024 * 1024)
static void* worker(void* arg) {
void** box = (void**)arg;
mi_free(box[0]); /* free from a different thread */
return NULL;
}
int main(void) {
void* p1 = mi_malloc(BLOCK);
memset(p1, 0xCD, BLOCK);
void* box[1] = { p1 };
pthread_t t;
pthread_create(&t, NULL, worker, box);
pthread_join(t, NULL);
for (int i = 0; i < 800; i++) { /* let the owner heap recycle the block */
void* q = mi_malloc(BLOCK);
mi_free(q);
}
void* p2 = mi_malloc(BLOCK);
printf("reused: %d\n", p2 == p1);
size_t zeros_before = 0;
for (size_t i = 0; i < BLOCK; i += 4096) {
unsigned char* page = (unsigned char*)p2;
size_t nz = 0;
for (size_t j = 0; j < 4096; j++) if (page[i+j] == 0) nz++;
if (nz == 4096) zeros_before++;
}
printf("zero pages before: %zu / %u\n", zeros_before, BLOCK/4096);
unsigned long base = ((unsigned long)p2 + 4095) & ~4095UL; /* mimalloc pointers carry a header offset */
size_t len = (BLOCK - (base - (unsigned long)p2)) & ~4095UL;
int rc = madvise((void*)base, len, MADV_PAGEOUT); /* what kswapd does under pressure */
printf("madvise(PAGEOUT) rc=%d\n", rc);
size_t zeros_after = 0;
for (size_t i = 0; i < BLOCK; i += 4096) {
unsigned char* page = (unsigned char*)p2;
size_t nz = 0;
for (size_t j = 0; j < 4096; j++) if (page[i+j] == 0) nz++;
if (nz == 4096) zeros_after++;
}
printf("zero pages after: %zu / %u\n", zeros_after, BLOCK/4096);
return zeros_after == zeros_before ? 0 : 1;
}
Results:
| mimalloc | result |
|---|---|
| v2.5.2, v2.5.3, v2 dev | reused: 1, zero pages before: 0/8192, zero pages after: 8175–8190/8192 → ~32 MiB of a live allocation silently zeroed (strace confirms the cross-thread mi_free emits madvise(..., MADV_FREE)) |
| v3 (main3 tip) | reused: 1, zero pages before: 8192/8192, zero pages after: 8192/8192 → no corruption (v3 no longer has _mi_segment_huge_page_reset) |
Why this matters beyond the uninitialized-memory case: pages written via DMA (device-to-host copies) go through the SMMU and never clear the lazyfree mark through the CPU MMU, so explicitly written data can be silently zeroed by kernel reclaim — that is the production scenario in pytorch/pytorch#200905 (PyTorch frees >16 MiB tensors cross-thread from DataLoader workers on aarch64, where mimalloc is the default CPU allocator).
Experimental observations (for reference, not a fix proposal):
- On the v3 line (
main3tip) this problem does not exist, and the cross-thread huge free path never emitsMADV_FREE— strace of the same scenario shows 0MADV_FREEvs. 802MADV_DONTNEEDcalls (all from the arena purge, honoring the default options), and reused memory comes back as clean zero pages. I could not find a commit on the v3 history that removed_mi_segment_huge_page_resetfor this reason — the function is gone as part of the v3 rewrite of the cross-thread free machinery (QSBR/abandon-based), so this may well be a byproduct of the rework rather than a deliberate fix; either way, the resulting behavior is what the v2 path arguably should match. - Dropping the
_mi_os_resetcall inside_mi_segment_huge_page_reseton the v2 dev tip mimics that behavior. In my testing (the repro above + a 2000×17MiB concurrent cross-thread free stress + a debug build with assertions) it fixes the default-config corruption, keeps theMADV_FREEbehavior underMIMALLOC_PURGE_DECOMMITS=0unchanged, and is slightly faster (one less syscall per cross-thread free). The trade-off is that physical pages are released later (when the owning thread processes the delayed free, instead of immediately at the cross-thread free) — the original comment on the reset suggests prompt availability was a design goal, so this is a decision for the maintainers.
Related pitfall worth knowing: routing the call through _mi_os_purge instead (the "obvious" one-line fix) also fixes the default-config corruption, but breaks debug builds: with MI_DEBUG>0 the decommit path does mprotect(PROT_NONE) (prim/unix/prim.c, the needs_recommit branch) while the block is still on the owner's delayed-free list, and the owner's subsequent metadata write (e.g. the debug memset in mi_free_block_local, or the free-list pointer store) faults:
#0 mi_free_block_local (free.c:40, debug memset)
#1 _mi_free_delayed_block (free.c:227)
#2 _mi_heap_delayed_free_partial
#3 _mi_malloc_generic
Reproduced on the v2 dev tip, default options, debug build only; release builds are unaffected. Whatever fix is chosen, this interaction (purge/decommit vs. in-flight delayed frees) is worth avoiding.
Environment
- aarch64, Linux 6.6, gcc 12.2.1
- mimalloc: v2.5.2
2406cc5, v2.5.39593e865, dev9b6a46d3, v3 main31d8fd85aa - build:
cmake -S . -B build -DMI_BUILD_SHARED=OFF -DMI_BUILD_OBJECT=OFF -DMI_BUILD_TESTS=OFF -DCMAKE_BUILD_TYPE=Release(+-DMI_EXTRA_CPPDEFS="MI_HUGE_PAGE_ABANDON=1"where noted)
- 主要言語
- C
- スター
- 13.6k
- フォーク
- 1.2k
- 平均マージ
- 12時間 44分
- マージ済み PR(30日)
- 28
環境構築
このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
microsoft/mimalloc のほかの issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 65/100
メンテナーはふだん 1 日以内に返信
-
Building mimalloc with `-Wundef` compiler flag logs many warnings on undefined macros対応中かも @vstinner が今日担当しました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 64/100
microsoft/mimalloc#1506 · コメント 5 件 ·
メンテナーはふだん 1 日以内に返信
-
stale
難易度 2/5 1〜3時間 初心者へのやさしさ 74/100
microsoft/mimalloc#1113 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 65/100
microsoft/mimalloc#886 · コメント 4 件 · リアクション 1 件 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 64/100
microsoft/mimalloc#368 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
microsoft/mimalloc の issue をすべて見る
似ている issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
メンテナーはふだん 1 日以内に返信
-
isolation2 lockmodes fails on cassert builds: extra AccessShareLock on partition root対応中かも @tuhaihe が今日担当しました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
apache/cloudberry#2356 ·
メンテナーはふだん 2 日以内に返信
-
windows-artifacts: the win-arm64 job uses a Visual Studio generator missing from windows-11-armオープン
難易度 1/5 1時間未満 初心者へのやさしさ 86/100
-
難易度 2/5 1〜3時間 初心者へのやさしさ 66/100
wasm-micro-runtime/wasm-micro-runtime#5217 ·
メンテナーはふだん 3 日以内に返信
-
Enhancement
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
HandBrake/HandBrake#8278 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信