meilisearch/MeiliSearch

Cannot take snapshots on Windows due to map_size and page_size values

开放

#6,051 创建于 2025年12月12日

 (4 条评论) (0 个反应) (0 位负责人)Rust (2,542 个派生)batch import
good first issue

仓库指标

星标
 (57,572 个星标)
PR 合并指标
 (平均合并 11天 20小时) (30 天内合并 41 个 PR)

描述

Describe the bugIndex scheduler fails to initialize during snapshot creation on Windows, leading to an internal error and the inability to create snapshots.

To ReproduceSteps to reproduce the behavior:

  1. Run Meilisearch on Windows (x86) with default options
  2. Attempt to create snapshot
  3. Snapshot fails with the following error:
Batch failed while doing: {"steps":[{"currentStep":"processing tasks","finished":0,"total":2},{"currentStep":"snapshot the index scheduler","finished":1,"total":6}],"percentage":8.333334}

Batch failed map size (1310824018739) must be a multiple of the system page size (4096)

Expected behaviorThe snapshot succeeds

ScreenshotsIf applicable, add screenshots to help explain your problem.

Meilisearch version:Reproduced on v1.27.0 and v1.29.0

Additional contextWe have done analysis of the code where the error originates (found via trace logging) and we believe there is a missing call to clamp_to_page_size here

We see that the DEFAULT_BUDGET on Windows is 6TiB:

const DEFAULT_BUDGET: usize = 6 * 1024 * 1024 * 1024 * 1024; // 6 TiB, 1 index

Naturally, this properly fits into the default page size of 4KiB (4096) for x86-64 machines:

6597069766656÷4096 = 1610612736

Given the following trace logs, we can further deduce what happens in the function to make things go awry:

memmap budget: 6597069766656B
index budget: 3277060046848B
1 index of 1310824018739B can be opened simultaneously.

With the default task_db_size of 20GiB here and a default base_map_size set to the max index size (2TiB) here we can trace the calculations:

let mut budget = budget / 2;  # 3298534883328
...

budget -= task_db_size;  # 3277060046848 (value from trace log)
...
let mut index_count = budget / base_map_size;   # 1.490234375 < 2 -> enter the conditional
...
let map_size = (budget * 2) / 5;  # 1310824018739.2 floored to the value from the trace

This gets returned as the map_size and instantly fails when passed off to heed: 1310824018739 mod 4096 = 819

If we clamped the map_size to the page size before returning, we believe the issue would be fixed.

贡献者指南