Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Collection-per-file mode: creating a collection can route another collection's page to the main file (lock-free read of a Dictionary)

Closed
  • Is something wrong?
#312 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

A pull request for this has already been merged.

  • #314 by @CaffeinatedCoder — merged

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
58/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
csharp
Domain
databases

Research direction

Start by reading StorageEngine.Memory.cs around GetCollectionNameBySlot, GetPageFile, and GetOrAssignCollectionSlot, then inspect _collectionSlotToName in src/BLite.Core/Storage/StorageEngine.cs. Reproduce the race with ConcurrentConsistencyTests under the described concurrent collection-creation load, especially around the fourth and eighth collection-slot assignments. Done means collection-page routing no longer falls back to the main file during concurrent reads or writes, while LoadCollectionSlots uses the same safe mapping and ordering.

Written by the indexing model from the issue text.

Description

bug package/core severity/medium triage

In collection-per-file mode (PageFileConfig.Server), creating a collection while other collections read or write their pages can send one of those pages to the main file. The operation then fails with:

InvalidDataException: Page 3254779953 is outside the page file (file holds 65 pages): the file is corrupt.

3254779953 is 0xC2000031: the collection-page marker, slot 2, local page 49. Nothing on disk is corrupt. Slot 2's file exists, but the slot lookup missed.

Mechanism (0450b97)

  • _collectionSlotToName is a plain Dictionary<int, string> (src/BLite.Core/Storage/StorageEngine.cs:79).
  • It is written under _collectionSlotLock when a collection gets its slot (GetOrAssignCollectionSlot, StorageEngine.Memory.cs:663-683).
  • It is read without that lock by GetCollectionNameBySlot (StorageEngine.Memory.cs:333-335). GetPageFile calls that for every read, write and free of a collection page.
  • Dictionary does not support a read that runs concurrently with a write. When the 4th and the 8th collection get their slots, the dictionary resizes (capacity 3 → 7 → 17). Resize assigns a new, still empty _buckets array before it re-links the entries, so a concurrent TryGetValue for an existing slot can return false.
  • On a miss, GetPageFile falls through to the main file and passes the whole encoded id as the physical page number (StorageEngine.Memory.cs:327-331).

A related ordering detail: GetOrAssignCollectionSlot publishes name → slot before slot → name (lines 679-680). A thread that finds the slot through _collectionNameToSlot can route a page before its name is in _collectionSlotToName.

Reads fail with the exception above. That is the only outcome observed so far. Writes use the same routing: WritePage on the main file would take its grow path (PageFile.EnsureCapacityCore) and try to extend the main file to the page's offset. With 16 KB pages that is tens of TB. The collection file would never get that page. I have not observed this.

Reproduction

The error appeared once while running ConcurrentConsistencyTests 60 times under load (DOTNET_PROCESSOR_COUNT=2, two full-suite copies running alongside). It happened in the workload of HighContention_ManySmallTransactions_ConsistentFinalState: 8 threads each create their own collection at the same moment and insert into it. It hit writer 2 in its second transaction, while the other collections were still being created. The race window is the few instructions of each resize, which happen only at the 4th and the 8th collection of an engine. So it is rare, but it hits exactly the workload that test runs on a loaded CI runner.

Fix considerations

  • Make _collectionSlotToName safe for lock-free readers: a ConcurrentDictionary<int, string>, or a string?[MaxCollections] array (64 entries) written with Volatile.Write. The array fits because slots are dense and bounded.
  • Publish slot → name before name → slot.
  • LoadCollectionSlots writes the same maps (at open, before any reader). It should use the same types.

Found while investigating #311 (the other failure of that test). Related: #288.

Dominant language
C#
Stars
66
Forks
6
Avg merge
1d 2h
Merged PRs (30d)
58

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from EntglDb/BLite

All issues in EntglDb/BLite

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.