Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

DynamicCollection cannot write a document larger than one page: no overflow on the schema-less write path

Open
  • Is something wrong?
#320 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
csharp
Domain
database

Research direction

Start by reading DynamicCollection and the existing overflow writer in DocumentCollection.InsertWithOverflow / AllocateOverflowPage, then review OverflowChainReader and DeleteSlot for the layout they already support. Add coverage to the DynamicCollection oracle for documents larger than one page, and run the migration round trip for a typed database containing a 40 KB document in both layouts. Done means schema-less writes, reads, deletes, and both migration layouts handle the large document.

Written by the indexing model from the issue text.

Description

enhancement package/core

DynamicCollection rejects a document larger than MaxDocumentSize (one page minus header, slot and reserve; about 16 KB with the default page size) with BLiteValidationException. A typed DocumentCollection stores such a document across overflow pages (InsertWithOverflow), and since #319 the schema-less API reads those. It still cannot write them, so:

  • BLiteMigration.ToMultiFileAsync / ToSingleFileAsync refuse a database whose typed collections hold a large document (#319 makes the refusal name the collection and _id; before, the BSON reader threw).
  • The LiteDB import of #297 writes to DynamicCollection; LiteDB documents can be far larger than a page.
  • A server or scripting client of the schema-less API has a size limit the typed API does not have.

Suggested change

Give DynamicCollection the same overflow write path as DocumentCollection: insert and relocating update write [TotalLength][FirstOverflowPage][first chunk] into the slot with SlotFlags.HasOverflow and the rest into PageType.Overflow pages; the read side (OverflowChainReader, #319) and the delete side (DeleteSlot frees the chain, #319) already handle that layout. Candidates for sharing: move DocumentCollection.InsertWithOverflow / AllocateOverflowPage next to OverflowChainReader so both collections use one writer. Then BLiteMigration can drop its size check and MaxDocumentSize becomes the size of a single document BLite stores (today int.MaxValue-ish, bounded by the page count).

Tests: the DynamicCollection oracle (#272) with document sizes past one page; migration round trip of a typed database with a 40 KB document, both layouts.

Related: #301 (read side, fixed by #319), #297, #272.

Dominant language
C#
Stars
66
Forks
6
Avg merge
1d 2h
Merged PRs (30d)
58

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from EntglDb/BLite

All issues in EntglDb/BLite

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.