Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Typed collections: == null and OrderBy(nullable).Take(n) served from an index return no null-keyed documents (indexes skip null keys, planner reads the null bucket)

Open
  • Is something wrong?
#247 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

@CaffeinatedCoder is already working on this.

Since Oct 16, 2026.

  • #253 by @CaffeinatedCoder — open

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
csharp
Domain
backend, databases

Research direction

Start with CollectionSecondaryIndex.Insert and IndexOptimizer.OptimizeExpression, then compare their null handling with DynamicCollection and the BTreeQueryProvider ordering fast paths. Read the linked pull request #253 for the current direction before changing behavior. Done means indexed null comparisons, aggregates, and nullable ordering windows agree with scans and LINQ-to-Objects, while preserving the documented unique-index behavior.

Written by the indexing model from the issue text.

Description

area/indexing area/linq bug package/core severity/high

Found by a differential test that runs ~330 LINQ shapes over the same documents on a typed collection with an index on every queried property, on the same collection without indexes, and with LINQ-to-Objects as the reference (PR to follow). Ten rows disagree, all on the indexed path only: the plain collection and LINQ-to-Objects agree with each other.

Repro

// DiffDoc has string? Nick and int? MaybeCount, both with .HasIndex(...) in OnModelCreating.
await ctx.Docs.InsertBulkAsync([
    new DiffDoc { Name = "d1", Nick = null, MaybeCount = null },
    new DiffDoc { Name = "d2", Nick = "a",  MaybeCount = 5 },
    new DiffDoc { Name = "d3", Nick = "b",  MaybeCount = null }]);

ctx.Docs.AsQueryable().Where(x => x.Nick == null).ToList();        // expected [d1],     actual []
ctx.Docs.AsQueryable().Where(x => x.MaybeCount == null).ToList();  // expected [d1, d3], actual []
ctx.Docs.AsQueryable().Count(x => x.MaybeCount == null);           // expected 2,        actual 0
ctx.Docs.AsQueryable().Where(x => x.MaybeCount == null).First();   // throws "Sequence contains no elements"

// Same root cause through the ordering fast path:
ctx.Docs.AsQueryable().OrderBy(x => x.MaybeCount).Take(3).Select(x => x.MaybeCount).ToList();
// expected [null, null, 5] (LINQ orders null first), actual [5]: the null-keyed documents are absent, not misplaced

Every shape that routes the null comparison through the index is affected: AND/OR combinations, Contains on a list holding a null, the aggregate push-downs, and OrderBy(nullable).Skip/Take served from the index without a Where.

Expected: the documents whose property is null, as LINQ-to-Objects and the same query on a collection without the index return them.

Actual: an empty result; 0 for Count; InvalidOperationException for First; a window that silently lacks the null-keyed documents.

Mechanism (verified on 0450b97)

The planner and the indexer disagree about whether typed indexes have a null bucket.

  • CollectionSecondaryIndex.Insert skips a document whose key is null (src/BLite.Core/Indexing/CollectionSecondaryIndex.cs:118, "Skip null keys or failed traversal"); Update and Delete do the same. No typed index ever contains IndexKey.NullSentinel. README documents this ("typed collections do not index null keys", unique-index section, added with #176) and UniqueIndexTests.Typed_Unique_Index_Ignores_Null_Values pins it.
  • IndexOptimizer.OptimizeExpression plans x.Prop == null on an indexed property as an exact point lookup with MinValue = MaxValue = DBNull.Value and FilterCompleteness.Exact (src/BLite.Core/Query/IndexOptimizer.cs:293-304); ConvertToIndexKey maps DBNull to IndexKey.NullSentinel (CollectionSecondaryIndex.cs:776); a null element of a Contains list is normalised to DBNull the same way (IndexOptimizer.cs:894). The lookup scans an empty bucket and, because the plan is exact, no post-filter runs.
  • BTreeQueryProvider serves OrderBy(indexedField).Skip(s).Take(n) by reading skip + take entries straight from the index (src/BLite.Core/Query/BTreeQueryProvider.cs:290-311), so null-keyed documents can never appear in the window; the replay variant (:315-325) has the same gap.
  • DynamicCollection does write the bucket (DynamicCollection.cs:2720, BsonType.Null => IndexKey.NullSentinel), so the same query on a schema-less collection is correct.

Affected versions: the null-skip at insert dates from fd5f9e8 (2026-03-06); the planner has answered == null from the null bucket since c2ba69a (#34, first in v4.1.3); the ordering fast path since f83e0d5 (v3.7.0). Reproduced on v5.2.3 / 0450b97.

Two ways out (your call)

  1. Typed indexes write NullSentinel, as DynamicCollection does. Side effects to decide: a typed unique index would then reject a second null (the opposite of what #176 documented); existing indexes need a rebuild to pick up null-keyed documents already stored; CollectionSecondaryIndex.Range(null, max) uses IndexKey.MinKey as the open lower bound (CollectionSecondaryIndex.cs:448) and would include the null bucket in x.Prop < v / <= v, so the typed path needs DynamicCollection's NullSentinelNext (DynamicCollection.cs:1065). Ordering then comes out right in both directions, since NullSentinel sorts before every encoded value.
  2. The optimizer declines. No index plan for null equality (or a Contains list holding null) on typed collections, falling back to the BSON predicate, which handles BsonType.Null already (CreateNullCheckPredicate); and TryOptimizeOrderBy declines when the key property is a reference type or Nullable<T>, or the window is merged with the null-keyed documents found by a scan.

Either way the index-served answer and the scan answer must agree; today they do not.

Dominant language
C#
Stars
66
Forks
6
Avg merge
1d 2h
Merged PRs (30d)
58

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from EntglDb/BLite

All issues in EntglDb/BLite

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.