Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

BLQL filters and sorts cannot reach a nested field, though a dynamic index can be built on one

Open
  • Is something wrong?
#264 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
1-2 days
Newbie friendliness
57/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
csharp
Domain
databases

Research direction

Start with src/BLite.Core/Query/Blql/BlqlFilter.cs and BlqlSort.cs, where BLQL reads fields, then compare those lookups with DynamicCollection.TryGetNestedValue in src/BLite.Core/DynamicCollection.cs. Use the oracle cases in tests/BLite.Tests/Oracles/DynamicDifferentialTests.cs and the repro here to check filters and sorting both with and without an index. Done means dotted paths return consistent, expected results across those cases.

Written by the indexing model from the issue text.

Description

area/linq bug package/core severity/low

On a DynamicCollection, a BLQL filter or sort on a dotted path ("meta.kind") treats the field as missing: $eq, $gt, $in match nothing, $ne and $nin match every document, and the in-memory sort keeps the scan order. CreateIndexAsync("meta.kind") and QueryIndexAsync on the same path work, and with that index BLQL scans it and then drops every document in its residual filter.

Found by the DynamicCollection oracle tests (tests/BLite.Tests/Oracles/DynamicDifferentialTests.cs, #272), which carry the affected rows (48: indexed/blql and plain/blql against LINQ-to-Objects and the typed collection, e.g. kind == Blue, kind in list, orderby kind) as a known candidate until this is fixed.

Repro

var col = engine.GetOrCreateCollection("p");          // and once more after: await col.CreateIndexAsync("meta.kind");
foreach (var (n, k) in new[] { ("a", 2), ("b", 0), ("c", 2) })
    await col.InsertAsync(col.CreateDocument(["n", "meta", "kind"],
        b => b.AddString("n", n).AddDocument("meta", m => m.AddInt32("kind", k))));

col.Query(BlqlFilter.Eq("meta.kind", 2)).ToList();     // expected [a, c],    actual [] (indexed: [])
col.Query("{\"meta.kind\": 2}").ToList();              // expected [a, c],    actual []
col.Query(BlqlFilter.Ne("meta.kind", 2)).ToList();     // expected [b],       actual [a, b, c]
col.Query().OrderBy("meta.kind").ToList();             // expected [b, a, c], actual [a, b, c] (indexed: [b, c, a])
await col.QueryIndexAsync("idx_meta.kind", 2, 2).ToListAsync();   // [c, a], correct

Expected: a dotted path names the nested field in BLQL as it does for indexes: "dot-notation strings ("address.city") for schema-less collections" (README.md:65) and "Nested path index (dot-notation) — indexes a field inside an embedded document" (README.md:861). BLQL is "MQL-inspired" (README.md:97), and in MQL {"meta.kind": 2} is a path. The README's BLQL section (README.md:991-1130) does not say either way.

Actual: see above. With the index the filter is worse than useless: the scan finds the right documents and the residual filter removes them all. Only the index-served sort is right.

Mechanism (verified on 0450b97)

  • Every BLQL operator reads its field with BsonDocument.TryGetValue (CompareFilter.Matches, src/BLite.Core/Query/Blql/BlqlFilter.cs:315; InFilter, :408; the same for $exists, $type, $regex, the string and the array operators), and so does BlqlSort.ToComparison (src/BLite.Core/Query/Blql/BlqlSort.cs:53-54).
  • BsonDocument.TryGetValue (src/BLite.Bson/BsonDocument.cs:230) looks up a top-level name only. The dotted path is resolved only by DynamicCollection.TryGetNestedValue (src/BLite.Core/DynamicCollection.cs:2759), which the index paths use.
  • BlqlQuery.GetScanSource (BlqlQuery.cs:302) finds the nested index through FindBTreeIndexForField (DynamicCollection.cs:1091), which is why the single-key sort fast path (BlqlQuery.cs:241-258) orders by it correctly while the filter does not match.

Affected versions: BLQL has read fields this way since it was added (a86d458, 2026-02-24, v1.9.0). Reproduced on v5.2.3 / 0450b97.

Fix considerations

  • One path lookup for BLQL (TryGetNestedValue made shared, or a BsonDocument.TryGetPath) used by every filter and by BlqlSort. MQL forbids dots in field names, so a dotted name can always mean a path; whether BLQL should follow that is your call, but the indexed and unindexed answers should agree either way.
  • BlqlProjection takes field names too; whether Include("meta.kind") should project the nested field was not checked.

Related: #254 (open, BLQL's index-served sort and range scan, same GetScanSource), and the other BLQL rows of the same tests: #265, #266, #267, #268.

Dominant language
C#
Stars
66
Forks
6
Avg merge
1d 2h
Merged PRs (30d)
58

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from EntglDb/BLite

All issues in EntglDb/BLite

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.