bug: all-empty index keeps lowest dataset_version segment instead of earliest created_at
Maintainers usually reply within 2 days
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 76/100
Research direction
Start in rust/lance-table/src/transaction/index_maintenance.rs and read retain_relevant_indices, focusing on how it selects a segment when all segments have empty live coverage. Check the relevant tests for index maintenance and add or update cases for earliest created_at, missing timestamps, and ties by lowest dataset_version; done means selection follows those rules.
Written by the indexing model from the issue text.
Description
When every segment of an index has empty live coverage, retain_relevant_indices (rust/lance-table/src/transaction/index_maintenance.rs) keeps the segment with the lowest dataset_version as the index definition.
dataset_version is a watermark, not a creation time. Merged segments take the minimum version of their sources, and remaps raise it. So "lowest version" is not "first created". It can pick a newer segment, which loses the earliest created_at; callers use that as an approximation of when the index was created.
Expected (per #10090): keep the segment with the earliest created_at. Segments without created_at sort after those with one, and ties go to the lowest dataset_version.
- Dominant language
- Rust
- Stars
- 7.2k
- Forks
- 882
- Avg merge
- 3d 15h
- Merged PRs (30d)
- 223
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- Ships a Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from lance-format/lance
-
feature performance
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
lance-format/lance#10137 ·
Maintainers usually reply within 2 days
-
bug: zero max_rows_per_file panics, zero V1 max_rows_per_group writes nothingPossibly taken @LuciferYang claimed this 1 day ago. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
lance-format/lance#10130 ·
Maintainers usually reply within 2 days
-
perf: miniblock binary decode copies the chunk's value bytes instead of slicing themPossibly taken @LuciferYang claimed this 6 days ago. Openperformance
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
lance-format/lance#9928 ·
Maintainers usually reply within 2 days
-
After initialize_mem_wal, every FTS query logs "ERROR Index __lance_mem_wal has no fields"Possibly taken @lance-gatefixer claimed this 17 days ago. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
lance-format/lance#9655 ·
Maintainers usually reply within 2 days
-
bug: main does not compile, MinHash calls build_prefilter without the metrics argumentPossibly taken @lance-gatefixer claimed this 26 days ago. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
lance-format/lance#9506 · 1 comment ·
Maintainers usually reply within 2 days
All issues in lance-format/lance
Similar issues
-
area:io needs-info needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 63/100
bherbruck/solvecraft#3216 · 1 comment ·
-
bug triage/needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
praxis-proxy/grid#412 ·
Maintainers usually reply within 1 day
-
priority: low
Difficulty 2/5 1-3 hours Newbie friendliness 63/100
storytold/photocraft#7824 · 1 comment ·
Maintainers usually reply within 1 day
-
area:ui chore could help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day