merge_insert analyze_plan: expose structured metrics
Maintainers usually reply within 3 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
Research direction
Start from MergeInsertJob::analyze_plan and its variants, and find where the per-node metrics that appear in the formatted plan string are collected. The issue asks for a structured result returned alongside that string, covering write mode, join index usage, fragments touched, rows and bytes written, and join versus write time. Scope is open (the proposal is an example list), so agree the struct's shape with maintainers before implementing. Done means a test asserts those fields on a merge_insert plan while the existing string output is unchanged.
Written by the indexing model from the issue text.
Description
MergeInsertJob::analyze_plan* returns only a formatted plan string with metrics inline per node. Callers who want to size merge_insert batches (lancedb/lancedb#4257) need to parse that text to get the numbers that actually drive cost.
Proposal: expose a structured result alongside the string, e.g.:
- write mode (
RewriteRows/RewriteColumns) - whether the join used a scalar index, and index segment count for the join key
- fragments touched
- rows read, rows/bytes written
- time in join vs write
LanceDB would then surface these in its merge_insert analyze_plan (currently a plain string).
- Dominant language
- Rust
- Stars
- 7.2k
- Forks
- 900
- Avg merge
- 4d 22h
- Merged PRs (30d)
- 366
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- Ships a Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from lance-format/lance
-
documentation
Difficulty 1/5 Under an hour Newbie friendliness 78/100
lance-format/lance#10225 ·
Maintainers usually reply within 3 days
-
feature performance
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
lance-format/lance#10137 ·
Maintainers usually reply within 3 days
-
After initialize_mem_wal, every FTS query logs "ERROR Index __lance_mem_wal has no fields"Possibly taken @lance-gatefixer claimed this 24 days ago. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
lance-format/lance#9655 ·
Maintainers usually reply within 3 days
-
Upgrade DataFusion for the unparser stack-overflow fixMay be free again A pull request for this issue was closed without being merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
lance-format/lance#8225 ·
Maintainers usually reply within 3 days
-
enhancement
Difficulty 1/5 Under an hour Newbie friendliness 84/100
lance-format/lance#6425 · 2 comments ·
Maintainers usually reply within 3 days
All issues in lance-format/lance
Similar issues
-
bug priority: P3 ready-for-agent
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
shep-pm/shep-kelpie#1332 ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
ramensoftware/windhawk#1156 · 2 comments ·
-
Difficulty 1/5 Under an hour Newbie friendliness 67/100
rust-bitcoin/rust-bitcoin#7089 · 1 comment · 1 reaction ·
Maintainers usually reply within 2 days
-
phase:3
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
pchrysostomou/ananke#183 ·
Maintainers usually reply within 1 day
-
area/tools bug session-analysis
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day