merge_insert: support explain_plan/analyze_plan on the scalar-index path
Maintainers usually reply within 3 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- rust
- Domain
- data-engineering, databases
Research direction
Start with the can_use_create_plan check in MergeInsertJob and the explain_plan and analyze_plan* methods that return NotSupported when it is false. Then trace the scalar-index merge_insert execution path to see how its indexed probe and fragment column patching are built. Done means explain and analyze work on that path and tests cover it. This is a change to the execution-plan design, so confirm the approach with maintainers first.
Written by the indexing model from the issue text.
Description
MergeInsertJob::explain_plan / analyze_plan* return NotSupported when can_use_create_plan is false, including when the join takes the scalar-index execution path. That path is the one where cost is hardest to predict (indexed probe plus column patching of every touched fragment), which is the motivating case in lancedb/lancedb#4257.
Proposal: migrate the scalar-index merge_insert path to a single create_plan execution plan so explain/analyze work unconditionally.
- Dominant language
- Rust
- Stars
- 7.2k
- Forks
- 900
- Avg merge
- 4d 22h
- Merged PRs (30d)
- 366
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- Ships a Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from lance-format/lance
-
documentation
Difficulty 1/5 Under an hour Newbie friendliness 78/100
lance-format/lance#10225 ·
Maintainers usually reply within 3 days
-
feature performance
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
lance-format/lance#10137 ·
Maintainers usually reply within 3 days
-
After initialize_mem_wal, every FTS query logs "ERROR Index __lance_mem_wal has no fields"Possibly taken @lance-gatefixer claimed this 24 days ago. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
lance-format/lance#9655 ·
Maintainers usually reply within 3 days
-
Upgrade DataFusion for the unparser stack-overflow fixMay be free again A pull request for this issue was closed without being merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
lance-format/lance#8225 ·
Maintainers usually reply within 3 days
-
enhancement
Difficulty 1/5 Under an hour Newbie friendliness 84/100
lance-format/lance#6425 · 2 comments ·
Maintainers usually reply within 3 days
All issues in lance-format/lance
Similar issues
-
bug priority: P3 ready-for-agent
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
shep-pm/shep-kelpie#1332 ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
ramensoftware/windhawk#1156 · 2 comments ·
-
units: No `#` in rustdoc headingPossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 1/5 Under an hour Newbie friendliness 67/100
rust-bitcoin/rust-bitcoin#7089 · 1 comment · 1 reaction ·
Maintainers usually reply within 2 days
-
phase:3
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
pchrysostomou/ananke#183 ·
Maintainers usually reply within 1 day
-
area/tools bug session-analysis
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day