CometIcebergNativeScanExec: propagate outputOrdering from originalPlan instead of hardcoding Nil
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 72/100
- Issue type
- Feature
- Clarity
- Clearly specified
- Activity status
- Quiet
- Tech stack
- scala
- Domain
- performance
Research direction
Start in CometIcebergNativeScanExec.scala and inspect how originalPlan wraps the Iceberg BatchScanExec. Verify how BatchScanExec.outputOrdering is exposed, then update the scan's ordering behavior so it preserves that value when originalPlan exists and remains Nil otherwise. Done means native Iceberg scans can report the original ordering without an unnecessary CometSortExec.
Written by the indexing model from the issue text.
Description
What is the problem the feature request solves?
CometIcebergNativeScanExec.scala hardcodes outputOrdering to Nil:
override lazy val outputOrdering: Seq[SortOrder] = Nil
Ref https://github.com/apache/iceberg/issues/16430
We can update it to :
override lazy val outputOrdering: Seq[SortOrder] =
if (originalPlan != null) originalPlan.outputOrdering else Nil
originalPlan is the wrapped BatchScanExec. Once Iceberg implements SupportsReportOrdering, BatchScanExec.outputOrdering returns the correct sort order and Comet inherits it
This wil eliminate CometSortExec above Iceberg native scans for sort-merge joins .
Describe the potential solution
No response
Additional context
No response
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 377
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 244
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/datafusion-comet
-
area:ci enhancement requires-triage
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
apache/datafusion-comet#6078 ·
-
area:ci bug priority:low
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
apache/datafusion-comet#6060 ·
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
apache/datafusion-comet#6028 ·
-
enhancement
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
apache/datafusion-comet#5861 ·
-
requires-triage
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
apache/datafusion-comet#5661 ·
All issues in apache/datafusion-comet
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
ergoplatform/ergodocs#614 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
[VL] madvise(WILLNEED) call fails in MmapFileStream because of wrong calculation of fetching length Openbug triage
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
chipsalliance/rocket-chip#3831 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100