delta-io/delta

[Feature Request] Skip creating FilteredColumnarBatch in ActiveAddFilesIterator if there is no active AddFiles in the batch

Open

#4,941 建立於 2025年7月17日

在 GitHub 查看
 (1 留言) (0 反應) (0 負責人)Scala (8,807 star) (2,100 fork)batch import
enhancementgood first issue

描述

Feature request

Which Delta project/connector is this regarding?

  • Spark
  • Standalone
  • Flink
  • Kernel
  • Other (fill in here)

Overview

Today, a FilteredColumnarBatch with no Active AddFiles will be returned as the result of Scan::getScanFiles (also PaginatedScan::getScanFiles() ). We want to skip creating FilteredColumnarBatch in ActiveAddFilesIterator if there is no active AddFiles in the batch.

Motivation

Further details

Willingness to contribute

The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?

  • Yes. I can contribute this feature independently.
  • Yes. I would be willing to contribute this feature with guidance from the Delta Lake community.
  • No. I cannot contribute this feature at this time.

貢獻者指南

技術棧
rust
領域
databasebackend
議題類型
feature
難度面向新貢獻者的預計實作難度，1 表示很小改動，5 表示專家級工作。
2
預計時間有經驗貢獻者完成調查、實作、測試並準備 pull request 的粗略時間範圍。
under 1 hour
活動狀態議題目前的可參與程度：新鮮、活躍、陳舊、阻塞或等待維護者輸入。
fresh
清晰度議題是否清楚說明預期改動、驗收標準和下一步。
mostly clear
前置要求
familiarity with Delta Kernel codebaseunderstanding of Scan API
新手友善度1-100 的估計分數，表示該議題對首次貢獻者的友善程度。
40
研究方向
Investigate the ActiveAddFilesIterator class in the Kernel module. The goal is to add a check before creating a FilteredColumnarBatch: if there are no active AddFiles in the current batch, skip creating the FilteredColumnarBatch and continue to the next batch. Look at how FilteredColumnarBatch is created in ActiveAddFilesIterator and modify the logic accordingly. No linked PR yet.