delta-io/delta

[Feature Request] Consolidate CDF TahoeFileIndex code paths

开放

#1,127 创建于 2022年5月12日

 (3 条评论) (0 个反应) (1 位负责人)Scala (2,157 个派生)batch import
enhancementgood first issuenot-stale

仓库指标

星标
 (8,945 个星标)
PR 合并指标
 (平均合并 7天 1小时) (30 天内合并 142 个 PR)

描述

Feature request

Overview

This commit added 3 new TahoeFileIndex's for the Change Data Feed feature currently being developed for Delta Lake. These are: CdcAddFileIndex, TahoeChangeFileIndex, and TahoeRemoveFileIndex.

These three classes have lots of similar code paths + logic, and can be consolidated to a simpler abstract base class + children to reduce duplicate code.

Feel free to give them better class names, too.

Motivation

Clean up the code. Less code duplication. Better file and class names. Easier to maintain and debug in the future.

For more info on the Change Data Feed project for Delta Lake, see #1105.

贡献者指南