apache/pinot
Refactor to eliminate duplicate code in Spark 2.x and 3.x Batch Ingestion
开放
#14,271 创建于 2024年10月22日
cleanupgood first issueingestion
仓库指标
- 星标
- (4,937 个星标)
- PR 合并指标
- (PR 指标待抓取)
描述
Currently between spark 2.x and spark 3.x batch ingestion lot of code is duplicated other than SparkSegmentMetadataPushJobRunner class. This ticket is to refactor the code to eliminate the duplicate code without breaking the compatibility. Code changes required are to :
- Abstract common functionality into shared methods/classes.
- Separate version-specific details.
- Ensure the resulting codebase is cleaner and easier to maintain.