apache/pinot
在 GitHub 查看Refactor to eliminate duplicate code in Spark 2.x and 3.x Batch Ingestion
Open
#14,271 建立於 2024年10月22日
cleanupgood first issueingestion
倉庫指標
- Star
- (4,937 star)
- PR 合併指標
- (平均合併 6天 7小時) (30 天內合併 186 個 PR)
描述
Currently between spark 2.x and spark 3.x batch ingestion lot of code is duplicated other than SparkSegmentMetadataPushJobRunner class. This ticket is to refactor the code to eliminate the duplicate code without breaking the compatibility. Code changes required are to :
- Abstract common functionality into shared methods/classes.
- Separate version-specific details.
- Ensure the resulting codebase is cleaner and easier to maintain.