apache/pinot
Refactor to eliminate duplicate code in Spark 2.x and 3.x Batch Ingestion
オープン
#14,271 opened on 2024/10/22
cleanupgood first issueingestion
Repository metrics
- Stars
- (4,937 個のスター)
- PR merge metrics
- (PR metrics pending)
説明
Currently between spark 2.x and spark 3.x batch ingestion lot of code is duplicated other than SparkSegmentMetadataPushJobRunner class. This ticket is to refactor the code to eliminate the duplicate code without breaking the compatibility. Code changes required are to :
- Abstract common functionality into shared methods/classes.
- Separate version-specific details.
- Ensure the resulting codebase is cleaner and easier to maintain.