Unify some behavior in TableDataManager

(2 commenti) (2 reazioni) (0 assegnatari)Java (1234 fork)batch import

bugcleanupgood first issue

Metriche repository

In the TableDataManager:

void addSegment(File indexDir, IndexLoadingConfig indexLoadingConfig) is never used in production code (the only non-testing caller addOfflineSegment() is not wired up)
void addSegment(String segmentName, TableConfig tableConfig, IndexLoadingConfig indexLoadingConfig) can be simplified because TableConfig is included in the IndexLoadingConfig
void addOrReplaceSegment(String segmentName, IndexLoadingConfig indexLoadingConfig, SegmentZKMetadata zkMetadata, @Nullable SegmentMetadata localMetadata) should be broken into 2 methods: addSegment() and refreshSegment()
Loading committed segment into real-time table should share the same logic as loading segment into offline table. Currently the logic is diverged, which causes moving tier not happening for real-time table

Ideally we want to keep the following methods:

Direzione di ricerca: Analizza i metodi correnti di TableDataManager: identifica addSegment non utilizzato, semplifica addSegment con TableConfig ridondante, suddividi addOrReplaceSegment in addSegment e refreshSegment, e allinea la logica di caricamento dei segmenti committati per le tabelle in tempo reale e offline.
Tech stack: java
Dominio: backend
Tipo issue: Refactoring
Difficoltà: 3
Tempo stimato: 1-2 giorni
Stato attività: Attiva
Chiarezza: Chiara
Prerequisiti: GitJavaApache Pinot
Adatta ai principianti: 60