apache/pinot

Unify some behavior in TableDataManager

クローズ

#9,752 opened on 2022/11/07

 (2 件のコメント) (2 件のリアクション) (0 人の担当者)Java (1,234 件のフォーク)batch import
bugcleanupgood first issue

Repository metrics

Stars
 (4,937 個のスター)
PR merge metrics
 (PR metrics pending)

説明

In the TableDataManager:

  • void addSegment(File indexDir, IndexLoadingConfig indexLoadingConfig) is never used in production code (the only non-testing caller addOfflineSegment() is not wired up)
  • void addSegment(String segmentName, TableConfig tableConfig, IndexLoadingConfig indexLoadingConfig) can be simplified because TableConfig is included in the IndexLoadingConfig
  • void addOrReplaceSegment(String segmentName, IndexLoadingConfig indexLoadingConfig, SegmentZKMetadata zkMetadata, @Nullable SegmentMetadata localMetadata) should be broken into 2 methods: addSegment() and refreshSegment()
  • Loading committed segment into real-time table should share the same logic as loading segment into offline table. Currently the logic is diverged, which causes moving tier not happening for real-time table

Ideally we want to keep the following methods:

  • addSegment
  • addConsumingSegment (real-time only)
  • refreshSegment
  • reloadSegment
  • removeSegment

コントリビューターガイド