apache/pinot
GitHub で見るWhat's the root cause to wiki's recommendation: "Pause consumption when add a new column"?
Open
#12,652 opened on 2024年3月14日
documentationgood first issuetroubleshooting
Repository metrics
- Stars
- (4,937 stars)
- PR merge metrics
- (平均マージ 6d 7h) (30d で 186 merged PRs)
説明
In the ingestion transformation wiki page, it mentioned that
If a new column is added to table or schema configuration during ingestion, incorrect data may appear in the consuming segment.
To ensure accurate values are reloaded, do the following: Pause consumption (and wait for pause status success)
Is it only limited to table which has ingestion transform config? If not, it seems a breaking change since we need to pause consumption for every schema update.
Another question: In which scenario the data will incorrectly appears?