apache/pinot

What's the root cause to wiki's recommendation: "Pause consumption when add a new column"?

已关闭

#12,652 创建于 2024年3月14日

 (4 条评论) (0 个反应) (0 位负责人)Java (1,234 个派生)batch import
documentationgood first issuetroubleshooting

仓库指标

星标
 (4,937 个星标)
PR 合并指标
 (PR 指标待抓取)

描述

In the ingestion transformation wiki page, it mentioned that

If a new column is added to table or schema configuration during ingestion, incorrect data may appear in the consuming segment.

To ensure accurate values are reloaded, do the following: Pause consumption (and wait for pause status success)

Is it only limited to table which has ingestion transform config? If not, it seems a breaking change since we need to pause consumption for every schema update.

Another question: In which scenario the data will incorrectly appears?

贡献者指南