apache/pinot

Enhance the default column handling

オープン

#8,699 opened on 2022/05/13

 (0 件のコメント) (0 件のリアクション) (0 人の担当者)Java (1,234 件のフォーク)batch import
help wanted

Repository metrics

Stars
 (4,937 個のスター)
PR merge metrics
 (平均マージ 6d 7h) (30d で 186 merged PRs)

説明

When new columns are added to a pinot table, the server is able to automatically generate default columns for the new added columns. There are 2 types of default column that can be auto-generated:

  1. New added column without transform function: default column will be filled with default null value (always dictionary encoded)
  2. New added column with transform function (derived column): default column will be filled with values calculated based on the transform function

There are 2 things not handled right now:

  1. Generate default column raw forward index (not dictionary encoded)
  2. Support transform function on another new added column: we need to calculate the dependency between new added columns, and generate them in sequence

See BaseDefaultColumnHandler for details

コントリビューターガイド