apache/pinot

Pinot can't skip creating derived column when configured in transform functions

Open

#7.627 geöffnet am 25. Okt. 2021

Auf GitHub ansehen
 (0 Kommentare) (0 Reaktionen) (0 zugewiesene Personen)Java (1.234 Forks)batch import
help wanted

Repository-Metriken

Stars
 (4.937 Stars)
PR-Merge-Metriken
 (Durchschn. Merge 6T 7h) (186 gemergte PRs in 30 T)

Beschreibung

When reloading segments, pinot will skip many segments when derived column not configured in schema or segments metadata , which will cause some segments not synced latest new columns

We got the following error message when query from table: image

Code need remove:

for (String argument : arguments) {
  ColumnMetadata columnMetadata = _segmentMetadata.getColumnMetadataFor(argument);
  if (columnMetadata == null) {
    LOGGER.warn("Skip creating derived column: {} because argument: {} does not exist in the segment", column,
        argument);
    return false;
  }
  argumentsMetadata.add(columnMetadata);
}

Contributor Guide