delta-io/delta
[Feature Request] Support additional generation expressions for automatic data skipping
オープン
#1,442 opened on 2022/10/18
enhancementgood first issuenot-stale
Repository metrics
- Stars
- (8,945 個のスター)
- PR merge metrics
- (平均マージ 7d 1h) (30d で 142 merged PRs)
説明
Overview
For partition columns that are generated columns, we are able to automatically generate partition filters when we see a data filter on its generating columns. Right now we automatically generate these for a small subset of possible generation expressions (defined here.)
Details
We can add support for additional expressions. Here are the supported generation expressions in Delta.
A few specific expressions that would make sense to add include:
- Ceil: https://github.com/delta-io/delta/issues/1443
- Floor: https://github.com/delta-io/delta/issues/1444
- TruncDate: https://github.com/delta-io/delta/issues/1446
- TruncTimestamp: https://github.com/delta-io/delta/issues/1445
These are just a few, feel free to comment or create a new sub-issue for any additional expressions you think would be beneficial.