delta-io/delta
Vedi su GitHub[Kernel][Data skipping] Add the STARTS_WITH expression and support data skipping for it
Open
#2539 aperta il 18 gen 2024
enhancementgood first issuekernel
Metriche repository
- Star
- (8807 star)
- Metriche merge PR
- (Merge medio 7g 1h) (142 PR mergiate in 30 g)
Descrizione
Feature request
Which Delta project/connector is this regarding?
- Spark
- Standalone
- Flink
- Kernel
- Other (fill in here)
Overview
Currently Kernel supports a limited set of expressions. We should 1) add the STARTS_WITH expression and 2) use file statistics to prune files based on the expression.
Motivation
Better file pruning.
Further details
This means we should
- add STARTS_WITH to the Kernel Predicate and support it in the kernel-defaults project
- Generate a data skipping filter according to the same rules we use in delta-spark
Willingness to contribute
The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?
- Yes. I can contribute this feature independently.
- Yes. I would be willing to contribute this feature with guidance from the Delta Lake community.
- No. I cannot contribute this feature at this time.