delta-io/delta
Voir sur GitHub[Kernel][Data skipping] Add the STARTS_WITH expression and support data skipping for it
Open
#2 539 ouverte le 18 janv. 2024
enhancementgood first issuekernel
Métriques du dépôt
- Stars
- (8 807 stars)
- Métriques de merge PR
- (Merge moyen 7j 1h) (142 PRs mergées en 30 j)
Description
Feature request
Which Delta project/connector is this regarding?
- Spark
- Standalone
- Flink
- Kernel
- Other (fill in here)
Overview
Currently Kernel supports a limited set of expressions. We should 1) add the STARTS_WITH expression and 2) use file statistics to prune files based on the expression.
Motivation
Better file pruning.
Further details
This means we should
- add STARTS_WITH to the Kernel Predicate and support it in the kernel-defaults project
- Generate a data skipping filter according to the same rules we use in delta-spark
Willingness to contribute
The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?
- Yes. I can contribute this feature independently.
- Yes. I would be willing to contribute this feature with guidance from the Delta Lake community.
- No. I cannot contribute this feature at this time.