delta-io/delta
Ver no GitHub[Kernel][Data skipping] Add the STARTS_WITH expression and support data skipping for it
Open
#2.539 aberto em 18 de jan. de 2024
enhancementgood first issuekernel
Métricas do repositório
- Stars
- (8.807 stars)
- Métricas de merge de PR
- (Mesclagem média 7d 1h) (142 fundiu PRs em 30d)
Description
Feature request
Which Delta project/connector is this regarding?
- Spark
- Standalone
- Flink
- Kernel
- Other (fill in here)
Overview
Currently Kernel supports a limited set of expressions. We should 1) add the STARTS_WITH expression and 2) use file statistics to prune files based on the expression.
Motivation
Better file pruning.
Further details
This means we should
- add STARTS_WITH to the Kernel Predicate and support it in the kernel-defaults project
- Generate a data skipping filter according to the same rules we use in delta-spark
Willingness to contribute
The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?
- Yes. I can contribute this feature independently.
- Yes. I would be willing to contribute this feature with guidance from the Delta Lake community.
- No. I cannot contribute this feature at this time.