rapidsai/cudf

[FEA] Add a method to check if row group stats are available

Open

#17,864 opened on Jan 29, 2025

View on GitHub
 (0 comments) (0 reactions) (0 assignees)C++ (6,000 stars) (735 forks)batch import
0 - Backloggood first issue

Description

Is your feature request related to a problem? Please describe. Currently, there isn't a method in predicate_pushdown.cpp (filter_row_groups) to check if row group stats aren't available. Hence, we can't ever set num_surviving_row_groups.after_stats_filter to std::nullopt if stats are unavailable.

Describe the solution you'd like We should figure out a relatively cheaper way to distinguish if row group stats aren't available if possible.

Describe alternatives you've considered We currently build the entire statsAST table in predicate_pushdown and try to filter row groups with it. In case no row groups are filtered, we can't distinguish if this was due to ineffective filter or missing stats.

Additional context Originally posted by @mhaseeb123 in https://github.com/rapidsai/cudf/pull/17594#discussion_r1932968249

Contributor guide

Tech stack
cpp
Domain
data
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
C++ programmingcuDF libraryrow group statistics
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40
Research direction
Investigate predicate pushdown.cpp, specifically the filter row groups function and the statsAST table. Look at the discussion in PR #17594 for context on distinguishing missing stats from ineffective filters. Consider implementing a lightweight check, such as tracking whether stats were available during parsing.
[FEA] Add a method to check if row group stats are available · rapidsai/cudf#17864 | Good First Issue