REGRESSION: SanityCheckPlan error for GROUP BY ... HAVING on a CASE key with parquet filter pushdown

オープン
  • 何か問題がありますか？
#27,287 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

@cetra3 がすでに取り組んでいます。

2026年11月10日 から。

  • #27288 @cetra3 による — オープン

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
35/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
停滞
技術スタック
rust, sql
領域
backend, databases

調査の方向性

Start with the regression identified by git bisect at 99e723dd47 and trace how EnsureRequirements handles orderings and constants across the ProjectionExec inserted by common subexpression elimination. Reproduce with the SQL in the issue, using parquet filter pushdown; the fix is done when planning succeeds and the query returns e = 'y', count = 1. A linked open pull request already addresses this issue.

索引モデルが issue の本文から書いたものです。

説明

Posted by Claude (an AI assistant) on behalf of cetra3.

Describe the bug

With datafusion.execution.parquet.pushdown_filters = true, a query that groups by a CASE expression and pins the group key with HAVING fails to plan with SanityCheckPlan. The CASE repeats a subexpression, so common subexpression elimination adds a ProjectionExec above the scan.

The HAVING filter is pushed into the parquet predicate, which makes the group key constant. The partial AggregateExec relies on that and requires its input ordered by the key. But the ProjectionExec between the scan and the aggregate rewrites a = 'x' to __common_expr_1, and the constant does not survive that projection, so the final plan check rejects the plan.

This plans and returns the right row on 55.2.0.

To Reproduce

datafusion-cli at main (c8265b6872, 2026-11-10):

SET datafusion.execution.parquet.pushdown_filters = true;
COPY (SELECT * FROM (VALUES ('x', 'y'), ('x', 'z'), ('w', 'y')) AS v(a, b)) TO 't.parquet';
CREATE EXTERNAL TABLE t STORED AS PARQUET LOCATION 't.parquet';
SELECT CASE WHEN a = 'x' AND b = 'y' THEN b WHEN a = 'x' THEN 'z' END AS e, count(*)
FROM t
GROUP BY e
HAVING e = 'y';

SanityCheckPlan: the plan produced by physical optimizer rule 'EnsureRequirements' does not satisfy the order and distribution requirements
caused by
Error during planning: Plan: ["AggregateExec: mode=Partial, gby=[CASE WHEN __common_expr_1@0 AND b@1 = y THEN b@1 WHEN __common_expr_1@0 THEN z END as CASE WHEN t.a = Utf8(\"x\") AND t.b = Utf8(\"y\") THEN t.b WHEN t.a = Utf8(\"x\") THEN Utf8(\"z\") END], aggr=[count(Int64(1))]", "  ProjectionExec: expr=[a@0 = x as __common_expr_1, b@1 as b]", "    RepartitionExec: partitioning=RoundRobinBatch(32), input_partitions=1", "      DataSourceExec: file_groups={1 group: [[.../t.parquet]]}, projection=[a, b], file_type=parquet, predicate=CASE WHEN a@0 = x AND b@1 = y THEN b@1 WHEN a@0 = x THEN z END = y"] does not satisfy order requirements: [CASE WHEN __common_expr_1@0 AND b@1 = y THEN b@1 WHEN __common_expr_1@0 THEN z END ASC]. Child-0 order: []
Expected behavior

One row, e = 'y', count = 1, as 55.2.0 returns.

Additional context
  • git bisect between 55.2.0's merge base with main and main points at 99e723dd47, "fix: retain only valid orderings after order-preserving repartition" (#26747). Its parent ce0e8293f9 returns the right row.
  • It also fails with target_partitions = 1.
  • Each of these makes it plan and return the right row:
    • pushdown_filters = false.
    • An in-memory table (CREATE TABLE t AS VALUES ...) in place of parquet.
    • A CASE with no repeated subexpression, so CSE adds no projection, for example CASE WHEN a = 'x' THEN b ELSE 'z' END.
    • WHERE <the same CASE> = 'y' ... ORDER BY e with no GROUP BY.
  • This looks related to #26624 (fixed by #27123), where a constant was lost when equivalence properties were projected through a sort-merge join. That fix is in main, and this query still fails. The open PR #27191 does not fix it either. I built its head (434a1272bb) and got the same error.
主要言語
Rust
スター
9.6k
フォーク
2.6k
平均マージ
3日 4時間
マージ済み PR（30日）
345

環境構築

Codespaces で開く

このプロジェクトの開発コンテナを、あなたの GitHub アカウントでブラウザ上に起動します。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

apache/datafusion のほかの issue

apache/datafusion の issue をすべて見る

似ている issue

Rust の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。