Let physical-plan producers reuse a CASE's first-WHEN operand as its ELSE value

Offen
  • Stimmt etwas nicht?
#27,280 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

  • #27281 von @Weijun-H — ohne Merge geschlossen

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
42/100
Issue-Typ
Feature
Klarheit
Klar beschrieben
Aktivitätsstatus
Veraltet
Tech-Stack
rust
Bereich
backend-api-design, databases

Rechercherichtung

Start with CaseExpr and the physical expression evaluation path; the issue does not name specific files or tests, so first locate CaseExpr and its existing tests. Check how the first WHEN operand is evaluated and how an opt-in can preserve caller-specified evaluation order. Done means ELSE reuses that operand’s evaluated array for ELSE rows while retaining the original evaluation order; add coverage for reuse and ordering.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Is your feature request related to a problem or challenge?

CASE WHEN f(x) = 'a' THEN 'b' ELSE f(x) END evaluates f(x) for every row in the first WHEN, then again for the ELSE rows. For an expensive f the second call can dominate.

For plans that go through the logical optimizer, CommonSubexprEliminate can remove the duplicate by moving f(x) into a lower projection: with an immutable UDF f, EXPLAIN SELECT CASE WHEN f(value) = 1 THEN 100 ELSE f(value) END FROM t shows Projection: f(t.value) AS __common_expr_1 below the CASE (not extracted when f is volatile). An engine that builds physical plans directly never runs that rule. It also may not be able to use a lower projection: that evaluates f(x) before the columns preceding the CASE, so when those columns and f fail on different rows, a different error surfaces first. Spark-compatible engines need to keep the original order.

Describe the solution you'd like

An opt-in on CaseExpr: the caller states that ELSE is the same call as the operand evaluated first inside the first WHEN, and CaseExpr evaluates it once, at that position, and reuses the array for the ELSE rows. The caller supplies the evaluation-order rules, since which child a PhysicalExpr evaluates first is not part of the trait contract.

Describe alternatives you've considered
  • A lower ProjectionExec in the producer: changes which error surfaces first, as above.
  • Automatic detection inside CaseExpr: needs DataFusion to know evaluation order for every expression on the path. A built-in policy for BinaryExpr/NotExpr/CastExpr could be a follow-up.
Vorherrschende Sprache
Rust
Sterne
9.6k
Forks
2.6k
Ø Merge
3 T. 4 Std.
Gemergte PRs (30 T.)
345

Entwicklungsumgebung

In Codespaces öffnen

Startet den Dev-Container des Projekts im Browser, mit Ihrem eigenen GitHub-Konto.

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus apache/datafusion

Alle Issues in apache/datafusion

Ähnliche Issues

Weitere Issues zu Rust

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.