LEAD/LAG with offset 0 and IGNORE NULLS returns the default for a NULL row, unlike the form without IGNORE NULLS

Offen
  • Stimmt etwas nicht?
#27,284 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

@0xamlab arbeitet bereits daran.

Seit 10.11.2026.

  • #27313 von @0xamlab — offen

Bewertung

Schwierigkeit
2/5
Geschätzter Aufwand
1-3 Stunden
Anfängerfreundlichkeit
35/100
Issue-Typ
Bug
Klarheit
Klar beschrieben
Aktivitätsstatus
Veraltet
Tech-Stack
rust, sql
Bereich
backend-api-design, databases

Rechercherichtung

Read datafusion/functions-window/src/lead_lag.rs, focusing on WindowShiftEvaluator::evaluate and the offset-zero path in evaluate_all_with_ignore_null. Start by checking the existing window function tests for LEAD and LAG with IGNORE NULLS; add coverage for offset 0 on a NULL current row. Done means both functions preserve NULL instead of returning the default, with the tests passing.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Describe the bug

lead(v, 0, d) and lag(v, 0, d) return the current row's own value. Without IGNORE NULLS a NULL row stays NULL. With IGNORE NULLS the default d replaces it.

To Reproduce
SELECT id,
       lead(v, 0, -7) OVER (ORDER BY id) AS lead_plain,
       lead(v, 0, -7) IGNORE NULLS OVER (ORDER BY id) AS lead_ignore_nulls,
       lag(v, 0, -7) OVER (ORDER BY id) AS lag_plain,
       lag(v, 0, -7) IGNORE NULLS OVER (ORDER BY id) AS lag_ignore_nulls
FROM (VALUES (1, 10), (2, NULL), (3, 30)) AS t(id, v)
ORDER BY id;

On 55.2.0 (datafusion-python 55.0.0):

| id | lead_plain | lead_ignore_nulls | lag_plain | lag_ignore_nulls |
| 1  | 10         | 10                | 10        | 10               |
| 2  | NULL       | -7                | NULL      | -7               |
| 3  | 30         | 30                | 30        | 30               |
Expected behavior

Row 2 should be NULL in all four columns. Offset 0 refers to the current row itself, so there is no row to skip and nothing out of range that would call for the default. Apache Spark 4.0 returns the row's own value (NULL) for lead(v, 0, -7) IGNORE NULLS, the same as without IGNORE NULLS.

Additional context

In datafusion/functions-window/src/lead_lag.rs the two code paths disagree on this input as written:

  • evaluate_all_with_ignore_null starts with if offset == 0 { return Ok(Arc::clone(array)) }, so a NULL stays NULL.
  • WindowShiftEvaluator::evaluate ends with "Set the default value if ... ignore nulls mode and current value is null", which turns a NULL current row into the default when the offset is 0.

Offsets of 1 or more are not affected by this; the offset 0 case is only reached by an explicit 0.

Separately, #25471 (fixed on main by #25472) covers LEAD with an offset of 2 or more returning wrong rows across NULL gaps. The published 55.2.0 crate still has the old code for that.

Vorherrschende Sprache
Rust
Sterne
9.6k
Forks
2.6k
Ø Merge
3 T. 4 Std.
Gemergte PRs (30 T.)
345

Entwicklungsumgebung

In Codespaces öffnen

Startet den Dev-Container des Projekts im Browser, mit Ihrem eigenen GitHub-Konto.

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus apache/datafusion

Alle Issues in apache/datafusion

Ähnliche Issues

Weitere Issues zu Rust

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.