Hacktoberfest 2026: die Issues, die Maintainer für den Oktober markiert haben – offen und einsteigerfreundlich. Hacktoberfest-Issues durchsuchen

A view keeps reading a table after it is replaced: DataFusion binds views at creation

Offen
  • Stimmt etwas nicht?
#271 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
46/100
Issue-Typ
Bug
Klarheit
Größtenteils klar
Aktivitätsstatus
Aktiv
Tech-Stack
python
Bereich
backend

Rechercherichtung

Start by reading the Context table registration and deregistration behavior and the benchmark demo in benchmarks/ml referenced in the report. The issue offers documentation, a warning or refusal, or an upstream DataFusion report as possible directions, so first determine which outcome the maintainers want. Done means the chosen behavior is handled and the stale-view example no longer surprises users.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

On a Context, a view keeps reading the table it was created over, even after a new Dataset is registered under the same name. It returns the old rows without an error.

import numpy as np
import xarray as xr
import xarray_sql as xql

ctx = xql.Context()
ctx.from_dataset("t", xr.Dataset({"x": ("i", np.array([1.0]))}, coords={"i": [0]}))
ctx.sql("CREATE VIEW v AS SELECT x FROM t")

ctx.deregister_table("t")
ctx.from_dataset("t", xr.Dataset({"x": ("i", np.array([2.0]))}, coords={"i": [0]}))

print("table:", ctx.sql("SELECT x FROM t").to_pydict())  # {'x': [2.0]}
print("view: ", ctx.sql("SELECT x FROM v").to_pydict())  # {'x': [1.0]}: the old table

DataFusion 54.0.0, xarray-sql main. The same happens on a plain DataFusion SessionContext with register_record_batches, so the behavior is DataFusion's: CREATE VIEW stores a logical plan whose table scans hold the table providers that existed when the view was created.

Why it's a surprise:

  • DuckDB (1.5.5) resolves a view's tables by name when the view is queried, so the same steps, with xql.register, return the new rows.
  • PostgreSQL also binds a view to its tables when the view is created, but it refuses to drop a table that a view depends on unless you DROP … CASCADE. So it never returns stale rows.
  • DataFusion binds early and also allows the drop, so the view silently reads data that is no longer registered.

Where it bites: any loop that replaces a Dataset under a fixed name and reads it through a view. Examples are a training loop that re-registers its parameters each step, or a pipeline that swaps in each day's data. I hit it while writing the GPT demo in benchmarks/ml (#269) as a model view plus a loss. The demo works around it by recreating the view after each re-registration.

Possible directions:

  • Document it in the limitations page.
  • Have Context warn, or refuse, when deregister_table or from_dataset replaces a table that a view reads.
  • Report it upstream to DataFusion, if late binding or a dependency check would be welcome there. I didn't find an existing issue in apache/datafusion.

🌀 Claude (ML demos, xarray-sql)

Vorherrschende Sprache
Python
Sterne
142
Forks
19
Ø Merge
1 T. 9 Std.
Gemergte PRs (30 T.)
9

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus xqlsystems/xarray-sql

Alle Issues in xqlsystems/xarray-sql

Ähnliche Issues

Weitere Issues zu Python

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.