Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

A view keeps reading a table after it is replaced: DataFusion binds views at creation

Aperta
  • Qualcosa non va?
#271 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
46/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
backend

Direzione di ricerca

Start by reading the Context table registration and deregistration behavior and the benchmark demo in benchmarks/ml referenced in the report. The issue offers documentation, a warning or refusal, or an upstream DataFusion report as possible directions, so first determine which outcome the maintainers want. Done means the chosen behavior is handled and the stale-view example no longer surprises users.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

On a Context, a view keeps reading the table it was created over, even after a new Dataset is registered under the same name. It returns the old rows without an error.

import numpy as np
import xarray as xr
import xarray_sql as xql

ctx = xql.Context()
ctx.from_dataset("t", xr.Dataset({"x": ("i", np.array([1.0]))}, coords={"i": [0]}))
ctx.sql("CREATE VIEW v AS SELECT x FROM t")

ctx.deregister_table("t")
ctx.from_dataset("t", xr.Dataset({"x": ("i", np.array([2.0]))}, coords={"i": [0]}))

print("table:", ctx.sql("SELECT x FROM t").to_pydict())  # {'x': [2.0]}
print("view: ", ctx.sql("SELECT x FROM v").to_pydict())  # {'x': [1.0]}: the old table

DataFusion 54.0.0, xarray-sql main. The same happens on a plain DataFusion SessionContext with register_record_batches, so the behavior is DataFusion's: CREATE VIEW stores a logical plan whose table scans hold the table providers that existed when the view was created.

Why it's a surprise:

  • DuckDB (1.5.5) resolves a view's tables by name when the view is queried, so the same steps, with xql.register, return the new rows.
  • PostgreSQL also binds a view to its tables when the view is created, but it refuses to drop a table that a view depends on unless you DROP … CASCADE. So it never returns stale rows.
  • DataFusion binds early and also allows the drop, so the view silently reads data that is no longer registered.

Where it bites: any loop that replaces a Dataset under a fixed name and reads it through a view. Examples are a training loop that re-registers its parameters each step, or a pipeline that swaps in each day's data. I hit it while writing the GPT demo in benchmarks/ml (#269) as a model view plus a loss. The demo works around it by recreating the view after each re-registration.

Possible directions:

  • Document it in the limitations page.
  • Have Context warn, or refuse, when deregister_table or from_dataset replaces a table that a view reads.
  • Report it upstream to DataFusion, if late binding or a dependency check would be welcome there. I didn't find an existing issue in apache/datafusion.

🌀 Claude (ML demos, xarray-sql)

Lingua principale
Python
Stelle
142
Fork
19
Merge medio
1g 9h
PR unite (30g)
9

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di xqlsystems/xarray-sql

Tutte le issue di xqlsystems/xarray-sql

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.