In-memory tables: an in-place write that moves a NaN reads wrongly (the validity bitmap is computed at registration)
維護者通常 3 天內回覆
還沒有人認領這個 Issue。
評估
研究方向
Start by reading the from_dataset docstring and the in-memory table registration path described in the issue, then compare how registration builds the validity bitmap with how shared values change after an in-place write. Reproduce the two NaN-to-number and number-to-NaN cases, including with Context.prepare. Done means the behavior or its documented limitation is consistent and both cases are covered.
由索引模型根據 Issue 內容生成。
描述
🌀 Claude (xarray-sql)
Summary
A small in-memory Dataset is registered as a native in-memory table that shares its data variables' memory (#275). from_dataset's docstring says that, as with a lazy table, "changing the Dataset's values in place changes what queries return". That holds for numbers, but not for NaN. xarray-sql reads a NaN as SQL NULL, and the in-memory table works out which values are NULL once, when it is registered, as a validity bitmap beside the shared values. After an in-place write:
- a NaN written where there was a number reads as NaN, not NULL;
- a number written where there was a NaN reads as NULL.
A lazy table, or the same Dataset registered again, reads the new values as expected.
Reproduction
On #275's branch (in-memory-tables), DataFusion 55.0.0:
import numpy as np
import xarray as xr
import xarray_sql as xql
x = np.array([1.0, np.nan, 3.0])
ctx = xql.Context()
ctx.from_dataset("t", xr.Dataset({"x": ("i", x)}, coords={"i": np.arange(3)}))
query = "SELECT i, x, x IS NULL AS null FROM t ORDER BY i"
print(ctx.sql(query).to_pydict())
# {'i': [0, 1, 2], 'x': [1.0, None, 3.0], 'null': [False, True, False]}
x[0] = np.nan
x[1] = 2.0
print(ctx.sql(query).to_pydict())
# {'i': [0, 1, 2], 'x': [nan, None, 3.0], 'null': [False, True, False]}
again = xql.Context()
again.from_dataset("t", xr.Dataset({"x": ("i", x)}, coords={"i": np.arange(3)}))
print(again.sql(query).to_pydict())
# {'i': [0, 1, 2], 'x': [None, 2.0, 3.0], 'null': [True, False, False]}
Options
- Say so in the docstring: an in-place write must keep its NaNs where they were, or the Dataset must be registered again.
- Keep NaN as a value in in-memory tables, with no bitmap (#251 asks for that mode in general). That would differ from lazy tables until #251 lands for both.
The same limit applies to Context.prepare, which runs a planned query again on values written in place. It checks that a write keeps its NaNs where they were.
- 主要語言
- Python
- 星號
- 142
- 分支
- 19
- 平均合併
- 3 天 47 分鐘
- 30 天內合併 PR
- 8
環境準備
- 沒有 Dockerfile 或 Docker Compose 檔案
- 沒有 Pull Request 範本
- 閱讀貢獻指南
從這裡開始
- 先讀完整個 Issue，再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫，在一個分支上完成修改。
- 送出 Pull Request，並在描述裡引用這個 Issue 編號。
xqlsystems/xarray-sql 的其他 Issue
-
dependencies github_actions
難度 2/5 1-3 小時 新手友好度 70/100
xqlsystems/xarray-sql#295 ·
維護者通常 3 天內回覆
-
bug
難度 2/5 1-3 小時 新手友好度 72/100
xqlsystems/xarray-sql#288 ·
維護者通常 3 天內回覆
-
documentation ml
難度 1/5 1-3 小時 新手友好度 85/100
xqlsystems/xarray-sql#268 ·
維護者通常 3 天內回覆
-
enhancement ml
難度 2/5 1-3 小時 新手友好度 64/100
xqlsystems/xarray-sql#267 · 4 則留言 ·
維護者通常 3 天內回覆
-
難度 1/5 1 小時以內 新手友好度 74/100
xqlsystems/xarray-sql#243 ·
維護者通常 3 天內回覆
查看 xqlsystems/xarray-sql 的全部 Issue
相似的 Issue
-
難度 2/5 1-3 小時 新手友好度 74/100
strawberry-graphql/strawberry#4794 ·
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 78/100
dataforgoodfr/cetolog#21 ·
-
難度 2/5 1-3 小時 新手友好度 76/100
btclib-org/btclib#2932 ·
維護者通常 1 天內回覆
-
metrics priority: medium
難度 1/5 1-3 小時 新手友好度 86/100
LearningCircuit/local-deep-research#8336 ·
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 80/100