yt-project/yt

Feature request: XArray backend

Open

#1,890 opened on Jul 13, 2018

View on GitHub
 (7 comments) (6 reactions) (1 assignee)Python (315 forks)auto 404
help wantedproposal

Repository metrics

Stars
 (556 stars)
PR merge metrics
 (PR metrics pending)

Description

XArray provides a pandas-like interface to regularly gridded (with missing values) data. This has a very nice analogy to the ds.r[] object, when it is sliced with imaginary step sizes -- i.e., covering grids or arbitrary grids. It would be excellent to provide either a conversion from a covering grid to an xarray or to make this a returned output from some functions.

While it is feasible to do this as-is, for instance:

ds = yt.load("IsolatedGalaxy/galaxy0030/galaxy0030")

d = ds.r[::512j, ::512j, ::512j]

density = d["density"]
coords = [('x', d["x"][:,0,0]), ('y', d["y"][0,:,0]), ('z', d["z"][0,0,:])]
rho = xr.DataArray(density, coords=coords)

This requires that the data be loaded in all-at-once. XArray provides the ability to defer loading, either by providing something as a dask array or by implementing a backend.

Implementing a backend seems to be the most straightforward way, as it would require subclassing AbstractDataStore, perhaps in the mode of the opendap backend. This would be a very cool way to make yt more interoperable with xarray.

Contributor guide