Add support for more stage-out data types; make xarray optional?

Open
#83 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
pandas, python
Domain
data-engineering

Research direction

Locate the current isinstance(thing, xr.Dataset) stage-out check and read how notebook outputs are detected and written. Compare the handling needed for xarray.DataArray, iris.cube.Cube, pandas.DataFrame, and geopandas.GeoDataFrame; done means supported libraries are detected without making unused data libraries, including xarray, hard dependencies.

Written by the indexing model from the issue text.

Description

Currently xcengine only automatically stages out xarray.Dataset instances. We should add other commonly used types. xarray.DataArray is an obvious candidate, but there are also (for example) iris.cube.Cube, pandas.DataFrame, geopandas.GeoDataFrame. This would require a more dynamic data type detection system than the current isinstance(thing, xr.Dataset) to avoid making every potential data library a hard dependency. Such a change could also involve making xarray itself optional -- no sense in having it as a dependency for a notebook that only uses e.g. pandas.

Somewhat related to #61 .

Dominant language
Python
Stars
4
Forks
0
Avg merge
2d 19h
Merged PRs (30d)
8

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from xcube-dev/xcengine

All issues in xcube-dev/xcengine

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.