Investigate Hermes VOL

Open
  • Is something wrong?
#636 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
20/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Stale
Tech stack
cpp
Domain
data-engineering, infrastructure

Research direction

No source files, tests, or entry points are identified. Start by clarifying how HDF5 VOL properties, schema files, or environment variables should pass intent to Hermes, then define how a 3D array is divided into regions and tagged. Done should include a concrete interface for custom region searches and data tagging.

Written by the indexing model from the issue text.

Description

There has been interest in transparent dynamic data tagging in Hermes.
However, this requires advanced knowledge of data format, which is difficult to know in a VFD.
We will likely need a VOL.

We have a scientist with climate data.
Some data has temperature, humidity, and pressure of ocean.
Scientist is always interested in searching regions - e.g., regions with extremely high temperature.

Scientist creates a dataset for the region.
Then, scientist gives it to the data management system to track temperature on the region.

Let's say we have an app with HDF5.
HDF5 writes a 3D array.
HDF5 calls Hermes.
How does Hermes divide the 3D array?

The user interacts with HDF5.
How does the user pass the intention through the HDF5 API?

  • I might use a schema file and environment variables.
  • HDF5 has VOL properties.
  • We can decompose that array into regions.
  • Then, we can find regions satisfying a custom, user-define function.
  • We tag it.
Dominant language
C++
Stars
34
Forks
19
No merged PRs in 30d

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from HDFGroup/hermes

All issues in HDFGroup/hermes

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.