declare ome-zarr-ness via `zarr-conventions`

Open
  • Is something wrong?
#644 0 comments 2 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Active
Tech stack
json, python
Domain
documentation

Research direction

The issue proposes an RFC and discussion about declaring ome-zarr conventions, but names no files, tests, or implementation entry point. Start by reading the linked Zarr Conventions project and the current ome-zarr specification; the issue does not define a concrete completion criterion beyond starting the conversation.

Written by the indexing model from the issue text.

Description

zarr-conventions

the Zarr Conventions project is a domain-agnostic way to declare in array / group attributes that the array / group has some special semantics. This supports:

what is a convention

basically anything that adds semantics on top of zarr arrays / groups. a new array codec, or a new field in array metadata, would not be a convention. but a special attribute, or special rules for interpreting array values, could be a convention.

how it works

the basic mechanism is extremely simple. you define a small JSON object that declares your convention. An array or group that uses this convention declares it in the top-level of the attributes field by appending a small JSON object to a zarr_conventions field, which is a JSON array. It looks like this, using the uom convention and a fake convention I just made up:

{
  "zarr_conventions": [
    {
      "schema_url": "https://raw.githubusercontent.com/clbarnes/zarr-convention-uom/refs/tags/v1/schema.json",
      "spec_url": "https://github.com/clbarnes/zarr-convention-uom/blob/v1/README.md",
      "uuid": "3bbe438d-df37-49fe-8e2b-739296d46dfb",
      "name": "uom",
      "description": "Units of measurement for Zarr arrays"
    },
    {
      "schema_url": "www.myschema.com/myconvention/schema.json",
      "spec_url": "www.myschema.com/myconvention/spec.md",
      "uuid": "3bbe123d-df45-49fe-9e1c-739296d46dfb",
      "name": "example convention",
      "description": "just a demo"
    }

  ]
}

a generic zarr reader can then inspect the zarr_conventions array and look for any conventions it recognizes. if that reader does recognize the declared conventions, that reader can use code paths specific to that convention (for example, the uom convention could enable displaying array data with the declared unit). if the reader does not recognize some conventions, it can just ignore them and read the data as plain zarr arrays / groups.

how ome-zarr would use it

the simplest thing would be for ome-zarr to mint an ome-zarr convention (maybe someone has already done this), and add references to the zarr-conventions spec + the structure of the ome-zarr specific convention to the ome-zarr spec. This is cheap because minting a new convention is completely decentralized (conventions are identified by uuid), and it changes nothing for ome-zarr readers (you can still identify ome-zarr data by the "ome" keyword if you don't want to read the zarr_conventions array).

the more ambitious thing would be to identify which of ome-zarr's features are themselves separate conventions, and define or use zarr-conventions metadata for each one. this would instantly give your data ecosystem-wide support for licenses, units of measure for data, and semantics for referencing other arrays. things that ome-zarr has developed internally, like the coordinate transforms framework, could also be declared as a zarr convention. I think this is worth considering seriously, because it makes the entire framework more modular and open to the wider zarr ecosystem beyond bioimaging.

yall probably need an rfc before you can do anything here, so this issue is just about starting the conversation. zarr-conventions have been very useful for geospatial data, and I would love to see bioimaging folks and geospatial folks build up the same tools.

Dominant language
Python
Stars
180
Forks
81
Avg merge
5d 7h
Merged PRs (30d)
9

Getting set up

  • No Dockerfile or Docker Compose file
  • No pull request template
  • Read the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ome/ngff

All issues in ome/ngff

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.