Extract asset anatomy from HERD ontology links in NWB files

Aperta
  • Qualcosa non va?
#1,979 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@bendichter ci sta già lavorando.

Dal 4/11/2026.

  • #1980 di @bendichter — aperta

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
backend, data

Direzione di ricerca

Start in pynwb_utils and trace the existing anatomy extraction from #1807, including how it fills wasDerivedFrom[BioSample].anatomy and handles a synthetic brain-region-sample. Then inspect how NWB exposes external_resources and HERD entity IDs, URIs, and keys; implement ontology filtering, Subject exclusions, labels, deduplication, and HERD precedence with tests for the DANDI:001888 example. Done when HERD anatomy is used across species and string matching remains the fallback for files without HERD.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

Follow-up to #1806. NWB files can carry a HERD table (/general/external_resources) that links values in the file to ontology terms. When a HERD row links a value to an anatomy term (UBERON, Allen MBA, ...), dandi should use that term for the asset's anatomy. The author asserted these links, so they should take precedence over the string matching added in #1807 (and extended in #1825).

In #1806 we agreed to put a BioSample with .anatomy into each asset's wasDerivedFrom, then aggregate across assets in dandischema. This issue feeds that same slot from HERD.

Example: DANDI:001888

https://dandiarchive.org/dandiset/001888/draft holds rat EEG/EMG, 23 files. Each file has a HERD table. For sub-GRIN2B-129/sub-GRIN2B-129_ses-2021-03-26_ecephys.nwb:

Object Key (value in file) Entity
electrodes.location S1_Tr, S1Hl_S1Fl UBERON:0008933 (primary somatosensory cortex)
electrodes.location M1_anterior UBERON:0001384 (primary motor cortex)
electrodes.location M2_Fra, M2_anterior UBERON:0016634 (premotor cortex)
electrodes.location V1_M UBERON:0002436 (primary visual cortex)
electrodes.location V2_ML UBERON:0022232 (secondary visual cortex)
electrodes.location EMG UBERON:0002380 (trapezius muscle)
ElectrodeGroup EEGArray.location cortex UBERON:0000956 (cerebral cortex)
ElectrodeGroup EMGArray.location trapezius muscle UBERON:0002380

The same table also links Subject.species, Subject.strain and Subject.genotype (NCBITaxon:10116, RRID:RGD_2308852, RRID:RGD_14394515). This issue covers anatomy only, but the HERD reader could also cover strain RRIDs, which #1724 proposed getting from HERD and which aren't extracted yet (#1716 added the strain text only).

The current asset metadata for these files (uploaded with dandi-cli 0.76.8) has no wasDerivedFrom, so no anatomy. String matching would not help here: the subject is a rat, which #1807 skips, and lab abbreviations like S1_Tr and M2_Fra would not match UBERON labels or synonyms even with #1825. HERD already holds the answer.

Proposed behavior

  1. In pynwb_utils, if the file has nwb.external_resources, collect every HERD entity whose ID uses an anatomy ontology prefix (UBERON:, MBA:, and other atlas prefixes we recognize), whatever object or field it is linked from.
  2. Skip entities linked from Subject objects. UBERON also contains life-stage terms (e.g. UBERON:0000113 "post-juvenile adult stage") that a converter could link from Subject.age. Species, strain and genotype are handled separately anyway.
  3. Turn each remaining entity into models.Anatomy(identifier=<entity_uri>, name=<label>) and dedupe by identifier.
  4. Put the result in the same place #1807 does: wasDerivedFrom[BioSample].anatomy, on the deepest real BioSample, or on the synthetic brain-region-sample when there is none.
  5. Precedence: if HERD gives anatomy for a file, use it and skip string matching. String matching stays as the fallback for files without HERD.
  6. Apply to all species. HERD needs no species check.

Why filter by ontology rather than by source field: anatomy is often linked from fields outside the standard location attributes and columns, such as a brain_area column on the units table, a custom region column, or an injection site in an extension type. A list of allowed fields would miss these and keep growing.

Open questions

  • Labels. HERD stores entity_id and entity_uri but no label, and Anatomy.name is required and, since dandi/dandi-schema#444, must be non-empty. Options: look up the label in the UBERON table bundled by #1825 (nervous system only, so it would miss trapezius muscle); query OLS at upload time; or fall back to the HERD key (S1_Tr), which is a poor display name. My suggestion: bundled table first, then the key.
  • Non-brain anatomy. EMG → trapezius muscle is a real recording site but not a brain area. I think it belongs in anatomy, since the field is anatomy rather than brain area, but #1807 is framed around brain areas.
  • Redundant ancestors. cortex (group level) is an ancestor of every electrode-level term. Keep both, or drop terms that have a more specific descendant in the same file?
  • Body parts that aren't recording sites. A stimulus table could link a column like "whisker touched" to UBERON. Under the rule above it would count as anatomy. This seems rare and arguably fits "anatomical part studied", so I'd accept it for now and add a skip list only if it turns up in practice.

Dandiset-level summary (out of scope here)

AssetsSummary has no anatomy field, and the archive computes it with dandischema's aggregate_assets_summary, so dandi-cli cannot fill it. Showing anatomy at the Dandiset level needs a dandi-schema change: add anatomy: Optional[List[Anatomy]] to AssetsSummary and collect it from wasDerivedFrom BioSamples in _add_asset_to_stats. Filed as dandi/dandi-schema#446. The Dandiset about field is curated by users, so it shouldn't be filled automatically.

Lingua principale
Python
Stelle
28
Fork
37
Merge medio
3g 21h
PR unite (30g)
13

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di dandi/dandi-cli

Tutte le issue di dandi/dandi-cli

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.