Extract asset anatomy from HERD ontology links in NWB files

未关闭
#1,979 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

@bendichter 已经在做这个了。

开始于 2026年11月4日。

  • #1980 来自 @bendichter —— 未关闭

评估

难度
4/5
预计耗时
3-5 天
新手友好度
48/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
python
领域
backend, data

调研方向

Start in pynwb_utils and trace the existing anatomy extraction from #1807, including how it fills wasDerivedFrom[BioSample].anatomy and handles a synthetic brain-region-sample. Then inspect how NWB exposes external_resources and HERD entity IDs, URIs, and keys; implement ontology filtering, Subject exclusions, labels, deduplication, and HERD precedence with tests for the DANDI:001888 example. Done when HERD anatomy is used across species and string matching remains the fallback for files without HERD.

由索引模型根据 Issue 内容生成。

描述

Summary

Follow-up to #1806. NWB files can carry a HERD table (/general/external_resources) that links values in the file to ontology terms. When a HERD row links a value to an anatomy term (UBERON, Allen MBA, ...), dandi should use that term for the asset's anatomy. The author asserted these links, so they should take precedence over the string matching added in #1807 (and extended in #1825).

In #1806 we agreed to put a BioSample with .anatomy into each asset's wasDerivedFrom, then aggregate across assets in dandischema. This issue feeds that same slot from HERD.

Example: DANDI:001888

https://dandiarchive.org/dandiset/001888/draft holds rat EEG/EMG, 23 files. Each file has a HERD table. For sub-GRIN2B-129/sub-GRIN2B-129_ses-2021-03-26_ecephys.nwb:

Object Key (value in file) Entity
electrodes.location S1_Tr, S1Hl_S1Fl UBERON:0008933 (primary somatosensory cortex)
electrodes.location M1_anterior UBERON:0001384 (primary motor cortex)
electrodes.location M2_Fra, M2_anterior UBERON:0016634 (premotor cortex)
electrodes.location V1_M UBERON:0002436 (primary visual cortex)
electrodes.location V2_ML UBERON:0022232 (secondary visual cortex)
electrodes.location EMG UBERON:0002380 (trapezius muscle)
ElectrodeGroup EEGArray.location cortex UBERON:0000956 (cerebral cortex)
ElectrodeGroup EMGArray.location trapezius muscle UBERON:0002380

The same table also links Subject.species, Subject.strain and Subject.genotype (NCBITaxon:10116, RRID:RGD_2308852, RRID:RGD_14394515). This issue covers anatomy only, but the HERD reader could also cover strain RRIDs, which #1724 proposed getting from HERD and which aren't extracted yet (#1716 added the strain text only).

The current asset metadata for these files (uploaded with dandi-cli 0.76.8) has no wasDerivedFrom, so no anatomy. String matching would not help here: the subject is a rat, which #1807 skips, and lab abbreviations like S1_Tr and M2_Fra would not match UBERON labels or synonyms even with #1825. HERD already holds the answer.

Proposed behavior

  1. In pynwb_utils, if the file has nwb.external_resources, collect every HERD entity whose ID uses an anatomy ontology prefix (UBERON:, MBA:, and other atlas prefixes we recognize), whatever object or field it is linked from.
  2. Skip entities linked from Subject objects. UBERON also contains life-stage terms (e.g. UBERON:0000113 "post-juvenile adult stage") that a converter could link from Subject.age. Species, strain and genotype are handled separately anyway.
  3. Turn each remaining entity into models.Anatomy(identifier=<entity_uri>, name=<label>) and dedupe by identifier.
  4. Put the result in the same place #1807 does: wasDerivedFrom[BioSample].anatomy, on the deepest real BioSample, or on the synthetic brain-region-sample when there is none.
  5. Precedence: if HERD gives anatomy for a file, use it and skip string matching. String matching stays as the fallback for files without HERD.
  6. Apply to all species. HERD needs no species check.

Why filter by ontology rather than by source field: anatomy is often linked from fields outside the standard location attributes and columns, such as a brain_area column on the units table, a custom region column, or an injection site in an extension type. A list of allowed fields would miss these and keep growing.

Open questions

  • Labels. HERD stores entity_id and entity_uri but no label, and Anatomy.name is required and, since dandi/dandi-schema#444, must be non-empty. Options: look up the label in the UBERON table bundled by #1825 (nervous system only, so it would miss trapezius muscle); query OLS at upload time; or fall back to the HERD key (S1_Tr), which is a poor display name. My suggestion: bundled table first, then the key.
  • Non-brain anatomy. EMG → trapezius muscle is a real recording site but not a brain area. I think it belongs in anatomy, since the field is anatomy rather than brain area, but #1807 is framed around brain areas.
  • Redundant ancestors. cortex (group level) is an ancestor of every electrode-level term. Keep both, or drop terms that have a more specific descendant in the same file?
  • Body parts that aren't recording sites. A stimulus table could link a column like "whisker touched" to UBERON. Under the rule above it would count as anatomy. This seems rare and arguably fits "anatomical part studied", so I'd accept it for now and add a skip list only if it turns up in practice.

Dandiset-level summary (out of scope here)

AssetsSummary has no anatomy field, and the archive computes it with dandischema's aggregate_assets_summary, so dandi-cli cannot fill it. Showing anatomy at the Dandiset level needs a dandi-schema change: add anatomy: Optional[List[Anatomy]] to AssetsSummary and collect it from wasDerivedFrom BioSamples in _add_asset_to_stats. Filed as dandi/dandi-schema#446. The Dandiset about field is curated by users, so it shouldn't be filled automatically.

主要语言
Python
星标
28
派生
37
平均合并
3 天 21 小时
30 天内合并 PR
13

环境准备

这个项目没有提供开发容器、Dockerfile 或贡献指南,环境需要你自己搭建:先看它的 README,通用步骤见我们的新手贡献指南。

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

dandi/dandi-cli 的其他 Issue

查看 dandi/dandi-cli 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。