Extract asset anatomy from HERD ontology links in NWB files
维护者通常 1 天内回复
评估
调研方向
Start in pynwb_utils and trace the existing anatomy extraction from #1807, including how it fills wasDerivedFrom[BioSample].anatomy and handles a synthetic brain-region-sample. Then inspect how NWB exposes external_resources and HERD entity IDs, URIs, and keys; implement ontology filtering, Subject exclusions, labels, deduplication, and HERD precedence with tests for the DANDI:001888 example. Done when HERD anatomy is used across species and string matching remains the fallback for files without HERD.
由索引模型根据 Issue 内容生成。
描述
Summary
Follow-up to #1806. NWB files can carry a HERD table (/general/external_resources) that links values in the file to ontology terms. When a HERD row links a value to an anatomy term (UBERON, Allen MBA, ...), dandi should use that term for the asset's anatomy. The author asserted these links, so they should take precedence over the string matching added in #1807 (and extended in #1825).
In #1806 we agreed to put a BioSample with .anatomy into each asset's wasDerivedFrom, then aggregate across assets in dandischema. This issue feeds that same slot from HERD.
Example: DANDI:001888
https://dandiarchive.org/dandiset/001888/draft holds rat EEG/EMG, 23 files. Each file has a HERD table. For sub-GRIN2B-129/sub-GRIN2B-129_ses-2021-03-26_ecephys.nwb:
| Object | Key (value in file) | Entity |
|---|---|---|
electrodes.location |
S1_Tr, S1Hl_S1Fl |
UBERON:0008933 (primary somatosensory cortex) |
electrodes.location |
M1_anterior |
UBERON:0001384 (primary motor cortex) |
electrodes.location |
M2_Fra, M2_anterior |
UBERON:0016634 (premotor cortex) |
electrodes.location |
V1_M |
UBERON:0002436 (primary visual cortex) |
electrodes.location |
V2_ML |
UBERON:0022232 (secondary visual cortex) |
electrodes.location |
EMG |
UBERON:0002380 (trapezius muscle) |
ElectrodeGroup EEGArray.location |
cortex |
UBERON:0000956 (cerebral cortex) |
ElectrodeGroup EMGArray.location |
trapezius muscle |
UBERON:0002380 |
The same table also links Subject.species, Subject.strain and Subject.genotype (NCBITaxon:10116, RRID:RGD_2308852, RRID:RGD_14394515). This issue covers anatomy only, but the HERD reader could also cover strain RRIDs, which #1724 proposed getting from HERD and which aren't extracted yet (#1716 added the strain text only).
The current asset metadata for these files (uploaded with dandi-cli 0.76.8) has no wasDerivedFrom, so no anatomy. String matching would not help here: the subject is a rat, which #1807 skips, and lab abbreviations like S1_Tr and M2_Fra would not match UBERON labels or synonyms even with #1825. HERD already holds the answer.
Proposed behavior
- In
pynwb_utils, if the file hasnwb.external_resources, collect every HERD entity whose ID uses an anatomy ontology prefix (UBERON:,MBA:, and other atlas prefixes we recognize), whatever object or field it is linked from. - Skip entities linked from
Subjectobjects. UBERON also contains life-stage terms (e.g.UBERON:0000113"post-juvenile adult stage") that a converter could link fromSubject.age. Species, strain and genotype are handled separately anyway. - Turn each remaining entity into
models.Anatomy(identifier=<entity_uri>, name=<label>)and dedupe by identifier. - Put the result in the same place #1807 does:
wasDerivedFrom[BioSample].anatomy, on the deepest real BioSample, or on the syntheticbrain-region-samplewhen there is none. - Precedence: if HERD gives anatomy for a file, use it and skip string matching. String matching stays as the fallback for files without HERD.
- Apply to all species. HERD needs no species check.
Why filter by ontology rather than by source field: anatomy is often linked from fields outside the standard location attributes and columns, such as a brain_area column on the units table, a custom region column, or an injection site in an extension type. A list of allowed fields would miss these and keep growing.
Open questions
- Labels. HERD stores
entity_idandentity_uribut no label, andAnatomy.nameis required and, since dandi/dandi-schema#444, must be non-empty. Options: look up the label in the UBERON table bundled by #1825 (nervous system only, so it would misstrapezius muscle); query OLS at upload time; or fall back to the HERD key (S1_Tr), which is a poor display name. My suggestion: bundled table first, then the key. - Non-brain anatomy.
EMG→ trapezius muscle is a real recording site but not a brain area. I think it belongs inanatomy, since the field is anatomy rather than brain area, but #1807 is framed around brain areas. - Redundant ancestors.
cortex(group level) is an ancestor of every electrode-level term. Keep both, or drop terms that have a more specific descendant in the same file? - Body parts that aren't recording sites. A stimulus table could link a column like "whisker touched" to UBERON. Under the rule above it would count as anatomy. This seems rare and arguably fits "anatomical part studied", so I'd accept it for now and add a skip list only if it turns up in practice.
Dandiset-level summary (out of scope here)
AssetsSummary has no anatomy field, and the archive computes it with dandischema's aggregate_assets_summary, so dandi-cli cannot fill it. Showing anatomy at the Dandiset level needs a dandi-schema change: add anatomy: Optional[List[Anatomy]] to AssetsSummary and collect it from wasDerivedFrom BioSamples in _add_asset_to_stats. Filed as dandi/dandi-schema#446. The Dandiset about field is curated by users, so it shouldn't be filled automatically.
- 主要语言
- Python
- 星标
- 28
- 派生
- 37
- 平均合并
- 3 天 21 小时
- 30 天内合并 PR
- 13
环境准备
这个项目没有提供开发容器、Dockerfile 或贡献指南,环境需要你自己搭建:先看它的 README,通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
dandi/dandi-cli 的其他 Issue
-
pynwb metadata: get neurodata types可能重新可做 关联的 PR 已关闭且未合并。 未关闭
难度 2/5 1-3 小时 新手友好度 65/100
维护者通常 1 天内回复
-
UX
难度 3/5 1-2 天 新手友好度 45/100
维护者通常 1 天内回复
-
`metadata.nwb.get_metadata()` validates the whole BIDS dataset for every NWB file, then discards the result可能已有人在做 @CodyCBakerPhD 于 22 天前认领。 未关闭BIDS performance
dandi/dandi-cli#1962 · 1 条评论 · 已指派 1 人 ·
维护者通常 1 天内回复
-
`bids_version` is always `None` since the switch to bids-validator-deno可能已有人在做 @yarikoptic-gitmate 于 1 天前认领。 未关闭BIDS bug
难度 3/5 半天 新手友好度 45/100
dandi/dandi-cli#1961 · 4 条评论 ·
维护者通常 1 天内回复
-
难度 4/5 3-5 天 新手友好度 48/100
dandi/dandi-cli#1920 · 2 条评论 ·
维护者通常 1 天内回复
相似的 Issue
-
bug
难度 2/5 1-3 小时 新手友好度 69/100
jejjohnson/gaussx#752 ·
维护者通常 1 天内回复
-
难度 1/5 1 小时以内 新手友好度 85/100
521xueweihan/HelloGitHub#4270 ·
-
enhancement owner: den-haag
难度 2/5 1-3 小时 新手友好度 67/100
维护者通常 6 天内回复
-
难度 2/5 1-3 小时 新手友好度 78/100
ok-oldking/ok-wuthering-waves#1828 ·
维护者通常 1 天内回复
-
lesson-submission pending-review
难度 1/5 1 小时以内 新手友好度 65/100
Ikalus1988/MisakaNet#4512 ·
维护者通常 1 天内回复