Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

`metadata.nwb.get_metadata()` validates the whole BIDS dataset for every NWB file, then discards the result

Aperta
  • Qualcosa non va?
#1,962 1 commento 0 reazioni 1 assegnatario Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@CodyCBakerPhD ci sta già lavorando.

Dal 13/10/2026.

Valutazione

Questa issue non è ancora stata valutata.

Descrizione

BIDS performance

For an NWB file inside a BIDS dataset, dandi.metadata.nwb.get_metadata() builds a new BIDSDatasetDescriptionAsset (via find_bids_dataset_description()) and asks it for the file's BIDS metadata and BIDS version:
https://github.com/dandi/dandi-cli/blob/ad8f64d9abeada4bf87b68eb1a69bea36895ba03/dandi/metadata/nwb.py#L56-L83

The object is new, so its caches are empty, and each call validates the whole dataset twice:

  • metadata: _get_metadata() → bidsschematools validate_bids(self.bids_root) (bids.py#L100)
  • version: get_validation_bids_version() → _validate() → deno bids_validate(self.bids_root) (bids.py#L121)

NWBBIDSAsset.get_metadata() (used by upload and validate) reaches this function through NWBAsset.get_metadata() → nwb2asset(). So the description object that find_dandi_files() shares across the dataset, whose results are cached, is not reused. A dataset with N NWB files gets 2N extra full-dataset validations. Counted on ieeg_epilepsyNWB from bids-examples with #1958 applied: one NWBBIDSAsset.get_metadata() runs validate_bids twice and bids_validate once.

For NWB files the result is discarded anyway. The BIDS branch can only set the Participant fields that are also in metadata_all_fields: age, genotype, identifier, sex, species, strain. _get_pynwb_metadata() then overwrites every one of them, with None when the NWB file lacks the value. The only key that survives is bids_version, which is always None (#1961). On ieeg_epilepsyNWB, get_metadata() returns the same dict with the BIDS branch disabled, apart from bids_version: None. (The BIDS metadata here comes only from path entities, i.e. subject_id/session_id; participants.tsv is not read.)

History
  • #1164 (dandi ls for BIDS) added the BIDS branch, at first only for non-NWB files (else:). Commit 11c18f5a then moved it in front of the NWB branch (if bids: … elif nwb:).
  • #1183 changed elif to if so that both branches run for NWB files ("joint BIDS-NWB metadata extraction"). That is also what closed the files.bids → metadata.nwb → files.bids loop that #1958 fixes.
Suggested fix

Skip the BIDS branch for NWB files in nwb.get_metadata(). It is still needed for non-NWB BIDS files in dandi ls and organize. If BIDS values should ever fill gaps in NWB metadata, NWBBIDSAsset.get_metadata() is the place to do it: it already merges both and holds the dataset-wide, cached description. That would also make metadata.nwb stop depending on files.bids for NWB files, which removes the call loop altogether rather than only the recursive call.

Lingua principale
Python
Stelle
29
Fork
37
Merge medio
1g 10h
PR unite (30g)
17

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di dandi/dandi-cli

Tutte le issue di dandi/dandi-cli

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.