Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

`metadata.nwb.get_metadata()` validates the whole BIDS dataset for every NWB file, then discards the result

Abierto
  • ¿Algo no está bien?
#1,962 1 comentario 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 1 día

@CodyCBakerPhD ya está trabajando en esto.

Desde el 13/10/2026.

Evaluación

Este issue todavía no se ha evaluado.

Descripción

BIDS performance

For an NWB file inside a BIDS dataset, dandi.metadata.nwb.get_metadata() builds a new BIDSDatasetDescriptionAsset (via find_bids_dataset_description()) and asks it for the file's BIDS metadata and BIDS version:
https://github.com/dandi/dandi-cli/blob/ad8f64d9abeada4bf87b68eb1a69bea36895ba03/dandi/metadata/nwb.py#L56-L83

The object is new, so its caches are empty, and each call validates the whole dataset twice:

  • metadata: _get_metadata() → bidsschematools validate_bids(self.bids_root) (bids.py#L100)
  • version: get_validation_bids_version() → _validate() → deno bids_validate(self.bids_root) (bids.py#L121)

NWBBIDSAsset.get_metadata() (used by upload and validate) reaches this function through NWBAsset.get_metadata() → nwb2asset(). So the description object that find_dandi_files() shares across the dataset, whose results are cached, is not reused. A dataset with N NWB files gets 2N extra full-dataset validations. Counted on ieeg_epilepsyNWB from bids-examples with #1958 applied: one NWBBIDSAsset.get_metadata() runs validate_bids twice and bids_validate once.

For NWB files the result is discarded anyway. The BIDS branch can only set the Participant fields that are also in metadata_all_fields: age, genotype, identifier, sex, species, strain. _get_pynwb_metadata() then overwrites every one of them, with None when the NWB file lacks the value. The only key that survives is bids_version, which is always None (#1961). On ieeg_epilepsyNWB, get_metadata() returns the same dict with the BIDS branch disabled, apart from bids_version: None. (The BIDS metadata here comes only from path entities, i.e. subject_id/session_id; participants.tsv is not read.)

History
  • #1164 (dandi ls for BIDS) added the BIDS branch, at first only for non-NWB files (else:). Commit 11c18f5a then moved it in front of the NWB branch (if bids: … elif nwb:).
  • #1183 changed elif to if so that both branches run for NWB files ("joint BIDS-NWB metadata extraction"). That is also what closed the files.bids → metadata.nwb → files.bids loop that #1958 fixes.
Suggested fix

Skip the BIDS branch for NWB files in nwb.get_metadata(). It is still needed for non-NWB BIDS files in dandi ls and organize. If BIDS values should ever fill gaps in NWB metadata, NWBBIDSAsset.get_metadata() is the place to do it: it already merges both and holds the dataset-wide, cached description. That would also make metadata.nwb stop depending on files.bids for NWB files, which removes the call loop altogether rather than only the recursive call.

Lenguaje dominante
Python
Estrellas
29
Forks
37
Merge medio
1 d 5 h
PR fusionados (30 d)
20

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de dandi/dandi-cli

Todos los issues de dandi/dandi-cli

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.