ENH: Reading TFRs lazily from disk (`h5io` to `h5py`)

Abierto
  • ¿Algo no está bien?
#14,454 2 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
35/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python
Área
data

Línea de trabajo

Start by reading mne.time_frequency.read_tfrs and the lazy loading patterns in mne.io.Raw and mne.Epochs; the issue does not name specific files or tests. Check how the current h5io interface reads TFR data and whether h5py can support the proposed on-demand loading. Done means TFRs can be inspected without eagerly loading large data and .load_data() loads it on demand; this is a broad, incremental migration.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

ENH
Describe the new feature or enhancement

Currently, mne.time_frequency.read_tfrs reads data eagerly from disk.

This is sometimes annoyingly slow when I don't immediately need to work with that data (which can often be enormous, ~10s of GB) and for e.g. just want to inspect it or pass some of its parameters to a downstream function.

It would be nice to follow e.g. mne.io.Raw and mne.Epochs and allow the loading to be lazy and .load_data() on demand.

Describe your proposed implementation

Currently, MNE-Python appears to use h5io for the HDF5 interface. From a cursory look, h5io doesn't appear to support lazy-loading.

This proposal would require migrating to e.g. h5py, which currently also appears to be the library with more contributors/activity.

I suppose this would be a rather large change, but perhaps it's worth it? It could also be done incrementally.

Describe possible alternatives

We could just let it be and allow eager loading. It's just inconvenient.

Additional context

No response

Lenguaje dominante
Python
Estrellas
3.6k
Forks
1.6k
Merge medio
1 d 19 h
PR fusionados (30 d)
67

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de mne-tools/mne-python

Todos los issues de mne-tools/mne-python

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.