openPMD/openPMD-api

Streaming: Histogramming of Particle Data

Offen

#1.248 geöffnet am 19.04.2022

 (0 Kommentare) (3 Reaktionen) (1 zugewiesene Person)C++ (57 Forks)auto 404
help wantedtools

Repository-Metriken

Stars
 (160 Sterne)
PR-Merge-Metriken
 (PR-Metriken ausstehend)

Beschreibung

@aeriforme would like to write a data pipeline that takes all particle data and bins it to a 1D or 2D user-defined histogram. This would be a great application of our work in @franzpoeschel et al. https://arxiv.org/abs/2107.06108.

Idea: Let's take the script in https://github.com/openPMD/openPMD-api/blob/dev/src/cli/pipe.py as a template and instead of forwarding the data 1:1 to another file format, we call np.histogram (1D) or np.histogram2d (2D) to store into a histogram per process. Finally, we reduce the histograms (sum) via mpi4py and write them out into a 2D mesh (or simple np.save from rank 0).

@aeriforme works with CINECA clusters.

Ref.:

Contributor Guide