Skip to content

[MeL/IO] Parallel writer based on HDF5 and MPI IO

Tobias Meisel requested to merge TobiasMeisel/ogs:parallelHDF5Writer into master
  • First parallel implementation of HDF5 writer. (activated when using XDMF as output format)
  • synchronous: computation->IO->computation->IO ..
  • The collective MPI IO is used as middleware.
  • Each process is writing its own data. This includes all ghost cells and nodes which can be viewed as a multi-block composite via xdmf. The inclusion of ghost cells will enable a simulation restart at already computer time step (not implemented yet).
  • The writing algorithm is not tuned for performance yet. Performance can be compared with VTK option, with this restriction: either use real world data or disable compression for both, see explanation in !3292 (merged)
  • all partition are written into a single hyperslab. Compression is off
  • views to HDF5 file via XDMF file. Default xdmf generated is view to first block (first partition)
  • parallel writing algorithm supports all features introduced for serial writing in !3246 (merged)
  1. Feature description was added to the changelog
  2. Tests covering your feature were added?
  3. Any new feature or behavior change was documented?

Next steps:

  • multi-write (for each attribute) to single-write
  • alternative to hyperslab to allow compression for each partition
  • analysis of bottleneck (darshan)
  • chunk-size
  • one process per node is writing data
  • async io
Edited by Tobias Meisel

Merge request reports