[MeL/IO] HDF: reduce data size
-
Tests are in !3380 (merged)
-
<compress_output>true</compress_output>
in prj-file -
only variable attributes get written at each time step
-
chunks can now be set independently of local size of data.
-
HDF is kept open - this reduces numerous reads (performance)
Caution:
- When compression and consequently chunking is turned in on (
<compress_output>true</compress_output>
in prj-file) it is not possible to measure with Darshan. The error has been reported to HDF Group https://forum.hdfgroup.org/t/parallel-hdf5-write-with-irregular-size-in-one-dimension/8284 and to the Darshan mailing list and will hopefully be fixed soon.