Skip to content

[MeL/IO] Fix default HDF5 compression factor

Tobias Meisel requested to merge TobiasMeisel/ogs:default_compression_fix into master

We are using deflate algorithm, a combination of LZSS and Huffman coding

With compression factor we define how much time we spend for analysis of repetition patterns. In OGS mesh data we either have very simple pattern (major parts of repetition or almost no repetition. Searching for advanced patterns is probably waste of time. We are warned that artificial data (e.g. cube) are more suitable for compression than real world data. A range from no to fast compression is suitable.

-For comparison with vtu files we will now set compression factor to 5, which is the upper bound of plausible settings. Examples for achieved compression ratios in table:

Hydraulic Flow geometry topology Hydraulic Flow step1 Hydraulic Flow step 11
dim 1e6 1e6 x 3 9e6 1e6 1e6
compression 1027:1 8.5:1 3.2:1 4.6:1 1.068:1
type float float int float float
value pattern all 0 loop x y z if structured grid repetition double exponent repetition because values are in a specific range repetition of type and using of same point id on boundaries chunks of equal values no pattern

Test done with isotropic gravity driven 3D (h5dump -H -p *.h5)

Edited by Tobias Meisel

Merge request reports