Skip to content

GitLab

  • Menu
Projects Groups Snippets
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in / Register
  • O ogs
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 72
    • Issues 72
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 29
    • Merge requests 29
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Wiki
    • Wiki
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • ogs
  • ogs
  • Merge requests
  • !3292

Merged
Created Dec 01, 2020 by Tobias Meisel@TobiasMeiselMaintainer

[MeL/IO] Fix default HDF5 compression factor

  • Overview 0
  • Commits 1
  • Pipelines 1
  • Changes 1

We are using deflate algorithm, a combination of LZSS and Huffman coding

With compression factor we define how much time we spend for analysis of repetition patterns. In OGS mesh data we either have very simple pattern (major parts of repetition or almost no repetition. Searching for advanced patterns is probably waste of time. We are warned that artificial data (e.g. cube) are more suitable for compression than real world data. A range from no to fast compression is suitable.

-For comparison with vtu files we will now set compression factor to 5, which is the upper bound of plausible settings. Examples for achieved compression ratios in table:

Hydraulic Flow geometry topology Hydraulic Flow step1 Hydraulic Flow step 11
dim 1e6 1e6 x 3 9e6 1e6 1e6
compression 1027:1 8.5:1 3.2:1 4.6:1 1.068:1
type float float int float float
value pattern all 0 loop x y z if structured grid repetition double exponent repetition because values are in a specific range repetition of type and using of same point id on boundaries chunks of equal values no pattern

Test done with isotropic gravity driven 3D (h5dump -H -p *.h5)

Edited Dec 02, 2020 by Tobias Meisel
Assignee
Assign to
Reviewer
Request review from
Time tracking
Source branch: default_compression_fix