Additionally, the dataset is compressed using ZLIB and initialized with values 4, 8 and 6. In HDFql, this can easily be implemented as follows: create and use file myfile.h5 create dataset mygroup/mydataset as int(3) enable zlib values(4, 8, 6). Summary HDF5 has long been a standard for sharing scientific data across High Performance Computing (HPC) platforms. From the HDF website 'HDF5 is a data model, library, and file format for storing and managing data. It supports an unlimited variety of datatypes, and is designed for flexible.
Hierarchical data format (HDF5) for Modflow, Modpath and ZoneBudget
Abstract
More and more spatially detailed time dependent groundwater models are used and hence input files for models like Modflow (USGS) are becoming larger and larger. These input files are in ASCII format and contain datasets with values for each cell, changing often for each stress period. To diminish the problem of huge ASCII files, the datasets of the input files can be stored in an HDF5 file (Hierarchical data format). HDF5 is a data model, library and file format designed by HDFgroup for storing and managing data, flexible and efficient input and output and high volume and complex data. The file has a binary format and can be compressed with different kinds of compression methods. An HDF5 file consists among others of groups and datasets, referencing a dataset in the HDF5 file is similar to referencing a file in a folder of a file system. The contents of an HDF5 file can be viewed with HDFview, a Java-based viewer. HDF5 files can be constructed manually with the help of HDFview, or with the help of programming languages like C, C++, Matlab, Fortran or Python. The goal of this contribution is to show how HDF can improve data management of Modflow. A similar HDF5 link to Modflow has been implemented in the interface Groundwater Modeling System. Modflow has been extended to be able to read datasets from the ASCII input files which contain a reference to an HDF5 file for every real 1 or 2-dimensional dataset, every 2-dimensional integer dataset and the stress lists described in the Modflow user manual as U1DREL, U2DREL, U2DINT and ULSTRD. Two other programs from USGS, Modpath and Zonebudget, are also using partly the same input files as Modflow, and have been extended to be able to read the requested datasets from the HDF5 file. The total Modflow input file size, i.e. converted ASCII files and HDF5 file, will be decimated compared to the original size. Partly this is due to 'zlib' compression, 'zlib' is a free lossless data compression library. Due to faster reading of the HDF5 data compared to the original ASCII data model the Modflow, Modpath and ZoneBudget run time decreases.
Hdf Zlib File Size
- Publication:
- Pub Date:
- April 2012
- Bibcode:
- 2012EGUGA..14.8028C