hdfgroup/hdf5lib repository overview

⁠Dockerfile Image for HDF5 library with Python 3

⁠Ingredients:

• Python 3.11
• HDF5 Library with ros3vfd
• HDF5 Tools (h5dump, h5ls, h5repack, etc.)
• h5py (Python package for HDF5)
• h5pyd (Python SDK for HDF REST API)
• s3fs (for reading hdf5 files stored in S3)

⁠Instructions

Launch the container interactively.

Example:

$docker run -it -v <mydata>:/data hdfgroup/hdf5lib:1.8.16 /bin/bash

Where "mydata" is the path to a folder on the host that (presumably) holds some HDF5 files.

From the bash prompt you should then be able to use tools such as h5dump or h5repack on the mounted data files.

To use h5pyd, define the environment variables: HS_ENDPOINT, HS_USERNAME, HS_PASSWORD.

To use s3fs to read HDF5 files stored in AWS S3 with h5py, create a s3fs object and use the h5py "file-like object" syntax:

   import s3fs
   import h5py
   s3 = s3fs.S3FileSystem(key=aws_key_id, secret=aws_secret_key)
   f = h5py.File(s3.open(s3path, 'rb'))

⁠See Also:

• https://www.hdfgroup.org/HDF5⁠ for information on how to use HDF5
• https://docs.h5py.org/en/stable/⁠ for information on using h5py
• https://github.com/HDFGroup/h5pyd⁠ for information on using h5pyd