SC19 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

Poster 100: Comparison of Array Management Library Performance - A Neuroscience Use Case


Authors: Donghe Kang (Ohio State University), Oliver RĂ¼bel (Lawrence Berkeley National Laboratory), Suren Byna (Lawrence Berkeley National Laboratory), Spyros Blanas (Ohio State University)

Abstract: Array management libraries, such as HDF5, Zarr, etc., depend on a complex software stack that consists of parallel I/O middleware (MPI-IO), POSIX-IO, and file systems. Components in the stack are interdependent, such that effort in tuning the parameters in these software libraries for optimal performance is non-trivial. On the other hand, it is challenging to choose an array management library based on the array configuration and access patterns. In this poster, we investigate the performance aspect of two array management libraries, i.e., HDF5 and Zarr, in the context of a neuroscience use case. We highlight the performance variability of HDF5 and Zarr in our preliminary results and discuss potential optimization strategies.

Best Poster Finalist (BP): no

Poster: PDF
Poster summary: PDF


Back to Poster Archive Listing