Presentation
High Performance I/O Frameworks 101
Event Type
Tutorial
TUT
Data Analytics
Data Management
I/O
Introductory
TimeMonday, 18 November 20198:30am - 5pm
Location405
DescriptionAs concurrency and complexity continue to increase on high-end machines, I/O performance is rapidly becoming a fundamental challenge to achieving exascale computing. Wider adoption of higher-level I/O abstractions will be critically important to address this challenge. Modern I/O libraries provide data models, portable APIs, storage abstractions, and self-describing data containers. They achieve high performance and scalability, allow data to be managed more effectively throughout the data lifecycle, and enable reproducible science.
Part I of this tutorial will provide an overview of parallel I/O systems and summarize the key techniques for obtaining high performance I/O on high-performance computing (HPC) resources at scale. Part II introduces ADIOS and HDF5 libraries, delving through their usage models and examples, showing how to achieve high performance scalable I/O. Part III explains data compression and shows how to use it with I/O libraries. Part IV covers techniques for creating in situ analytics and teaches how to generate visualization services using VTK-M. Finally, Part V will explain data indexing/querying and how to use the libraries to query data both in situ and on files. Over one half of this tutorial will be hands-on sessions, where we provide access to the software, and go through live examples.
Part I of this tutorial will provide an overview of parallel I/O systems and summarize the key techniques for obtaining high performance I/O on high-performance computing (HPC) resources at scale. Part II introduces ADIOS and HDF5 libraries, delving through their usage models and examples, showing how to achieve high performance scalable I/O. Part III explains data compression and shows how to use it with I/O libraries. Part IV covers techniques for creating in situ analytics and teaches how to generate visualization services using VTK-M. Finally, Part V will explain data indexing/querying and how to use the libraries to query data both in situ and on files. Over one half of this tutorial will be hands-on sessions, where we provide access to the software, and go through live examples.
