Presentation

DescriptionThe processor performance of high performance computing (HPC) systems is increasing at a much higher rate than storage performance. Storage and file system designers therefore require a deep understanding of how HPC application I/O behavior affects current storage system installations in order to improve storage performance. In this work, we contribute to this understanding using application-agnostic file system statistics gathered on compute nodes as well as metadata and object storage file system servers. We analyze file system statistics of more than 4 million jobs over a period of three years on two systems at Lawrence Livermore National Laboratory that include a 15 PiB Lustre file system for storage. Some key observations in our study show that more than 65% HPC users perform significant I/O which are mostly writes; and less than 22% of HPC users who submit write-intensive jobs perform efficient writes to the file system.