Workshop: Monitoring HPC Services with CheckMK
Abstract: Administrative monitoring of a range of HPC systems can be time consuming and inefficient with many HPC systems being provided with their own integrated monitoring solutions and an expectation that system managers will monitor each system separately. In order to save staff time, effort and in order to improve the potential for rapid and effective response to emerging problems where systems interact, a "single pane of glass" approach is considered optimal. HPC systems typically utilise relatively boutique technology however which is commonly not monitored by existing out-of-the-box monitoring solutions. In this presentation we detail the application of CheckMK, a general use monitoring system, to HPC systems using non-commodity hardware and software. We focus on the development and use of "check" scripts, which at EPCC have enabled the System Administrators team to simply and reliably monitor all relevant and service-critical aspects of a variety of HPC systems through a single "pane of glass" approach.