Presentation
An Evaluation of the CORAL Interconnects
SessionNetwork Evaluation
Event Type
Paper
TP
Architectures
Benchmarks
Communication Optimization
Networks
Performance
Photonics
Silicon Fabrication
TimeWednesday, 20 November 201910:30am - 11am
Location301-302-303
DescriptionIn 2019, the Department of Energy deployed the Summit and Sierra supercomputers, both employing the latest interconnect technology. In this paper, we provide an in-depth assessment of the systems' interconnects, that is based on Enhanced Data Rate (EDR) 100 Gb/s Mellanox Infiniband. Both systems use second-generation EDR Host Channel Adapters (HCAs) and switches adding several new features such as Adaptive Routing (AR), switch-based collectives, HCA-based tag matching, and NVMe-over-Fabrics offload. Although based on the same components, Summit's network is "non-blocking'' (i.e., fully provisioned) and Sierra's network has a 2:1 taper. We evaluate the two systems' interconnects using traditional communication benchmarks as well as real applications. We find that the new Adaptive Routing dramatically improves performance, but the other new features still need improvement.
Download PDF
Authors
