Using MPI-3 RMA for Active Messages

Workshop: Using MPI-3 RMA for Active Messages

Abstract: Distributed asynchronous programming systems require a scalable, low-latency way to exchange information on the state of units of execution (tasks) across process boundaries. These interactions can be modeled as active messages, i.e., a message that, upon reception, triggers an action on the receiving side. Such actions, generally of short duration, include the posting of the reception for a task input data or the transition of a task into the runnable state. Traditionally, such messages have been implemented using MPI two-sided communication primitives. However, processing reception queues for numerous small messages may incur significant overheads in the progress of the MPI library, notably when handling unexpected messages and iterating over a number of pre-posted requests. In this work, we investigate a different approach for implementing an active message queue using MPI-3 RMA primitives and compare the resulting performance in a task runtime in terms of scalable performance and latency against two-sided implementations. We discuss the latency of basic RMA operations and the resulting performance in terms of latency and scalable throughput of the RMA-based queues. Using a benchmark application, we show that using RMA-based queues may provide notable benefits compared to using MPI two-sided communication for inter-scheduler communication.

Back to Workshop on Exascale MPI (ExaMPI) Archive Listing

Back to Full Workshop Archive Listing