Workshop: Metadata Enabled Optimized Partitioning for M to N In Transit Processing with SENSEI
Abstract: In an M to N in transit setting, a simulation runs on M MPI ranks while an endpoint, such as analysis or visualization, runs on N MPI ranks in a separate application. Before data is moved or processed, a partitioning step occurs where decisions are made about how data is mapped from the simulation's M MPI ranks onto the end-point's N MPI ranks. We investigate metadata enabled, optimized partitioning for M to N in transit processing. We extended the SENSEI generic in situ infrastructure for M to N in transit processing. The design incorporates a partitioner object and API that is used to compute the mapping of simulation data onto end-point resources, and a metadata object that comprehensively yet compactly describes data and its mapping onto MPI ranks. In a scaling study using up to 16384 cores on NERSC's Cori system, we compare optimized and unoptimized partitioning approaches for two common analysis operations, slicing and isosurfacing, and vary N, the level of end-point concurrency, and show optimal ratios of M to N for both types of end-point operations and partitioners. We find that metadata enabled partitioning optimizations significantly reduce time to solution, amount of data moved, memory footprint, and cost of solution in node hours, especially when running in the regime where N << M.