The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop: Invited Presentation: Enabling Scientific Discovery from Diverse Data Sources through In Situ Data Management
Abstract: The successful development of in situ data management capabilities can potentially benefit real-time decision making, design optimization, and data-driven scientific discovery. There are two primary motivations for processing and managing data in situ. The first is the need to decrease data volume. The second motivation is that the in situ methodology can enable scientific discovery from a broad range of data sources, over a wide scale of computing platforms. This talk will highlight the findings of a recent U.S. Department of Energy workshop on in situ data management, outlining six priority research directions. The research directions identify the components and capabilities needed for in situ data management to be successful for a wide variety of applications: making in situ data management more pervasive, controllable, composable, and transparent, with a focus on greater coordination with the software stack, and a diversity of fundamentally new data algorithms.