The National Academies’ Report on Reproducibility and Replicability in Science: Inspirations for the SC Reproducibility Initiative

Authors: Lorena Barba (George Washington University), Juliana Freire (New York University)

Abstract: This BoF will disseminate the findings and recommendations of the NASEM report on Reproducibility and Replicability in Science, making connections with the SC Reproducibility Initiative. It will seed the community with ideas for taking inspiration from the report’s recommendations for the conference to continue leading in the computing community, increasing the transparency of research. The session will also communicate new initiatives stemming from the NASEM report, affecting the computing community, like the National Information Standards Organization (NISO) Badging Scheme Working Group.

Long Description: The National Academies if Science Engineering and Medicine released a consensus report on May 2019 titled “Reproducibility and Replicability in Science,” a product of nearly two years of study. It includes a number of findings and recommendations for various stakeholder groups, including funding agencies and professional societies. The committee’s definition of reproducibility focuses on computation, and several of its recommendations are relevant to the SC community, and reinforce the goals of the SC Reproducibility Initiative. It recommends to funding agencies and organizations to consider investing in research and development of open-source, usable tools and infrastructure that support reproducibility—presenting both challenges and potential opportunities for computing research. Recommendations that may be of particular interest to the SC community focus on including reproducibility in the merit review of proposals, characterizing uncertainty (including uncertainty in measurement and computation), and the need to develop alternate mechanisms for demonstrating reproducibility when research is reliant on non-public data or code.

With the SC19 Reproducibility Chair being a member of the study committee, this BOF aims to shine a light on the connections between the community efforts at SC and the growing emphasis on reproducibility at a national level. The BOF also aims to uncover new initiatives that may continue to influence the conversation and community norms in terms of transparency of computing and data-enabled research. The audience participation will allow connecting these efforts to productive directions for SC in the future of the Reproducibility Initiative.

The BOF leaders will invite to participate: Mike Heroux, SC18 Reproducibility Chair; Beth Plale, SC20 Reproducibility Chair; members of the three tracks under the SC19 Reproducibility Chair; SC leadership; and interested attendees from the conference at large.



Back to Birds of a Feather Archive Listing