This is the second workshop dedicated to the development of a Many Robots Replication Playbook, a guide to conducting collaborative HRI replications using real robot systems and broad, representative samples of human subjects. Inspired by the Many Labs replication projects in other disciplines studying human behavior (see Many Labs 1, 2, 3, 4, 5, Many Babies, Many EEGs) — and their impact on the adoption of practices that lead to reproducible empirical science — we propose a Many Robots replication project that can:

• Verify foundational empirical HRI results to confirm their robustness and reliability

• Facilitate the discovery of features and practices that make for replicable science

• Spur a credibility revolution with widespread adoption of structural, procedural, and community reforms to promote replicability and support scientists undertaking replication studies

The first workshop, held at RO-MAN in 2024, discussed both the general challenges presented by collaborative replications, and the unique technical challenges presented by replications of HRI. From the initial workshop, four knowledge gaps were identified that will be addressed in the Playbook to facilitate future Many Robots replications:  

1. Candidate studies for replication that provide theoretically important findings to the literature **

2. Robot systems best suited to executing candidate study replications

3. Strategies for facilitating broad, multi-site participation in replications

4. Incentives for original study author cooperation

** The current iteration of workshop is focused on addressing the first knowledge gap: identifying candidate studies for replication. A critique of collaborative replication efforts in other disciplines has been the selection criteria. Criticisms can be boiled down to "cherry-picking" studies because they were, for example, easy to run, received many citations, published in top-tier journals, or likely to replicate. These criticisms show the need for a community-driven study selection protocol. The selection of studies to replicate should be based on their theoretical or practical value (i.e., what we stand to learn by replicating the finding). Community identification of high-value replication candidates will support building on the existing body of research and extending the state of the art. HRI research has been criticized for lacking the theoretical principles that allow for systematic cumulative science. Without theory, HRI researchers have problems agreeing on the conceptualization and definition of constructs, and struggle to narrow down the potential variables that are most central to describing how constructs relate to one another. Identifying candidate studies for replication that provide theoretically important findings will therefore be valuable to the HRI community by allowing for corroboration across more diverse samples, and the development/refinement of theory.