The replicability of measurements plays a foundational role in scientific enterprise, as it is usually regarded as one of the criteria demarcating science from pseudo-science (Popper 1934, Schmidt 2009, Braude 1979, Nosek et al. 2012). It is hard to overestimate its value: it ensures reliable, objective science, it allows transparent communication, enables collaboration across scientific communities and fosters rapid scientific progress.

Its central significance in scientific theorizing and practice is striking in the following ways: (i) successful but nonreplicable measurements have been discarded and regarded as unscientific;^[1] (ii) the replication of measurements that conflict with a very well established theory has been considered to be sufficient to reject the latter;^[2] (iii) theories have been constructed in such a way as to allow for replicable measurements;^[3] (iv) researchers have often referred to replicability as the gold standard of scientific research.^[4]

In recent years, however, we have witnessed an increasing preoccupation of the scientific community with the weakening role of replicability in science. According to a survey in Nature (Baker 2016), 90% of the 1500 respondents declared that they are significantly or slightly concerned about the replicability of measurements presented in the scientific literature. This worry is well motivated by other results of the same survey, in which 70% of the respondents declared that they failed to replicate at least one scientific experiment presented in the literature and 50% of them even admitted to having failed to replicate at least one of their own experiments. Moreover, two other facts support the belief that there is an increasing rate of nonreplicable measurements in the literature: the fact that retraction rates of articles presenting scientific measurements have increased tenfold in the past decade (Steen 2013) and the fact that 34% of the respondents in the Nature survey (Baker 2016) admit to having published at least one study in the past 5 years without establishing a procedure for replicability. It has also been noted that the growing proliferation of non-repeatable findings is caused on the one hand by the pressure of publishing and on the other by the fact that important journals more highly value original findings than replication studies. This progressive heedlessness toward replicability and the apparent spawning of nonreplicable measurements have led to the belief that science is now facing a replication crisis (Baker 2016). This problem seems to be particularly acute in psychology and neuroscience (where 60% of experiments fail to replicate)^[5], medicine (only 10% of measurements in cancer medicine are reproducible)^[6], but also in the natural sciences, such as biology and physics (where in both cases 60% of scientists have failed, at least once, to replicate their own experiments).^[7]

The replication crisis is a complex problem that features different components, such as a lack of transparency of scientific reports on measurements that makes it is impossible to replicate them, the aforementioned absence of replication studies that confirm previous measurements and the increase in questionable, unrepeatable ‘scientific’ practices. While recent studies have suggested that this crisis is only a misguided narrative that is based on a distorted characterization of the actual situation in science (such studies claiming that only 2% of scientific results are non-repeatable), these opposing voices (Peng 2015) have not successfully silenced the debate on the replicability crisis but on the contrary have provoked an even richer discussion. Within this discussion, many have pointed out the impossibility of achieving absolute replicability due to inevitable changes in environment, instruments or the objects of investigation (Lynch et al. 2015), and the inevitability of non-repeatability featuring in all those sciences which deal with very rare materials or unique instances of certain phenomena. Some have even emphasized the crucial importance of non-reproducibility in science not only as a possible source for new findings, but even as a vital and essential part of scientific discovery (Redish et al 2018). Without a failure of replicability, they claim, there would be no progress.

The urge to write a philosophical project on the replication crisis stems from the P.I.’s realization that whilst there is a growing awareness of this problem, such that it has become the subject of persistent sociological analyses and has gained popularity within each scientific discipline, this topic surprisingly has not yet become a consolidated stream of research in the philosophical community.

Exceptions of course are there to be found. First of all, there are two loci where this topic has gained the attention of philosophers: in the recently-published entry in the Stanford Encyclopedia, The Reproducibilty of Scientific Results, and in a topical collection edited by Prof. Jan Sprenger and Dr. Mattia Andreoletti, which is still under preparation. Both cases show that the replication crisis will soon become a cutting-edge, central theme of interest in philosophy. However, in both cases the topic has been approached mainly from the sociological and epistemological perspectives, as their focus and goal are to combine social studies of science with epistemological methodologies and techniques. Indeed, the main research questions addressed in the topical collection are how statistical, sociological or methodological reforms can address the replication crisis, and whether replication failures impact on the authority of science. This goal will certainly be beneficial and will have a relevant impact on the scientific community. However, we believe that a more theoretical perspective is needed if our goal is not only to find practical means to stem the problem of the replication crisis but also to understand and conceptualize the problem. It is actually the P.I.’s belief that only after a careful metaphysical analysis of the concepts involved in the replicability crisis can the problem be framed and addressed in the most efficient and resolutive way.

Other notable works in the field of reproducibility are by Norton (2014), Leonelli (2018), Feest (2018) and Love (2019). However, none of these authors have approached this topic metaphysically. Their strategy was rather to review historical or contemporary scientific cases in order to draw philosophical conclusions about the meaning and value of replicability. Moreover, their goal was to inquire into the epistemic significance of replicability as an epistemic principle. In contrast, this project, although it will start in following their steps and contributing to the stream of research they have initiated, will also provide a fresh reconstruction of the problem of replicability from a metaphysical perspective by exposing the debate to key metaphysical issues that, according to the P.I., are central to the problem of repeatability, in particular the metaphysical debates on scientific realism, measurements, models. This will shed new light upon the dependence of the concept of replicability on these metaphysical issues and on the mathematical language in which physical theories and models are formulated. In the following, I list the project’s aspirations.

Within the approach we are proposing, the first questions that are relevant and need to be addressed at this stage are therefore not how to quantify the degrees of replicability of measurements or on how to apply formal epistemological techniques to control the quality of scientific research. In contrast, the very first step of this project will be of clarificatory nature. The term ‘replicability’, and its related terms, are ubiquitous in science. How are these terms significantly different and differently significant in different contexts? How can we bridge the different meanings of these terms without losing the specificity they need to apply to different fields? Should they be considered as overarching epistemic values or as necessary and sufficient conditions for science? In this part of the project, we shall extend the work initiated by Norton (2014), Leonelli (2018), Feest (2018), Love (2019), and others.

After such a clarification, it will be easier to address the more foundational questions on the role of repeatability in science from a metaphysical perspective. This approach, we believe, will constitute the most interesting and novel part of the project. It will connect the questions addressed in the first part of the project with metaphysical debates on the nature of our physical theories, measurements, and our accessibility to the world. It will show how the concept and value of replicability are connected to these other metaphysical issues, and how these help us approach and understand the problem of replicability in a more meaningful way.

After this metaphysical investigation it will be clear that a mature understanding of repeatability requires a careful analysis of mathematical models. Therefore, the next stage of the project, which also breaks new ground, will be devoted to the role that mathematical models and mathematics in general play in the concept of replicability, as we believe that mathematics, being used in metrology, in formulating our scientific theories and in creating models about measurements, shapes how we conceive replicability. In particular, it will be our aim to distinguish three different ways in which mathematics plays a crucial role in the concept of replicability: it allows transparent communication and thus the replicability of measurement; it allows a calculation of the degree of replicability of measurements and thus the determination of whether measurements are indeed replicable; and it provides models for designing effective reproducible measurements and ways to analyse their data. Finally, this project will address a provoking question. Normally mathematics is regarded as the discipline which enables experiments to be repeatable and reproducible—but is it not the case that the nature of mathematics itself makes some measurements nonrepeatable, nonreplicable or non-reproducible?

[1] For instance, the case of cold fusion, as is traditionally presented in the literature. But see Norton 2015 for a nuanced examination of this historical episode.

[2] For instance, the experiments by Bednorz and Muller, which revolutionized super-conductivity. Refer to Di Bucchianico 2014.

[3] For instance, the introduction of the collapse postulate in quantum mechanics by von Neumann. See von Neumann 1955, Wallace 2016, Zirper Ms.

[4] For instance, in Jasny et al. 2011.

[5] See Baker 2016, Fanelli 2018, and Manninen et al. 2018.

[6] See Wen et al. 2018.

[7] See Baker 2016.