The role of metaphysics in understanding replicability and reproducibility

It is a belief of the P.I. that a profound discussion on replicability and reproducibility requires not only an epistemological approach but also a metaphysical one. This kind of approach will reveal the extent to which the value and importance of these concepts may depend upon philosophical stances toward scientific theories and measurements. Indeed, it is the belief of the P.I. that a fair and complete analysis of repeatability cannot be disentangled from metaphysical theories on scientific theories and measurements. In this stage, the project will inquire into the problem of replicability from the perspectives of three different conceptions of scientific theories and measurement: the naïve scientific realist view, the coherentist view, and perspectival realism.

The first stance on scientific theories into which we shall inquire is the naïve scientific realist view. According to the naïve scientific realist view, our best scientific theories provide a true account of the physical world as their measurements provide direct access to reality. Measurement outcomes, disguised as numbers, via their representational role reveal objective real quantity. This does not mean that all measurement outcomes give us perfect information about reality, as it may turn out that human error or instrumental imperfections are responsible for wrong outcomes. In this regard, repetition and replications ensure a proper and correct result. Within this naïve scientific realist view, repeated and replicated measurement outcomes are epistemically basic and reveal ontological reality. There is one-directional and direct access to the world, via scientific measurements whose outcomes latch onto the world and for this reason constitute our scientific evidence. The conclusion is that repeatability and replicability are the only two necessary conditions for scientific measurements. Once we get the very same measurement outcomes, measurement outcomes are epistemically basic. They provide for us the values that describe and refer to quantities in the real world.

As mentioned above, this is of course a very naïve scientific realist view, based on direct measurements. It completely ignores the sophistication and complexity of metrology, the discipline that deals with measurement design and the maintenance, improvement and control of instruments and whose goal is to recover outcomes from instrument indicators and to ensure measurement objectivity. Metrology operates successfully only by relying on models of the complex interaction between instruments, the measured objects, the experimenters and the environment. The role of metrology is particularly emphasized within the coherentist view of scientific theories, supported by Eran Tal (Tal 2011, 2016, 2017, 2019) and Hasok Chang (Tal & Chang 2017). The coherentist view recognizes the important role of models, which are often mathematical, as mediators between measurement outcomes and reality, and they normally endorse a model-based account of measurement. It is impossible to reach un understanding of the world by simply reading our instrument indicators: we need models to model the interaction between measurement apparatuses and what is being measured. It is also important to bear in mind that the outcomes normally consist of an estimate of the value of the quantity being measured, with associated uncertainty. In order to infer from the final state of the measurement device to the measurement outcome, a model is needed. This kind of model will certainly be based on theoretical and statistical assumptions. In order to ensure the reliability of measurement outcomes, metrologists need to stabilize the way quantity-concepts are applied. Stability is normally understood as the tendency of a single measurement device to produce the same measurement outcome over time and the tendency of different measurement devices of the same kind to reproduce the same outcomes. However, given the crucial dependence of replicability on models and in turn on its theoretical and statistical assumptions, those assumptions are not dogmatic, but they are considered the best available after a process of back and forth adjustment between measurement assumptions and measurement outcomes. During the standardization procedure, metrologists can test whether the theoretical and statistical means of predicting are successful and adjust or replace them. It is interesting to note that standardization reveals an important difference within the concept of replicability which has hitherto been neglected, namely the stability of the very same particular measurement device over time, and stability across different instances of the same measurement device. One famous example to illustrate the point is ‘universal time’, whose standardization is ensured by the stability across time of one single clock and the stability across different clocks all over the earth. Replicability is essential not only concerning identical results (results replicability) and identical procedures (methodological replicability), but because it allows us to build up a result that eludes the specificity of the instruments, its particular instance. Contrary to the naïve scientific realist view, in the model-based account there is no one unique direction of information. Rather, replications help to build a coherent measurement account through epistemic interaction, by which our knowledge is refined. However, no matter how much our knowledge is refined, it will always be within a particular model and so sensitive to its theoretical and statistical assumptions.

The third stance on scientific theories that this project will consider is perspectival realism. Perspectival realism acknowledges the dependence of truth on models, and thus the dependence of the objectivity of our measurement results on models, but it also affirms that it is possible to achieve a transcendental objective characterization of our world via the ‘cross-checking requirement’ by which claims which are model M-dependent need to be evaluated from perspectives (models) other than M. Perspectival realism takes inspiration, for instance, from Thomson’s different strategies to measure electrons. Within this realist stance, it is clear that the ideal objectivity which science wants to achieve can be obtained only through reproducibility. In this regard, the more divergent we make our measurement procedures, the more confirmed are our results. The challenge is then to find a way to connect different models, and to find a common language to interpret different outcomes.