Embedded Files
I’ve published several articles on issues connected with the replication crisis in science. For example, I’ve argued that it's not always problematic to engage in questionable research practices (Rubin, 2025a) such as hypothesising after the results are known (HARKing; Rubin, 2017c, 2022) and uncorrected multiple testing (Rubin, 2017b, 2021a, 2021c, 2022a, 2024a, 2024c). I've also criticised some science reforms, such as preregistration (e.g., Rubin, 2020a, 2025c; Rubin & Donkin, 2024) and stricter adherence to Neyman and Pearson's approach to statistical hypothesis testing (Rubin, 2020b, 2021b), and I've written about what I call questionable metascience practices (Rubin, 2023). Nonetheless, I'm generally in favour of some other open science reforms, such as open access articles, preprints, and open data and materials in postpositivist science.
Summaries
Summaries
Publications
Publications
Rubin, M. (2025a). A brief review of research that questions the impact of questionable research practices. PsyArXiv.
Rubin, M. (2025b). What is critical metascience and why is it important? PsyArXiv.
Rubin, M. (2025c). Preregistration does not improve the transparent evaluation of severity in Popper’s philosophy of science or when deviations are allowed. Synthese, 206, Article 111.
Publisher's open access version.
Rubin, M. (2025d). The replication crisis is less of a “crisis” in Lakatos’ philosophy of science than it is in Popper’s. European Journal for Philosophy of Science, 15(5).
Publisher's open access version.
Rubin, M. (2024a). Inconsistent multiple testing corrections: The fallacy of using family-based error rates to make inferences about individual hypotheses. Methods in Psychology,10, Article 100140.
Publisher's open access version.
Rubin, M. (2024c). Type I error rates are not usually inflated. Journal of Trial and Error, 4(2), 46-71.
Publisher's open access version.
Rubin, M., & Donkin, C. (2024). Exploratory hypothesis tests can be more compelling than confirmatory hypothesis tests. Philosophical Psychology, 37(8), 2019-2047.
Publisher’s open access version.
Rubin, M. (2023). Questionable metascience practices. Journal of Trial and Error, 4(1), 5-20.
Publisher’s open access version.
Rubin, M. (2022a). That’s not a two-sided test! It’s two one-sided tests! Significance, 19(2), 50-53.
Publisher’s version. Self-archived version.
Rubin, M. (2022b). The costs of HARKing. The British Journal for the Philosophy of Science, 73(2), 535-560.
Publisher’s version. Publisher’s free access. Self-archived version.
Rubin, M. (2021a). There’s no need to lower the significance threshold when conducting single tests of multiple individual hypotheses. Academia Letters, Article 610.
Publisher’s open access version. Direct open access version.
Rubin, M. (2021b). What type of Type I error? Contrasting the Neyman-Pearson and Fisherian approaches in the context of exact and direct replications. Synthese, 198, 5809–5834.
Publisher’s version. Self-archived version.
Rubin, M. (2021c). When to adjust alpha during multiple testing: A consideration of disjunction, conjunction, and individual testing. Synthese, 199, 10969–11000.
Publisher’s version. Self-archived version.
Rubin, M. (2020a). Does preregistration improve the credibility of research findings? The Quantitative Methods in Psychology, 16(4), 376–390.
Publisher’s version. Self-archived version.
Rubin, M. (2020b). “Repeated sampling from the same population?” A critique of Neyman and Pearson’s responses to Fisher. European Journal for Philosophy of Science, 10, Article 42, 1-15.
Publisher’s version. Publisher's open access version. Self-archived version.
Rubin, M. (2017a). An evaluation of four solutions to the forking paths problem: Adjusted alpha, preregistration, sensitivity analyses, and abandoning the Neyman-Pearson approach. Review of General Psychology, 21, 321-329.
Publisher’s version. Self-archived version.
Rubin, M. (2017b). Do p values lose their meaning in exploratory analyses? It depends how you define the familywise error rate. Review of General Psychology, 21, 269-275.
Publisher’s version. Self-archived version.
Rubin, M. (2017c). When does HARKing hurt? Identifying when different types of undisclosed post hoc hypothesizing harm scientific progress. Review of General Psychology, 21, 308-320.
Comments and Responses
Comments and Responses
Rubin, M. (2024). Green jelly beans and studywise error rates: A “theory first” response to Goeman (2022). PsyArXiv.
Open access self-published version.
Rubin, M. (2023, June 7). The preregistration prescriptiveness trade-off and unknown unknowns in science: Comments on Van Drimmelen (2023). Critical Metascience: MetaArXiv.
Open access self-published version.
Rubin, M. (2023, May 4). Opening up open science to epistemic pluralism: Comment on Bazzoli (2022) and some additional thoughts. Critical Metascience.
Open access self-published version.
Rubin, M. (2017). The implications of significance testing based on hypothesiswise and studywise error. PsyArXiv.
Presentations
Presentations
Rubin, M. (2025, June). What is critical metascience and why is it important? In S. Ulpts & S. Bartscherer (Convenors), Critical metascience: Does metascience need to change? Metascience 2025 Preconference Virtual Symposium.
Video Recording. Slides. Blog Post.
Rubin, M. (2024, July). Questioning some metascience assumptions. Western Sydney University, Australia.
Rubin, M. (2022). Does preregistration improve the interpretability and credibility of research findings? In Research transparency: From preregistration to open access. Erasmus Research Institute of Management Research Transparency Campaign, Erasmus University Rotterdam.
"Deep Dive" Podcast
"Deep Dive" Podcast
Listen to a 22-minute AI-generated podcast summarizing my metascience work here.
Google Sites
Report abuse