cite: Heathers, J. A. & Meyerowitz-Katz, G. (2024). ‘Yes, but how much smaller?’ A simple observation about p-values in academic error detection. OSF preprint.
STALT stands for Smaller Than A Lowest Threshold
This one is so simple. It shouldn't really be a test! But it's my life, so there you go. Either way, we've found more problems with it in a month than we did with the other tests in a year.
A lot of papers report the outcome of statistical tests as 'p<.05', or 'p<.001' -- so the question you should always ask is 'oh, it's smaller is it, so HOW MUCH SMALLER?'
Would it change your mind if the p value 'smaller than .05' was actually 0.0000000005? You'd ask questions. So, you simply back-calculate the relevant statistical test where possible.
Some additional fun points:
STALT is just back-calculation in context. It's very very easy to understand.
STALT reveals something I've always said, which is p-values are annoying.