Post date: May 25, 2016 9:11:18 PM
Thanks, Kevin,
Well, I guess they are not too well-known. I asked my mentor on Bayesian stats, Sandy Zabell (prominant Bayesian statistician), about it. Although he agreed with me, he did not really have references stating how "pathological" these frequentists techniques are.
I will tell Sandy about your book. He still teachs stats at NU.
Best,
Rich
On 9/27/2014 1:08 PM, Kevin Murphy wrote:
Yes, these problems are very well known. I am attaching a brief summary ( from my textbook) of some of the most famous "pathologies of frequentist statistics" (cited references can be found in the bibliography here). There are several more pathologies, but I didn't want to go overboard :)
Kevin
PS. A very nice practical book for teaching undergrad stats from a Bayesian POV is this:
@book{Kruschke10,
title = {{Doing Bayesian Data Analysis: A Tutorial Introduction with R and
BUGS}},
author = "J. Kruschke",
year = 2010,
publisher = "Academic Press"
}
On Fri, Sep 26, 2014 at 1:59 PM, Richard E Neapolitan <richard.neapolitan@northwestern.edu> wrote:
Dear Colleagues,
Since I converted to Bayesian statistics in the late 1980's, I have not looked at most frequentist methods. However, every time I look at them again, I notice how apparently preposterous many of them are.
First that was the Bonferroni correction, which makes me update my belief about the results of an experiment based on how many other experiments I happen to conduct with it (and which of course implicitly assigns a low prior probability). One researcher even told me once that he has students first conduct fewer experiments so a finding has a better chance of being significant. I just walked away scratching my head.
Now, in the process of designing a small test for a student, I noticed that two-tailed hypothesis testing is completely unreasonable. Along with the one-tailed test, it gives me decision rules which enable me to reject the hypothesis that the mean is less than or equal to 0, but not reject the hypothesis that it equals 0. The explanation is wrapped up in a story about the question asked and long run behavior with other similar experiments, that are not even run. So two people can walk away from the same experiment with different updated beliefs about whether the mean is 0, not based on their prior beliefs, but based on the question they happened to ask. In general, hypothesis testing does not seem to be the way to go. We should simply compute confidence intervals or posterior probability intervals.
The Bayesian's world is so much simpler. She updates her belief solely on her prior beliefs and the data. There is no story that leads to strange results.
All this matters, especially in medical applications, because so many studies are deemed significant or not significant based on the enigmatic p-value and the Bonferroni correction. I like to say that in medicine for every study there is an equal and opposite study.
I am writing this because I wonder who else has noticed these oddities? I never read about them. I simply observed them independently. I find it curious that they have persisted for so long, and more is not said about them.
Best,
Rich
-- Richard E. Neapolitan, Ph.D., Professor Division of Health and Biomedical Informatics Department of Preventive Medicine Northwestern University Feinberg School of Medicine 750 N. Lake Shore Drive, 11th floor Chicago IL 60611