I have a considerable interest in systems of statistical inference, and believe the secret is Calibrated Bayes, where inferences are Bayesian but the model is chosen carefully to yield inferences with good frequency properties. See, for example, Box (1980) and Rubin (1984). I gave a talk on this topic when Fritz Scheuren asked me to give the President’s Invited Address at the Joint Statistical Meetings in 2005. This was later published in Little (2006). A more recent paper along these lines is Little (2011).
Box, GEP (1980), “Sampling and Bayes inference in scientific modelling and robustness” (with discussion), JRSSA 143, 383-430.
Little, R.J.A. (2006). Calibrated Bayes: A Bayes/Frequentist Roadmap. The American Statistician, 60, 3, 213-223. {64}
Little, R.J. (2011). Calibrated Bayes, for Statistics in General, and Missing Data in Particular (with Discussion and Rejoinder). Statistical Science 26, 2, 162-186.
Rubin, DB (1984), “Bayesianly justifiable and relevant frequency calculations for the applied statistician”, Annals of Statistics 12, 1151-1172.
(revised December 7, 2011)