If you are interested in this page, please visit the following paper.
Sumio Watanabe, Mathematical theory of Bayesian statistics for unknown information source. Philosophical Transactions, 2023. https://doi.org/10.1098/rsta.2022.0151
Note: We know that any statistical model is different from the phenomenon from which data are generated. In fact, in statistics, the claim of the following paper, ”All models are wrong”, is well known in all statisticians.
Box, G. E. P. "Science and statistics". Journal of American Statistical Association, Vol. 71,pp. 791–799, 1976.
Note: If a person believes or assumes that unknown uncertainty is not a probability distribution, however makes model and prior using probability distributions, then a person's inference by Bayesian statistics consists of contradictions, resulting that any statistical result is quite weak and unreliable (non-Bayesian statistics, too).
Note: If a person cannot judge scientifically whether unknown uncertainty is a probability distribution or not, then a person had better not choose any decision using statistics, and examine scientific background once again. For example, if an unknown data-generating process is a non-additive measure, it cannot be estimated by either Bayesian or non-Bayesian statistics. In fact, human's natural preference relations cannot be handled by Bayesian statistics, because they do not satisfy Savage's axiom, resulting that you need more general framework than probability theory.
Remark: Modern Bayesian statistics has evolved significantly from the older nonscientific Bayesianism between 1920s and 1950s.
If you are interested in the small and large worlds, the following paper and book are recommended.
(1) Ken Binmore, On the foundations of decision theory. Homo Oeconomicus, Vol.34, pp.259-273, 2017.
In the paper (1), the author says that Savage said the old Bayesianism has no meaning in the large world. In the book (2), it is clearly explained the reason why statistics of a small world is useless in scientific research.
If you are interested in the modern viewpoints about subjectivity and objectivity, the following paper is recommended.
This paper (3) proposes that thinking about statistics in terms of the virtues of context-sensitivity and transparency provides a better perspective for the future, rather than viewing statistics in terms of the old opposition between subjectivity and objectivity.
In the most leading textbook of the modern Bayesian statistics,
(4) "Bayesian Data Analysis 2013" by Gelman et. al., (BDA3),
the authors say (In Chapter 6),
"A good Bayesian analysis, therefore, should include at least some check of the adequacy of the fit of the model to the data and the plausibility of the model for the purposes for which the model will be used. "
The older Bayesianism between 1920s and 1950s, which had the premise that a person must believe a person's own pair of model and prior completely equals unknown uncertainty before sampling, prohibited model checking and comparing. The modern Bayesian statistics has evolved significantly, which is free from the nonscientific Bayesianism philosophy of 1920s -1950s. It should be emphasized that, Bayes and Laplace, who are the original creators of Bayesian statistics before the 20th century, were also free from Bayesianism. Professor Andrew Gelman, who is the greatest modern Bayesian statistician, says "Bayesians are frequentists", repeatedly. Nowadays, we are aware that a pair of model and prior is not a belief but a candidate setting which should be checked by samples and scientific knowledge. Hence the modern Bayesian statistics can be employed in science and engineering based on clear understanding of assumptions set by a scientist and an engineer who are responsible for science and engineering.
Moreover, if you learn singular learning theory, you can understand that Bayesian statistics provides the more precise inference than the maximum likelihood method even asymptotically, when a model contains hidden variables or hierarchical structure. Bayesian statistics becomes more important as a statistical model becomes larger and more complex. For the future study, we had better be aware that the importance of Bayesian statistics originates from not subjective philosophy but mathematical accuracy.
Note: In Bayesian epistemology in Stanford Encyclopedia of philosophy, it is introduced that
"In fact, those results have already appeared in standard textbooks on Bayesian statistics, such as the influential one by Gelman et al. (2014: sec. 4.4 and ch. 6). The line between frequentist and Bayesian statistics is blurring. "
Note: Posterior predictive check (PPC) is a good alternative method for validation of a model and a prior. Gelman-Shalizi (2012) recommends PPC from the viewpoint of falsifiability. Remark that the older Bayesianism cannot allow PPC. Professor Gelman explains that a prior distribution is often not a belief but a part of a model. We need a modern Bayesian statistics where an unknown uncertainty and a statistical model can be distinguished. If textbooks you read recommend PPC, then they are free from the older Bayesianism philosophy.
A. Gelman and C. R. Shalizi, Philosophy and the practice of Bayesian statistics, British Journal of Mathematical and Statistical Psychology, 2012, https://doi.org/10.1111/j.2044-8317.2011.02037.x
In this paper, the authors say ''the most successful forms of Bayesian statistics do not actually support that particular philosophy but rather accord much better with sophisticated forms of hypothetico-deductivism."
If you are interested in the optimization of the hyperparameter, please visit,
S. Watanabe, Higher Order Equivalence of Bayes Cross Validation and WAIC. Springer Proceedings in Mathematics and Statistics, Information Geometry and Its Application, pp.47-73, 2018.
If you are interested in the further mathematics, please visit
Sumio Watanabe, Recent advances in algebraic geometry and Bayesian statistics. Information Geometry,VVol.7, pp.187–209, 2024 https://doi.org/10.1007/s41884-022-00083-9