Bayesian Inference

The following is a general setup for a statistical inference problem: There is an unknown quantity that we would like to estimate. We get some data. From the data, we estimate the desired quantity. In the previous chapter, we discussed the frequentist approach to this problem. In that approach, the unknown quantity θ is assumed to be a fixed (non-random) quantity that is to be estimated by the observed data.

In this chapter, we would like to discuss a different framework for inference, namely the Bayesian approach. In the Bayesian framework, we treat the unknown quantity, Θ as a random variable. More specifically, we assume that we have some initial guess about the distribution of Θ. This distribution is called the prior distribution. After observing some data, we update the distribution of Θ(based on the observed data). This step is usually done using Bayes' Rule. That is why this approach is called the Bayesian approach. The details of this approach will be clearer as you go through the chapter. Here, to motivate the Bayesian approach, we will provide two examples of statistical problems that might be solved using the Bayesian approach.