Introduction

Quick Bayes:  Computing with Confidence

Quick Bayes is a way to estimate parameters in probabilistic models when the available empirical information about those parameters and the output from the model are scarce or imprecise. Quick Bayes is a kind of robust Bayes analysis (Berger 1994), so it can be thought of as a kind of automated Bayesian sensitivity analysis which accounts for uncertainty about the prior, the likelihood or both.  But Quick Bayes is especially convenient to use because the analyst need not choose a prior distribution when no experience, expectation or evidence justifies the choice. Quick Bayes doesn’t force analysts to specify noninformative priors when they can't give informative ones.  Quick Bayes uses structures called “confidence boxes” or “c-boxes” to capture inferential uncertainty which contains both epistemic and aleatory uncertainty. The confidence boxes can be projected through calculations in a way that preserves the distinction between these two kinds of uncertainty. The structures are closely related to the Imprecise Beta Models and their generalisation Imprecise Dirichlet Models (Walley 1996).

Quick Bayes also has coverage properties that make it be interpretable to frequentists who want to compute traditional confidence intervals about input parameters and output values. Unlike a conventional Bayesian analysis, Quick Bayes yields estimates that have the property that they always encode Neyman confidence. Remarkably, the calculations derived from these estimates can also have his property. A Quick Bayes c-box tells you confidence intervals for a parameter or calculated quantity at any confidence level you like.  For instance, the confidence box depicted below yields several confidence intervals for the parameter θ. Although you can't generally compute with traditional frequentist confidence intervals, you can compute with Quick Bayes confidence boxes, and you can get arbitrary confidence intervals for the results.

Quick Bayes c-boxes can be computed in a variety of ways directly from random sample data.  There are confidence boxes for both parametric problems where the family of the underlying distribution from which the data were randomly generated is known (including normal, lognormal, exponential, binomial, Poisson, etc.), and nonparametric problems in which the shape of the underlying distribution is unknown.  Confidence boxes account for the uncertainty about a parameter that comes from the inference from observations, including the effect of small sample size, but also the effects of imprecision in the data and demographic uncertainty which arises from trying to characterize a continuous parameter from discrete data observations.

When confidence boxes have the form of probability boxes, they can be propagated through mathematical expressions using the ordinary machinery of probability bounds analysis, and this allows analysts to compute with confidence, both figuratively and literally, because the results also have this confidence interpretation.

This website is a portal to several papers and presentations about confidence boxes, including

• 1. Slide presentations and posters,

  2. Introductory paper on c-boxes with R functions and a comparison with the Imprecise Beta Model,

  3. Paper with several c-box formulas and a comparison between c-box results and analogous Bayesian and maximum likelihood results,

  4. Application of c-boxes in common inference problems arising in risk analysis,

  5. Application of the c-box for nonparametric difference in a calibration/validation study,

  6. Simulations illustrating the use and performance of c-boxes in estimating distributions and their parameters,

  7. Original paper on confidence structures, and

  8. Review paper on confidence distributions; another one.

Confidence boxes are imprecise generalizations of traditional confidence distributions.  Like Student's t distribution, they encode frequentist confidence intervals for parameters of interest at every confidence level.  They also generalize traditional Bayesian posterior distributions in that they characterize the inferential uncertainty about distribution parameters estimated from sparse or imprecise sample data, but they also have a frequentist interpretation that makes them useful in engineering because they offer a guarantee of statistical performance through repeated use.  Unlike traditional confidence intervals which cannot usually be propagated through mathematical calculations, c-boxes can be used in calculations to yield results that also admit the same confidence interpretation.  For instance, they can be used to compute probability boxes for both prediction and tolerance distributions.  They are easy to construct and use in calculations; see the software page for R functions to construct several c-boxes.

Note that c-boxes are completely different from confidence bands such as the Kolmogorov-Smirnov distributional bands which are nonparametric confidence limits at some particular confidence level for the distribution from which sample data were randomly drawn.  C-boxes encode confidence intervals at all possible confidence levels at the same time.

References

Berger, J.O. (1994). An overview of robust Bayesian analysis (with discussion). Test 3: 5-124.

Walley, P. (1996). Inferences from multinomial data: learning about a bag of marbles. Journal of the Royal Statistical Society: Series B (Methodological) 58: 3-34.