In robust Bayes analysis, there is uncertainty about the prior, or the likelihood, or both. The three graphs below each represent a robust Bayes analysis. The blue curves represent priors; the red curves represent likelihood functions; and the gray curves represent the resulting posterior distributions. In the left graph, there are several blue prior distributions. Each of these priors is combined with the single red likelihood to give several gray posteriors. This is the problem that Basu (1994) addresses. In the middle graph, there is only a single prior, but there are many likelihoods, which could have arisen from incertitude in the observed data. In the graph on the right, there are both many priors and many likelihoods. This of course is the general problem.
We hope to use our p-box technology to facilitate routine robust Bayes analyses. For example, for these inputs
Prior: normal(μ, σ) Likelihood: normal(x, s)
μ = [1, 6] x = [5,13]
σ = [2.5, 6] s = [4, 5]
we can derive from Bayes’ rule
Posterior: normal(θ, v)
q = (μ/σ2 + x/s2)/(1/σ2 + 1/s2) = [2.28, 10.35]
v = 1/Ö(1/σ2 + 1/s2) = [1.24, 1.66].
Conjugacy greatly simplifies the needed calculations, although, in this case, σ2 and s2 are repeated in the formula for the mean q so care must be exercised in evaluating it. There are many such formulas available, including
Sampling process Prior Posterior
bernoulli(q) uniform(0, 1) beta(1+Sxi,1+nS-xi)
bernoulli(q) beta(a, b) beta(a+Sxi, b+nS-xi)
poisson(q) gamma(a, b) gamma(a+Sxi,b+n)
normal(q, s) normal(m, s) normal((ms2+s2Sxi)/(s2+ns2),1/Ö(1/s2+n/s2))
exponential(q) gamma(a, b) gamma(a+n,b+Sxi)
binomial(k, q) beta(a, b) beta(k+a,n-k+b-1)
uniform(0, q) pareto(b, c) pareto(n+b,max(max(data),c))
<<Masatoshi says to Scott on 2012AUG6: I assumed that the pareto(b,c) has a form, b*c^b*q^(-b+1)*Indicator(c<=q)>>
negbinomial(r,q) beta(a, b) beta(k+a,r+b)
<<Masatoshi says to Scott on 2012AUG6: r is number of failures until the k th success is observed.>>
normal(q, s) inversegamma(a, b) inversegamma(n/2+a,1/2/s^2*sum((xi-q)^2)+b)
<<Masatoshi says to Scott on 2012AUG6: I changed the gamma to inversegamma.>>
exponential(q) invgamma(a, b) inversegamma(a+n,b+Sxi)
<<Masatoshi says to Scott on 2012AUG6: I am assuming that the exponential likelihood has a form, (1/q)*exp(-1/q*x), and the inverse gamma has a form q^(-a-1)*exp(-b/q).>>
multinomial(k,qj) dirichlet(s, tj) dirichlet(n+s,(xj+stj)/(n+s))
These sorts of formulas can readily be implemented in Risk Calc or deployed on other platforms. We need to complete the above list of conjugacy relationships and fill in the full formulas where they are missing. It is surprising to me that these conjugacy rules are not all on Wikipedia. (Are they perhaps there now?) Am I missing something? Would the envisioned software library be a violation of some Bayesian precept? Am I misunderstanding something fundamental?
Both the prior and the posterior are distributions, so uncertainty about them can be naturally represented by p-boxes using our existing technology. The likelihood is another matter. For one thing, the likelihood is usually incarnated as sample data, rather than some function or set of functions. In many cases the sample data will consist of intervals rather than point values. Such data should be easy to wrangle in Risk Calc which has features to represent and manipulate intervals. Are there other reasons, besides incertitude in sample data, that might cause the likelihood to be uncertain? If so, how could we capture and represent that uncertainty in Risk Calc? The likelihood function is determined by the sampling model, but the two are not the same thing. For instance, the likelihood function that is determined by a binomial sampling process has the shape of a beta distribution. In fact, any likelihood function can be rescaled to have unit area and could be encoded as its integral, so would it therefore make sense to represent a likelihood with p-box? This might be possible in principle, but it is not clear whether it would be useful or even workable in practice.
We need to design the software library and its features. Should we imagine a Risk Calc function called “bayes” that accepts a prior p-box, a list of interval sample data together with a (named) sampling process, and spits out a posterior p-box? Could we also support a likelihood function specified or bounded in some other way? Can we implement empirical likelihood (Owen 2001) in the software library? Can those methods be extended to interval sample data? What additional features, if any, would we have to add to Risk Calc to support empirical likelihood? Can the methods of Basu (see Implement Basu's methods) be incorporated into the software library in a natural way? What should happen if the user supplies a prior and a likelihood for which there is no conjugacy rule known? It would be nice if some useful default answer could be given (see Escaping triviality in Bayes’ rule with p-boxes).