Quantifying the Uncertainty of a Belief Net Response

Reports

Information about Bayesian belief nets in general

• "Improved Mean and Variance Approximations for Belief Net Responses via Network Doubling" Peter Hooper, Yasin Abbasi-Yadkori, Russ Greiner, Bret Hoehn UAI-2009

• UAI'01: Error Bars for Belief Net Inference (see below)

• 2007 Artificial Intelligence: doi:10.1016/j.artint.2007.09.004

• Tech Report (draft version)

If you want to skim:

-Fig 1 (p4) overviews the system -- MeanVar is the main challenge

-Thm 2 (p11-12) gives the actual formal result; the discussion afterwards illustrates some relevant details

-Fig 7 (p23) summarizes the experiment runs to validate this idea

-Fig 8 (p26) shows how close our variance measures are to the "truth"

-Fig 10 and Fig 11 (p28) show why we use the Beta form, rather than Normal

-Appendix A discusses how to implement this FOR COMPLETE QUERIES -- showing straightline code for the computation (see Thm6, p40-41)(Sec 4.2 [p19-21] describes one way to compute the relevant quantities for general queries.)

Applications of "variance"

• Combining different BeliefNet classifiers using variance (Using Query-Specific Variance Estimates to Combine Bayesian Classifiers)

• Using Bias2+Variance to select the "best" BeliefNet classifier (Discriminative Model Selection for Belief Net Structures)

Auxiliary information related to the experiments run

• All 100 Quantile-Quantile Plots

• Data Associated With The Experiments

• Actual Alarm network, Queries, Parameters

Information about Bucket Elimination

• Original Paper (R Dechter)

• Additional Details

• Notes from Cmput 366

Bayesian Error-Bars for Belief Net Inference (ps) (PDF)

Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence (UAI-01), Seattle, August 2001​

A Bayesian Belief Network (BN) is a model of a joint distribution over a finite set of variables, with a DAG structure to represent the immediate dependencies between the variables, and a set of parameters (aka CPTables) to represent the local conditional probabilities of a node, given each assignment to its parents. In many situations, the parameters are themselves treated as random variables --- reflecting the uncertainty remaining after drawing on knowledge of domain experts and/or observing data generated by the network. A distribution over the CPtable parameters induces a distribution for the response the BN will return to any ``What is P(H | E)?'' query. This paper investigates the distribution of this response, shows that it is asymptotically normal, and derives closed-form expressions for its mean and asymptotic variance. We show that this computation has the same complexity as simply computing the (mean value of the) response --- ie, O(n * exp(w)), where n is the number of variables and w is the effective tree width. We also provide empirical evidence showing that the error-bars computed from our estimates are fairly accurate in practice, over a wide range of belief net structures and queries.

Other Possible Ways To Use Variance

To make reliable decisions

• Related to the 2006 KDD Cup​

In the context of active learning:

• Related to the Active Learning With Statistical Methods​