The following ten problems, involving epistemic uncertainty about data and model structure, are presented as challenges for quantitative risk analysts. Different approaches to modelling epistemic uncertainty such as subjective probability, bounded probability and confidence boxes will produce different solutions to these challenge problems. The thinkover will address various proffered solutions to these problems and consider pedagogical improvements to the challenge problems themselves.
A handful of random samples were collected to estimate the quantity X in units of μg per liter: 12.1, 6.45, 73, 24.6, 15.2, 44.3, 19.0. This variable is believed to be roughly lognormal.
Problem 1.0 What can we say about the chance that X exceeds 100 μg per liter?
Problem 1.1 An expert panel provided quantiles representing their uncertainty about the median of the quantity X as being 8, 10 and 20 corresponding to the 25%, 50% and 75% quantiles. What can we say now about the chance that X exceeds 100?
Problem 1.2 The chemist reported that the detection limit was 20 μg per liter, so samples of X below this value might actually be zeros, and could be as high as the detection limit. What can we say now about the chance that X exceeds 100?
Problem 1.3 The samples of X were not collected randomly, but rather from a ‘hotspot’, which implies they may be overestimates, not representative of the true distribution. What can we say now about the chance that X exceeds 100?
Problem 1.4 Random sample data is available for a separate but commensurate variable Y. The sample values are 2.1, 55, 68, 12, 26, 33, 29, 36, 54, 1.0, 28, 22. What is the chance that X+Y > 100?
Problem 1.5 The last six Y-values were associated with the samples observed for X. What can we say now about the chance that X exceeds 100?
If the chance that a ‘bad thing’ happens in any one trial is p, then the frequency that a bad thing happens in at least one out of N future trials is Q = 1 − (1 − p)N. The event of interest is whether at least one person is infected out of those arriving during a single day, but we are not sure about the frequency p and the number of trials N on any given day. We want to express our uncertainty that Q is greater than 0.05.
The available data about p is that the bad thing happened only once in 153 previous trials. We believe the number of future trials N will be the result of a Poisson process, which in the past has had counts of 12, 0, 21, 14, 6.
Problem 2.0 What is the estimate of Q, and what is our uncertainty that Q is greater than 0.05?
Problem 2.1 There is doubt about whether or not the bad thing actually occurred in the 153 trials. What can be said about Q?
Problem 2.2 There were more sample values previously observed for the Poisson process, but the counts were binned so they are only known as interval ranges. The number N was in the range [0,4] six times, in [5,9] four times, in [10,14] eight times, in [15,19] three times, and once each in [20,24] and [25,29]. (The binned observations were collected before the counts {12, 0, 21, 14, 6} and can be pooled with them.) What can be said about Q?
Problem 2.3 We suspect that the frequency p depends on the number of trials N such that p increases with N. What can be said about Q?