Monte Carlo and probability bounds analysis
with hardly any data using R or Python
Monte Carlo and probability bounds analysis
with hardly any data using R or Python
A tutorial workshop
held in conjunction with the virtual
Society for Risk Analysis Annual Meeting
8:30 am - 5:30 pm ET
Thursday, 8 December 2022
This tutorial shows you how you can develop fully probabilistic risk assessments even though there may be very little empirical data available on which to base the analysis. It compares the strengths and weakness of a traditional Monte Carlo assessment with probability bounds analysis. You can easily do the calculations yourself using high-level tools that run in both Python and R (which are freely available over the web). The tutorial will set you up.
This full-day workshop features hands-on examples worked in R on your own laptop, from raw data to final decision. The workshop introduces and compares Monte Carlo simulation and probability bounds analysis for developing probabilistic risk analyses when little or no empirical data are available. You can use your laptop to work the examples, or just follow along if you prefer. The examples illustrate the basic problems risk analysts face: not having much data to estimate inputs, not knowing the distribution shapes, not knowing their correlations, and not even being sure about the model form. Monte Carlo models will be parameterized using the method of matching moments and other common strategies. Probability bounds will be developed from both large and small data sets, from data with non-negligible measurement uncertainty, and from published summaries that lack data altogether. The workshop explains how to avoid common pitfalls in risk analyses, including the multiple instantiation problem, unjustified independence assumptions, repeated variable problem, and what to do when there’s little or no data. The numerical examples will be developed into fully probabilistic estimates useful for quantitative decisions and other risk-informed planning. Emphasis will be placed on the interpretation of results and on how defensible decisions can be made even when little information is available. The presentation style will be casual and interactive. Participants will receive handouts of the slides and on-line access to the presentation slides and data sets for the examples.
Welcome
Case studies: civil and aerospace engineering, exposure analysis, and conservation biology
Installation of workshop software for R and Python
Random values and replications
Distributions
Independent and perfect sampling
Calculations in R and Python
Interpreting results: tails are where the action is
Kinds of uncertainty: the ‘open question’
Probability boxes
Independent, perfect, and Fréchet
Calculation in R and Python
Interpreting results: fully probabilistic answers
Integrating Monte Carlo and probability bounding
Fixed but unknown, or actually varying?
Distributions, p-boxes, and interval ranges
What you know and what you assume
Moments and ranges
Random sample data
Maximum likelihood and maximum entropy
Confidence boxes
Shape assumptions to refine estimates
Making no assumptions about dependence
Perfect correlations
Dispersive Monte Carlo dependence
Independence maximizes entropy
Civil engineering: dam safety
Aerospace engineering: spacecraft design
Environmental protection: contaminant exposure analysis
Conservation biology: estimating endangerment
What-if studies
Stochastic mixtures and Bayes model averaging
Bounding methods
Conservative methods for polynomial models
Bang-for-buck control analysis
Value of information: what data to collect
More samples or better measurements
Scott Ferson, Chair of Risk and Uncertainty at the University of Liverpool School of Engineering and director of the Institute for Risk and Uncertainty; developing reliable mathematical and statistical tools for risk assessments and on methods for uncertainty analysis when empirical information is very sparse
Nick Gray, Ph.D. (2023), University of Liverpool, investigating the importance of risk and uncertainty for ethical machine learning and humane artificial intelligence from a background in theoretical and mathematical physics
Alexander Wimbush, Ph.D. (2023), University of Liverpool, optimising medical diagnostic algorithms under uncertainty, communicating risks to patients, and calculation with confidence structures and possibility distributions
You can register for the workshop by going to https://members.sra.org/ev_calendar_day.asp?date=12/4/2022&eventid=30 and clicking on a large red registration button (for either SRA members or non-members). On the registration page, select the "Sessions" tab and check the workshop option 12 on Thursday at the bottom of the page. The regular workshop registration fee this year is $400, but you do not need to register for the annual meeting to attend the workshop. Student are eligible to attend the workshop for a subsidized student rate of only $35. [The registration pages are being updated to allow registering at the subsidized student rate or without attending the annual meeting.] For problems with on-line registration, consult registration@sra.org. For other questions concerning the SRA annual meeting, consult the SRA Secretariat by email at meetings@sra.org (telephone +1-703-790-1745).
If you need a hotel room, you can book it through SRA's link at https://book.passkey.com/gt/218443989?gtid=c95dbe0cce6c42f4f2c78b960a038150 to get a discounted rate (starting at $169) at the Tampa Waterside Marriott.
This event is workshop 12 at https://www.sra.org/events-webinars/annual-meeting/program/annual-meeting-workshops/, where you can see the full list of other workshop events this year.
Society for Risk Analysis www.sra.org
Society for Risk Analysis Annual Meeting https://www.sra.org/events-webinars/annual-meeting/
On-line workshop registration https://members.sra.org/ev_calendar_day.asp?date=12/4/2022&eventid=30 (click a red button, select "Sessions" tab, check workshop 12)
Society for Imprecise Probabilities http://www.sipta.org/
Liverpool Institute for Risk and Uncertainty https://riskinstitute.uk https://www.liverpool.ac.uk/risk-and-uncertainty/
Sandia National Laboratories' Epistemic Uncertainty Project https://sites.google.com/site/uncertaintyprojection/
Other uncertainty projects https://sites.google.com/site/abuncertainty/
NSF workshop on risk perception and communication https://sites.google.com/site/montaukriskcommunication/
https://drive.google.com/drive/folders/1VunIJa8vhl9P6Xko7YR8yyzXRW9Twykb?usp=sharing
Intervals and Probability Distributions website http://ualr.edu/jdberleant/intprob/
https://drive.google.com/drive/folders/1GcDaWNGdXAsGeA5Plrdw_hg5SL-IR0Ci?usp=sharing