Inferential Techniques that Use Simulation (UCLA Winter 2014)

Instructor
  • Donatello Telesca
  • Office location: CHS 21-254B
  • Office Hours: by appointment
  • dtelesca at ucla dot edu

  • Lecture/Discussion:       TR   12:00P  -   1:50P   PUB HLT 71.257   
Coursework
  • Take home projects (about 3) each worth 1/3 of your final grade

Approximate Schedule
Lectures
  • (01/06) Review of Likelihood and Bayesian Inference  [Bayesian Core, Ch 1]
  • (01/08) Numerical Approximation of Integrals and Monte Carlo Methods
  • (01/13) An Introduction to the Theory of Markov Chains and MCMC [Handbook of MCMC Ch 1]
  • (01/15) Case Studies using Hastings and Gibbs Sampling [Bayesian Core]
  • (01/20) Parameter Expansions  (PX-DA Algorithms) - [Liu and Wu, 1999]
  • (01/22) Common Design Strategies for Hastings Transitions + Adaptive Sampling   -   Project 1 Due
  • (01/27) Discussion of Project 1 - Introduction to Bayesian Model Comparison
  • (01/29) Model Comparison and Trans-dimensional Problems (RJMCMC) [Handbook of MCMC Ch 3]
  • (02/03) Case Studies using RJMCMC [See related manuscripts]
  • (02/05) Population Monte Carlo and Tempering [Handbook, Ch 11 + references on request]
  • (02/10) Dirichlet Process and Non Parametric Bayesian Inference [See slides]
  • (02/12) Class Cancelled  
  • (02/17) Project 2 Due - Discussion of Project 2
  • (02/19) Hamiltonian Monte Carlo [Handbook of MCMC Ch 5]
  • (02/24) INLA [Rue et al. 2009]
  • (02/26) Variational Bayesian Inference [Omerod and Wand, 2010]
  • (03/03) Dynamic Models 
  • (03/05) Class Cancelled 
  • (03/10) Students Presentation of Project 3
  • (03/12) Students Presentation of Project 3


Reading List

Books:
  • (Recommended) Brooks, Gelman, Jones and Meng. Handbook of Markov Chain Monte Carlo. Chapman and Hall/CRC  
  • (Optional) Robert, CP, Casella, G (2004). Monte Carlo Statistical Methods. New York: Springer.
  • (Optional) Gelman, A, Carlin, JB, Stern, HS, and Rubin, DB (2003). Bayesian Data Analysis, Second Edition. New York: Chapman and Hall.
  • (Optional) Marin, J-M and Robert C (2007). Bayesian Core. New York: Springe-Verlag. (Available online for UCLA students)
  • (Optional) Hoff, P (2009). A First Course in Bayesian Statistical Methods. New York: Springer. (Available online for UCLA students)
  • (Optional) Carlin, BP, Louis, TA (2000). Bayes and Empirical Bayes Methods for Data Analysis, Second Edition. New York: Chapman and Hall.
Manuscripts:
  • Lindley, DV, and Smith, AFM, "Bayes Estimates for the Linear Model". Journal of the Royal Statistical Society. Series B (Methodological), Vol. 34, No. 1. (1972), pp. 1-41. Get it from JSTOR (basics for Bayesian hierarchical models).
  • Tanner, MA, and Wong, WH, "The Calculation of Posterior Distributions by Data Augmentation". Journal of the American Statistical Association, Vol. 82, No. 398. (Jun., 1987), pp. 528-540. Get it from JSTOR (demonstration that two block Gibbs sampling works).
  • Gelfand, AE, and Smith, AFM, "Sampling-Based Approaches to Calculating Marginal Densities". Journal of the American Statistical Association, Vol. 85, No. 410. (Jun., 1990), pp. 398-409. Get it from JSTOR (one of the most cited papers in statistics).
  • Gelfand, AE, Hills, SE, Racine-Poon, A, and Smith, AFM, "Illustration of Bayesian Inference in Normal Data Models Using Gibbs Sampling". Journal of the American Statistical Association, Vol. 85, No. 412. (Dec., 1990), pp. 972-985. Get it from JSTOR (good example list).
  • Tierney L. (1994). Markov Chains for Exploring Posterior Distributions. Annals of Statistics, 22 (4), 1701-1728 (Convergence Results)
  • Liu JS. and Wu YN (1999). Parameter Expansion for Data Augmentation. JASA, 94, 448, pp 1264-1274. 
  • Green P.J. (1995). Reversible jump Markov Chain Monte Carlo Computation and Bayesian Model Determination. Biometrika, 82 (4), pp 711-32.
  • Di Matteo I, et. Al (2001). Bayesian Curve-Fitting Using Free-Knot Splines. Biometrika, 88, pp 1055-1071. (Curve Fitting using RJ-MCMC)
  • Rue H, Martino S and Chopin N. (2009) Approximate Bayesian Inference for Latent Gaussian Models by Using Integrated Nested Laplace Approximations. JRSS-B, 71, 319-392.
  • Omerod JT and Wand MP (2010). Explaining Variational Approximations. The American Statistician, 64, 140-153.
  • More to come! ....

Computing

Computing for Biostat 276 will be based on the R and C programming languages.
It is highly recommended to set up an account on the UCLA Hoffman Cluster (http://www.ats.ucla.edu/clusters/hoffman2/accessing.htm). 
Instructions will be given in class on how to set up your computational platform.