Materials
This is a week-by-week list of course materials. I will post new files and website links as we progress through the course.
Readings for week 2: See files Week 2--MoonStirling_excerpt.pdf and Week 2--ISSO_Sect13-3.pdf below.
Week 3 reading -- Efron (1982) - Week 3 reading: Efron, B. (1982), “Maximum Likelihood and Decision Theory,” Annals of Statistics, vol. 10, pp. 340-356.
Week 3 reading -- Wikibooks on numerical methods - Wikibooks (2022), “Statistics/Numerical Methods/Optimization.”
Week 4 reading -- Redner and Walker (1984): Redner, R. A. and Walker, H. F. (1984), “Mixture Densities, Maximum Likelihood, and the EM Algorithm,” SIAM Review, vol. 26(2), pp. 195−239. http://dx.doi.org/10.1137/1026034
Week 5 reading -- Vaida (2005) - Vaida, F. (2005), “Parameter Convergence for EM and MM Algorithms,” Statistica Sinica, vol. 15, pp. 831–840.
Week 5 reading -- Wu (1983) - Wu, C. F. J. (1983), “On the Convergence Properties of the EM Algorithm,” Annals of Statistics, vol. 11, pp. 95–103.
Week 6 readings: Spall (1988) (historical context), Rauch et al. (1965) (filter and smoother), Simon (2006) (excerpt Chaps. 5 and 6; class focus in on "basic" form in Chap. 5), and Terejanu (2008) (extended Kalman filter). These readings are posted below. Priority of reading for these for documents is given on the Homework page of this website.
Week 7 reading - Gupta, N. and Mehra, R. (1974), “Computational Aspects of Maximum Likelihood Estimation and Reduction in Sensitivity Function Calculations,” IEEE Transactions on Automatic Control, vol. AC-19(6), pp. 774–783.
Week 7 reading--Gibson and Ninness - Gibson, S. and Ninness, B. (2005), “Robust Maximum-Likelihood Estimation of Multivariable Dynamic Systems,” Automatica, vol. 41(10), pp. 1667–1682.
Week 7 reading--Goodrich and Caines - Goodrich, R. L. and Caines, P. E. (1979), “Linear System Identification from Nonstationary Cross-Sectional Data,” IEEE Transactions on Automatic Control, vol. 24, pp. 403–411.
Week 8 reading--Spall (2014) - Spall, J. C. (2014), “Identification for Systems with Binary Subsystems,” IEEE Transactions on Automatic Control, vol. 59(1), pp. 3–17.
Week 8 reading--Wang and Spall (2017) - Wang, L. and Spall, J. C. (2017), “Beyond the Identification of Reliability for System with Binary Subsystems,” Proceedings of the American Control Conference, Seattle, WA, 24–26 May 2017, pp. 158–163.
Week 9 reading--Ljung (1977): Ljung, L. (1977), “Analysis of Recursive Stochastic Algorithms,” IEEE Transactions on Automatic Control, vol. AC-22, pp. 551–575.
Week 10 readings: Primary: Schwarz, G. (1978), “Estimating the Dimension of a Model,” Annals of Statistics, vol. 6, pp. 461–464. Secondary: Chap. 16 of Ljung, L. (1999), System Identification—Theory for the User (2nd ed.), Prentice Hall PTR, Upper Saddle River, NJ (see file posted below).
Week 11 readings: Primary: Sjöberg, J. and Ljung, L. (1995), “Overtraining, Regularization and Searching for a Minimum, with Application to Neural Networks,” International Journal of Control, vol. 62(6), pp. 1391–1407, and LeCun, Y., Bengio, Y., and Hinton, G. (2015), “Deep Learning,” Nature, 521(7553), 436–444. Secondary: Raper, S. (2020), “Leo Breiman's ‘Two Cultures’,” Significance, vol. 17, pp. 34–37.
Week 12 readings: Primary: Pillonetto G., Aravkin A., Gedon D., Ljung L., Ribeiro A.H., Schön T.B. (2023), “Deep Networks for System Identification: A Survey,” arXiv preprint arXiv:2301.12832, 30 Jan. 2023. Secondary: Adlam, B. and Pennington, J. (2020), “Understanding Double Descent Requires A Fine-Grained Bias-Variance Decomposition,” Advances in Neural Information Processing Systems, vol. 33, pp. 11022–11032; Hastie, T., Montanari, A., Rosset, S., and Tibshirani, R. J. (2022), “Surprises in High-Dimensional Ridgeless Least Squares Interpolation,” Annals of Statistics, vol. 50(2), pp. 949–986.
Week 13 readings: Primary (2 papers): Spall, J. C. (2010), “Factorial Design for Choosing Input Values in Experimentation: Generating Informative Data for System Identification,” IEEE Control Systems Magazine, vol. 30(5), pp. 38–53. http://dx.doi.org/10.1109/MCS.2010.937677
Pronzato, L. (2008), “Optimal Experimental Design and Some Related Control Problems,” Automatica, vol. 44(2), pp. 303–325. http://dx.doi.org/10.1016/j.automatica.2007.05.016
Presentations_roundtable_systID_handout.pdfWeek 1--Presentation guidelines and assignment
SysID_Likelihood_class1_handout.pdfWeek 1--Intro. to system identification
Mardia_etal_excerpt_estimation_1979.pdfWeek 1--Mardia_etal_excerpt_estimation_ 1979.pdf
MoonStirling_excerpt.pdfWeek 2--MoonStirling_excerpt.pdf
ISSO_Sect13-3.pdfWeek 2--ISSO_Sect13-3.pdf
Spall_BayesAnalTimeSerDynModel88_KalmanIntro.pdfWeek 6--Spall (1988)
Simon_2006_Chaps5-6.pdfWeek 6--Simon (2006)
Rauch_Tung_Striebel_AIAAJ_1965.pdfWeek 6--Rauch et al. (1965)
Terejanu_tutorialEKF_2008.pdfWeek 6--Terejanu (2008)
Ljung_SystID_Chap16_ModelSelection.pdfWeek 10--Ljung (1999)--Chap. 16