This course will provide a comprehensive introduction to the fundamentals of learning dynamical systems. The primary objectives of this course are twofold: to estimate unknown physical quantities and to develop models for time-series prediction, such as forecasting the yearly energy consumption of buildings. The theoretical foundation of this course lies at the intersection of system identification (a sub-discipline of control theory), econometrics, statistics, and machine learning. We will explore autoregressive models, state-space models, parameter estimation, and learning predictive black-box models.
The course is divided into two parts. The first part consists of 18 hours of introduction to the theoretical tools associated withh exercises. The second part, 6 hours, will focus on a complete case study.
We will begin by delving into autoregressive models for time-series analysis, followed by an exploration of state-space models. Regarding autoregressive models, we will start with classical linear models (ARX, ARMA) before progressing to models inspired by machine learning, such as neural networks, SVM, decision trees, and transformers In the context of state-space models, we will initially focus on linear state-space models and the associated algorithms like the Kalman filter and subspace methods. Furthermore, we will explore recurrent neural networks.
02/04/2024: basics of machine learning, linear regression, ARX models.
Handwritten notes: part1, part2 , part3, part4 ,part5, part6
Python code: kaggle link
15/04/2024: Continuation on ARX models and non-linear regression using neural networks: kaggle notebook , handwritten notes:
22/04/2024: ARMAX models: kaggle notebook https://www.kaggle.com/code/mihalypetreczky/notebookaafbf777df,
Linear state-space models: state-space identification using subspace method in Python, using SLIPPY package: file
Handwritten notes: part1, part2, part3 , part4, part5
24/04/2024:
State space models/ARMAX models using Matlab System Identification Toolbox: example: Matlab file.
RNNs using Keras: kaggle notebook: notebook
CDC player data from DAISY data set, csv file: HW use this data set for forecasting
Notes on seasonality: Part 1, Part 2
Recommended literature:
ARX,ARMAX models, seasonality, detrending:
Introductory Econometrics: A Modern Approach, Jeffrey M. Wooldridge, ISE - International Student Edition, 2008.
Introduction to Time Series and Forecasting. A Brockwell, P.J. and Davis, R.A. 2006, Springer New York
System Identification: Theory for the User, Lennart Ljung, Prentice Hall, 1999.
Linear state-space models:
System Identification: Theory for the User, Lennart Ljung, Prentice Hall, 1999.
Introduction to Time Series and Forecasting. A Brockwell, P.J. and Davis, R.A. 2006, Springer New York
Machine Learning:
T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Prediction, Inference and Data Mining,
Second Edition, Springer Verlag, 2009.
Understanding Machine Learning: From Theory to Algorithms, Shai Shalev-Shwartz Shai Ben-David Cambridge University Press, 2014
Deep Neural Networks in a Mathematical Framework. Anthony L. Caterini, Dong Eui Chang, 2018.
In the second part, students will have the opportunity to apply the techniques learned in Part I to a specific use case. The use case will be related to energy consumption optimization and forecasting of public buildings (like the one of University of Lille). The objective of the work will be to predict energy consumption and the building GhG (green house gaz) emissions based on uncontrolled parameters (eg: weather conditions) as well as controlled one (eg: opening hours).
Zoom link: link