Process Identification and Control - 2025

News: 

Page under construction. The course will start in late September. This year, the course will be taught entirely by Francesco Liberati.

Logistic

• Office hours: Thursdays, 11:30 to 13:00, in presence or online (meet.google.com/zob-fhvk-bsz). Please send me an email to book an appointment.  You are encouraged to come to the office hours, to ask any clarification.

• Lessons schedule: 

  • To be defined.

Program

• Introduction to the course: process modelling, identification and control

• Process identification:

  • Problem definition

  • Traditional methods

  • Advanced methods

• Process control with Model Predictive Control (MPC):

  • MPC formulation: objective function (stage cost, terminal cost), constraints, terminal constraints

  • Analysis of the main properties of MPC: recursive feasibility, optimality, stability.

  • Economic MPC.;

  • MPC problems with mixed-integer variables and dynamical-locigal constraints. Examples.

  • Examples of applications in various sectors: task scheduling in industrial assembly lines, smart charging of electric vehicles, smart grids, traffic control, etc.

  • New: Reinforcement learning (RL) and hybrid MPC-RL algorithms.

  • Implementation of simple MPC algorithms using the Julia technical language (https://julialang.org/). Learn how to practically write and solve an MPC problem.

Exams and Grading

• Exam modality: Written exam with exercises and questions. There is ONE exam, with exercises and questions on the two modules. For Module 2, there will be questions on theory, plus excercises on model predictive control formulation. THE EXAM MUST BE BOOKED ON INFOSTUD.

• Exam dates (check at this link and at this link):

• 28-gen. Room 5 San Pietro in Vincoli, hour 09:00-13:00.

• 19-feb. Room 5 San Pietro in Vincoli, hour 09:00-13:00.

• June 5 2025 - Room 9 (San Pietro in Vincoli) - 09:30 .

• ven 4/7/25, Aula 5 SPV, 08:00-12:00.

• ven 5/9/25, Aula 5 SPV, 08:00-12:00.

Lecture Summaries

For every lesson, a summary of the content will be available below, with the precise indication of the related study materials. Note: you can take benefit also of the materials of the past editions of the course, available on my website.

Lesson 1 (03/10/2024): Introduction to module 2 content

We introduced module 2 of the course, which focuses on Model Predictive Control. We discuss the course logistics, office hours, study matherials, the exam, etc. We discussed about the high-level idea of MPC. General working idea, pros and cons. Nominal MPC, rodust and atochastic MPC, hybrid MPC based on mixed-integer programming, to take into account both physical and logical constraints, and finally MPC mixed with deep reinforcement learning (DRL), to improve the model or to tune the parameters of MPC.

• Read chapter 1 of book [Cannon_2016]. See below.

Lesson 2 (04/10/2024): Introduction to MPC. Nomenclature and a simple example on how to formulate an MPC problem.

We introduced nominal MPC, discussing some notation formalism (the use of two time indices k and i), and providing a simple example of MPC control of a water tank. 

• Blackboard notes at this link.

Lesson 3 (07/10/2024): General MPC formulation. Optimal control.

We introduced an abstract and general formulation of the optimization problem at the base of MPC, discussing all its main ingredients (stage cost, terminal cost, model, state and input constraints, terminal constraint, etc.). We introduced the idea of optimal control and provided the corresponding mathematical formulation. We then discussed about Linear Quadratic Regulation (LQR), a particularly simple optimal control problem that can be solved analytically.  

• Blackboard notes at these links: part 1 and part 2.

• Read sections 2.1 and 2.2 of book [Cannon_2016]. Skip proofs.

Lesson 4 (17/10/2024): Dual Mode Control: the relation between infinite horizon constrained LQR and MPC. Online Lesson.

We discussed about a practical way of solving a constrained infinite horizon linear quadratic regulation problem, by dividing the infinite horizon into two regions, one from 0 to N, and the second from N to infinity. We saw that, if N is large enough, the optimal solution in region 2 is equal to the unconstrained optimal solution (u=Kx). This allows to transform the infinite horizon constrained problem into a finite horizon one, which can be solved in practice by the computer, and which has the same solution. We will see how to exploit this with MPC.  

• Blackboard notes at this link.

• Zoom recordings at this link.

• Read section 2.3 of book [Cannon_2016]. Skip proofs.

Lesson 5 (18/10/2024): Dual Mode Control, conclusions.

Conclusions and final considerations on dual mode control. The role of value functions.  

• Blackboard notes at this link.

• Read section 2.3 of book [Cannon_2016]. Skip proofs.

Lesson 6 (21/10/2024): Exercise: Fast charging control of plug-in electric vehicles (PEVs) in a service area.

We discussed one simple example of MPC control, focusing on the problem of ensuring fast charging to PEVs in a service area (the equivalent of petrol stations). We presented the problem, the control requirements, and then we wrote the mathematical formulation and finally we implemented the algorithm using  The Julia Programming Language..

• Blackboard notes at this link: 20241021 - Lesson 6.pdf . The notes contain the mathematical formulation of the problem.

• The code is in this folder: 20241021 - SERVICE AREA CONTROL . Run the file "main.jl". You can use either Gurobi or Cbc solver. If you use Gurobi, you should first install it (download the Gurobi optimizer from here Gurobi Software - Gurobi Optimization - login with the Sapienza email), then get the academic licence (Gurobi Academic License Program: Free Licenses for Academics | Gurobi - Gurobi Optimization ), download the licence on your computer, and then install the Gurobi.jl Julia package (Pkg.add("Gurobi") - not this should be the last step). If instead you use Cbc, you only have to install it (Pkg.add("Cbc")) and load it in the program.

• Zoom recording of the code implementation.

Lesson 7 (31/10/2024): Recursive feasibility.

We defined recursive feasibility of MPC, presenting one common way to prove it. Central to it, is the idea of Controlled Positive Invariance (CPI).  

• Blackboard notes at this link.

Lesson 8 (4/11/2024): Stability of MPC.

We discussed about stability of MPC. We defined the concept of equilibrium point, and that of stability, asymptotic stability and unstability of an equilibrium. We introduced the Lyapunov theorem, which provides a sufficient condition to check the stability of the equilibrium of a nonlinear system. We saw how stability of regulation MPC can be proved by suding the optimal cost as a candidate Lyapunov function.

• Blackboard notes at this link (part 1) and this link (part 2).

• Read [Mayne_2000] Sections 1.1, 1.2, 3.1, 3.2, 3.3  - see below for link to the paper.

Lesson 9 (14/11/2024): Recap of stability of MPC + Introduction to the example on MPC for PEV recharging

We did a quick recap of MPC stability concepts. Then we moved to the introduction to the example of MPC use to control the recharging process of a set of PEVs.

• Blackboard notes at this link (part 1) and this link (part 2).

Lesson 10 (15/11/2024): MPC for PEV recharging

We continued the example, writing all the constraints.

• Blackboard notes at this link.

Lesson 11 (18/11/2024): MPC for PEV recharging + MPC for building control

We finished the example on MPC for PEV recharging and we introduced and finished a new example on MPC for building control.

• Blackboard notes at this link (part 1) and this link (part 2).

Lesson 12 (21/11/2024): MPC for task scheduling in assembly lines

We started to write an MPC algorithm for controlling the execution of tasks in an assembly line.

• Saving of Blackboard notes from the blackboard failed, I have to retreive them again.

• Study Section II of the paper at this link. Password sent via Google group.

Lesson 13 (28/11/2024): MPC for task scheduling in assembly lines

We started the lesson by writing the complete formulation of the problem of building control with MPC (Lesson 1). Then we continued with the example on task control. We will finish it in lesson 14.

• Blackboard notes at this link.

• Study Section II of the paper at this link. Password sent via Google group.

Lesson 14 (29/11/2024): MPC for task scheduling in assembly lines

We continued with the example on task control, detailing the lastt constraints and the target function.

• Blackboard notes at this link.

• Study Section II of the paper at this link. Password sent via Google group.

• Zoom recordings of the lesson at this link.

• Read sections  and 2 of paper [Bemporad_1999]. See below.

• Consult the IP reference guide. Link below under reference [MIT_2003].

Lesson 15 (02/12/2024): Finished MPC for task scheduling in assembly lines + MPC for traffic control

We finished the example on task control. We started the example on MPC for traffic control.

• Blackboard notes at this link and this link.

• Read Section II and Section III.B and III.D of paper Single Intersection MPC Traffic Signal Control in Presence of Automated Vehicles | IEEE Journals & Magazine | IEEE Xplore .

Lesson 16 (09/12/2024): Finished MPC for traffic control at single intersection

We finished the example on traffic control.

• Blackboard notes at this link.

Lesson 17 (12/12/2024): Introduction to Reinforcement Learning (RL)

We finished discussing the example on traffic control and then we introduced to the new topic of RL.

• Read Chapter 1 of book [Barto & Sutton]  of up to Section 1.4 included.

• Blackboard notes at this link.

Lesson 18 (13/12/2024): Introduction to Reinforcement Learning (RL) - Two examples on tabular methods

We gave two examples working with tabular methods. The tic-tac-toe game and the multi-armed bandit problem.

• Read Chapter 1 of book [Barto & Sutton]  of up to Section 1.4 included.

• Zoom recordings at this link.

• Blackboard notes at this link.

Lesson 19 (16/12/2024): Mathematical modelling of RL problems

We discussed the mathematical modelling of a RL problem: policy functions, value functions, Markov decision processes. The Bellman equation. Opimal value functions and policies.

• Read book [Barto & Sutton]  sections 2.1, 2.2, 2.4, and from 3.1 to 3.6 included.

• Zoom recordings at this link.

• Blackboard notes at this link.

Lesson 20 (19/12/2024): Mathematical modelling of RL problems  -aula 33

We discussed dynamic programming for policy evaluaion and policy improvement. In dynamic programming it is assumed that a model of the environment p(s',r|s,a) is available. We started then discussing the Monte Carlo methods, which can do policy evaluaiton and improvemen without assuming knowledge of p, by learning from the environment, doing averages of the return observed episode after episode.

• Read book [Barto & Sutton]  chapter 4 and sections 5.1, 5.2, 5.3.

• Zoom recordings at this link.

• Blackboard notes at this link.

Lesson 21 (20/12/2024): Conclusions on RL and Module 2 of the course

We discussed Monte Carlo and Temporal Difference Methods to estimate and improve the value functions when the model of the environment is not known. We discussed briefly how deep RL can be used to optimally tune MPC controllers and other parametreized controllers in general. We concluded the course with advice on the exam.

• Read book [Barto & Sutton]  chapter 4 and sections 6.1, 6.2, 6.4 and 6.5. Read the summary sections at the end of chapters from 1 to 6.

• Zoom recordings at this link.

• Blackboard notes at this link.

References and Study Materials

For each lesson, I will indicate the exact study material to consult. In the following you will find a preliminary list. I will strive to use only study material that is free and openly accessible over the internet.

Slides of the course are available at this link. Disclaimer: the slides provide only an overview of the content of the course. They are not sufficient for achieving a good preparation (you have to read also the additional reading matherial suggested in the above for each lesson).

BOOKS

Excellent books on MPC are:

• [Cannon_2016]. Kouvaritakis, Basil, and Mark Cannon. "Model predictive control." Switzerland: Springer International Publishing (2016): 38.

https://www.springer.com/gp/book/9783319248516. Available here: https://www.academia.edu/28894969/Advanced_Textbooks_in_Control_and_Signal_Processing_Model_Predictive_Control_Basil_Kouvaritakis_Mark_Cannon_Classical_Robust_and_Stochastic

• [Rawlings_2017]. Rawlings, James Blake, David Q. Mayne, and Moritz Diehl. Model predictive control: theory, computation, and design. Vol. 2. Madison, WI: Nob Hill Publishing, 2017.  Available online at: https://sites.engineering.ucsb.edu/~jbraw/mpc/MPC-book-2nd-edition-1st-printing.pdf

These are advanced books, we will use only specific sections, as I will mention during the course.

Reference book on (deep) reinforcement learning 

• [Barto & Sutton]. Andrew Barto and Richard S. Sutton. Reinforcement Learning: An Introduction. Second edition. Available online here:  http://incompleteideas.net/book/the-book-2nd.html

Some scientific articles we will consult.

The following ones are some excellent articles on MPC.

• [Bemporad_1999]. Bemporad , Alberto, and Manfred Morari . "Control of systems integrating logic, dynamics, and constraints." Automatica 35, no. 3 (1999): 407 427. 

https://www.sciencedirect.com/science/article/abs/pii/S0005109898001782

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.35.2690&rep=rep1&type=pdf

• [Mayne_2000]. Mayne, David Q., James B. Rawlings, Christopher V. Rao, and Pierre OM Scokaert. "Constrained model predictive control: Stability and optimality." Automatica 36, no. 6 (2000): 789-814. 

https://www.sciencedirect.com/science/article/pii/S0005109899002149

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.231.3109&rep=rep1&type=pdf

• [MIT_2003]. MIT Course “Optimization Methods in Management Science”. Available at this link.

See lecture notes at this link.

•I suggest checking in particular lessons L10 and L11.

•Read the IP reference guide (PDF).

• [Hax_1977]. Bradley, Stephen P., Arnoldo C. Hax, and Thomas L. Magnanti. Applied mathematical programming. Addison-Wesley, 1977.

http://web.mit.edu/15.053/www/AMP.htm

•Consult Chapter 9.

• Another interesting presentation on logical constraints and integer programming is at this link.

Reference for the application of MPC to traffic management.

• [Liberati_2020]. Liberati, Francesco. "Model predictive control of a road junction." Smart Cities 3.3 (2020): 806-817.

https://www.mdpi.com/2624-6511/3/3/41

Reference for the application of MPC to assembly line control.

• [Liberati_2021]. Liberati, Francesco. "Task Execution Control in an Assembly Line via Shrinking Horizon Model Predictive Control”. Password –protected file in the shared Google drive folder. The password has been sent via message on Google Class. Ask in case. Note: this file is confidential, for personal study only, it cannot be shared in any way.

Additional references on MPC for building energy management. These are not mandatory, but I suggest to have at least a quick read. Skip the parts we did not do at lesson. See the simulation sections to see how the MPC controller manages the loads, the generation and the storage of energy in the building:

• [2014_Di Giorgio] Di Giorgio, Alessandro, and Francesco Liberati. "Near real time load shifting control for residential electricity prosumers under designed and market indexed pricing models." Applied Energy 128 (2014): 119-132.

https://www.sciencedirect.com/science/article/abs/pii/S0306261914003857

• [2019_Liberati] Liberati, Francesco, et al. "Joint model predictive control of electric and heating resources in a smart building." IEEE Transactions on Industry Applications 55.6 (2019): 7015-7027.

https://ieeexplore.ieee.org/abstract/document/8786121/

• [2018_Liberati] Liberati, Francesco, and Alessandro Di Giorgio. "Economic model predictive and feedback control of a smart grid prosumer node." Energies 11.1 (2018): 48.

https://www.mdpi.com/1996-1073/11/1/48

Past Module 2 Exams