Process Identification and Control - 2025

News:

Exams dates have beed defined, see below.

Students please join the Google Class of the course by following this link. Please register only with the Sapienza email (@studenti.uniroma1.it).

For any doubt or question, write me an email (liberati@diag.uniroma1.it).

Logistic

• Office hours: Thursdays, 14:00 to 16:00, in presence or online (meet.google.com/zob-fhvk-bsz). Please send me an email to book an appointment in advance.  You are encouraged to come to the office hours, to ask any clarification.

• Lessons schedule - Link to official schedule

  • Monday 15:00-19:00 classroom 28 San Pietro in Vincoli (SPV); 

  • Thursday 12:00-14:00 room 28 SPV; 

  • Friday 14:00-16:00 room 9 SPV.

Starting from this year, the course is tought entirely by Francesco Liberati. In the past years, the process identification part was tought by Prof. Daniela Iacoviello (link to webpage).

Program

• Introduction to the course: process modelling, identification and control

• Process control with Model Predictive Control (MPC):

  • MPC formulation: objective function (stage cost, terminal cost), constraints, terminal constraints

  • Analysis of the main properties of MPC: recursive feasibility, optimality, stability.

  • Economic MPC.

  • MPC problems with mixed-integer variables and dynamical-locigal constraints. Examples.

  • Examples of applications in various sectors: task scheduling in industrial assembly lines, smart charging of electric vehicles, smart grids, traffic control, etc.

  • New: Reinforcement learning (RL) and hybrid MPC-RL algorithms.

  • Implementation of simple MPC algorithms using the Julia technical language (https://julialang.org/). Learn how to practically write and solve an MPC problem.

• Process identification:

  • Problem definition

  • Traditional methods

  • Advanced methods

  • Implementation examples in Julia and Python.

Exams and Grading

• Exam modality: Written exam with exercises and open questions. Closed book exam: at the exam, the student can use only paper, pen and calculator. It is forbidded to use any other electronic device or resources such as books, notes, slides, etc.

• Exam dates: see the official calendar at this link.

  • • mar 20/01/2026. Time and location to be decided.

    • lun 16/02/2026. Time and location to be decided.

• THE EXAM MUST BE BOOKED ON INFOSTUD. 

Lecture Summaries

For every lesson, a summary of the content will be available below, with the precise indication of the related study materials. Note: you can take benefit also of the materials of the past editions of the course, available on my website.

Lesson 1 (25/09/2025): Introduction to the course

We discuss the course logistics, office hours, study matherials, the exam, etc. High level introduction to the technical content of the course, focusing first on the control part. Intoduction to MPC. Nominal MPC and the thre main theoretical properties (optimality, recursive feasibility, stability). Robust, sthocastic and probabilistic MPC. Centralized vs decentralized MPC.

• Link to blackboard notes.

• Read chapter 1 of book [Cannon_2016].

• Read my slides on MPC up to slide 20.

• Read slides 1-26 of the presentation at this link of Prof. Bemporad.

Lesson 2 (26/09/2025): Nominal MPC

Finished the recap of the technical content of the course (control part). Started the discussion of nominal MPC. Definition of the MPC control pseudo-code. Definition of the nemenclature with two time indices. 

• Blackboard notes.

• Zoom recording of the lesson.

• Read my slides on MPC up to slide 25.

Lesson 3 (29/09/2025): MPC and Infinite-horizon constrained control

Recap of MPC pseudocode, MPC nomenclature. Nominal assumption. Toy example with full formulation. Alternative PID control. We then introduced an abstract and general formulation of the optimization problem at the base of MPC, discussing all its main ingredients (stage cost, terminal cost, model, state and input constraints, terminal constraint, etc.). Ideal infinite-horizon control. LQR in the constrained and unconstrained case. In the unconstrained LQR case, a closed-form solution can be found.

• Blackboard notes.

• Zoom recording of the lesson.

• Read chapter 2 of book [Cannon_2016] up to section 2.2 (included). Proofs are not mandatory.

• Read my slides on MPC up to slide 30.

Lesson 4 (02/10/2025): Example on modelling plus cost of infinite horizon unconstrained LQR

We did a simple modelling exercise, to see how to derive the state space model of a linear system. Then we resumed the discussion of infinite horizon constrained control. We computed the optimal cost associated to the solution (u=Kx) of the unconstrained LQR problem.

• Blackboard notes.

• Zoom recording off the lesson.

• Read my slides on MPC up to slide 30.

Lesson 5 (03/10/2025): Dual mode control

We  discussed the relation between the unconstrained LQR problem and the solution of the constrained LQR problem. We found that the solution of the constrained problem is given by the formula of the solution of the unconstrained LQR, after a given time instant.

• Blackboard notes.

• Zoom recording of the lesson.

• Read section 2.3 of book [Cannon_2016]. You can skip proofs. You can skip the part after Theorem 2.2.

Lesson 6 (06/10/2025): Dual mode control - conclusions + Started the first case study, on the control of an EV charging area 

We  finished discussing dual mode control, showing that the solution of the constrained LQR problem can be divided into two parts, of which the second is given by the formula of the unconstrained LQR. We discussed the implications on MPC. We started the discussion of the first case study of the course, on the control of a charging area for electric vehicles (EVs). We defined the problem and started the mathematical formulation.

• Blackboard notes - part 1.

• Blackboard notes - part 1.

• Zoom recording of the lesson.

• Read section 2.3 of book [Cannon_2016]. You can skip proofs. You can skip the part after Theorem 2.2.

Lesson 7 (09/10/2025): First case study, on the control of an EV charging area - Formulation and code implementation 

We completed the discussion of the first case study of the course, on the control of a charging area for electric vehicles (EVs). We completed the mathematical formulation and discussed the code implementation.

• Blackboard notes.

• Zoom recording of the lesson.

• Code folder.  Run the file "main.jl". You can use either Gurobi or Cbc solver. If you use Gurobi, you should first install it (download the Gurobi optimizer from here Gurobi Software - Gurobi Optimization - login with the Sapienza email), then get the academic licence (Gurobi Academic License Program: Free Licenses for Academics | Gurobi - Gurobi Optimization ), download the licence on your computer, and then install the Gurobi.jl Julia package (Pkg.add("Gurobi") - note this should be the last step). If instead you use Cbc, you only have to install it (Pkg.add("Cbc")) and load it in the program. 

Lesson 8 (10/10/2025): Completed first case study on the control of an EV charging area - Some final notes on the optimality of MPC, and how to compute performance 

We completed the discussion of the first case study of the course, running the limulations in Julia. Then we added some theoretical notes and considerations on the computation of the performance of infinite-horizon constrained control and the performance of MPC. We rewrote the infinite horizon problem into an outer and inner optimization problem, showing that the internal problem actually gives the value function of the control problem. This provides an alternative explanation of why in some case (when the value function si known), MPC is optimal. We also noticed that MPC is actually a feedback control strategy, and we said that in some simple case the MPC control can be computed explicitly, without having to solve the MPC iteration ("explicit MPC").

• Blackboard notes.

• Zoom recording of the lesson.

Lesson 9 (13/10/2025): Recursive feasibility + Introduction to stability

We presented a standard technique that can be used to prove the recursive feasibility of an MPc algorithm. It is based on the derivation of a tail-based tentative solution of the iteration at k+1, which uses part of the optimal solution at k. Next, we started the new topic of stability, introducing the concepts of equilibrium and stability.

• Blackboard notes.

• Zoom recording of the lesson.

Lesson 10 (16/10/2025): Lyapunov criterion for assessing the stability of nonlinear systems

We continued discussing about the stability of an equilibrium of a dynamical system. We introduced the Lyapunov stability criterion, which can be used to check the stability of the equilibrium of a nonlinear system.

• Blackboard notes.

• Zoom recording of the lesson.

• Read Sections 1.1, 1.2, 3.1, 3.2, 3.3  of the paper [Mayne_2000]. Mayne, David Q., James B. Rawlings, Christopher V. Rao, and Pierre OM Scokaert. "Constrained model predictive control: Stability and optimality." Automatica 36, no. 6 (2000): 789-814.  You can find it online easily.

Lesson 11 (17/10/2025): Started the discussion on stability of tracking MPC

We started the discussion of stability of tracking MPC, by attemting to use the optimal value of the MPC target function as the candidate Lyapunov function.

• Blackboard notes.

• Zoom recording of the lesson.

• Read Sections 1.1, 1.2, 3.1, 3.2, 3.3  of the paper [Mayne_2000]. Mayne, David Q., James B. Rawlings, Christopher V. Rao, and Pierre OM Scokaert. "Constrained model predictive control: Stability and optimality." Automatica 36, no. 6 (2000): 789-814.  You can find it online easily.

Lesson 12 (20/10/2025): Finished the discussion on stability of tracking MPC, started example on Economic MPC

We finished the discussion of stability of tracking MPC, busing the optimal value of the MPC target function as the candidate Lyapunov function. We then started an application example on economic MPC.

• Blackboard notes, Part 1.

• Blackboard notes, Part 2.

• Zoom recording of the lesson.

• Read Sections 1.1, 1.2, 3.1, 3.2, 3.3  of the paper [Mayne_2000]. Mayne, David Q., James B. Rawlings, Christopher V. Rao, and Pierre OM Scokaert. "Constrained model predictive control: Stability and optimality." Automatica 36, no. 6 (2000): 789-814.  You can find it online easily.

Lesson 13 (23/10/2025): Example on large scale PEV charging control

We discussed the example on large scale PEV charging control. In the next lesson, we finish the target function and improve the battery model.

• Blackboard notes.

• Zoom recording of the lesson.

Lesson 14 (24/10/2025): Example on large scale PEV charging control

We continued the example on large scale PEV charging control, introducing auxiliary variables and constraints to model the battery losses.

• Blackboard notes.

• Zoom recording of the lesson.

Lesson 15 (27/10/2025): Example on large scale PEV charging control - end

We finished the example on large scale PEV charging control, also mentioning decentralized strategies to solve the problem.

• Blackboard notes.

• Zoom recording of the lesson.

Lesson 16 (30/10/2025): Started the example on building management system design with MPC

We started the example on the design of a building management system using MPC.

• Blackboard notes.

• Zoom recording of the lesson.

Lesson 17 (31/10/2025): Continued the example on building management system design with MPC

We continued the example on the design of a building management system using MPC. We finished the lesson writing the objective function term to model the cost/remuneration from energy consumption.

• Blackboard notes.

• Zoom recording of the lesson.

Lesson 18 (03/11/2025): Continued the example on building management system design with MPC

We continued the example on the design of a building management system using MPC. We detailed the objective function and discussed some conclusions.

• Blackboard notes.

• Zoom recording of the lesson.

Lesson 19 (06/11/2025): Finished the example on building management system design with MPC - Started the example on task scheduling with MPC

We finished the example on the design of a building management system using MPC. We then started defining the problem of task scheduling in an assembly line.

• Blackboard notes.

• Zoom recording of the lesson.

Lesson 20 (07/11/2025): Continued the example on task scheduling with MPC

We continued the discussion of MPC for task scheduling in an assembly line.

• Blackboard notes.

• Zoom recording of the lesson.

Lesson 21 (10/11/2025): Finished the example on task scheduling with MPC. Started and completed the last example, on MPC for traffic control

We finished the discussion of MPC for task scheduling in an assembly line. Then we started and completed one last example, on the control of traffic with MPC.

• Blackboard notes.

• Zoom recording of the lesson. Part of the recording of the lesson after the break was lost.

• Study Section II and Section III.B and III.D of paper Single Intersection MPC Traffic Signal Control in Presence of Automated Vehicles | IEEE Journals & Magazine | IEEE Xplore .

• Study Section II of the paper Fast Task Scheduling With Model Predictive Control Integrating a Priority-Based Heuristic | IEEE Journals & Magazine | IEEE Xplore  

• Study paper Control of systems integrating logic, dynamics, and constraints - ScienceDirect  up to Definition 1 (excluded). Link to paper: PII: S0005-1098(98)00178-2 .

• The above papers can be accessed from Sapienza networks, after login with student email.

• Consult the IP reference guide. Link below under reference [MIT_2003].

**************** Start of Reinforcement Learning Lessons **********************

Lesson 22 (13/11/2025): Finished MPC and Started Introduction to Reinforcement Learning

We finished with a couple of notes on earlier MPC examples, and we introduced reinforcement learning.

• Blackboard notes.

• Zoom recording of the lesson.

• Study up to section 1.4 included of book [Barto & Sutton]. Andrew Barto and Richard S. Sutton. Reinforcement Learning: An Introduction. Second edition. Available online here:  http://incompleteideas.net/book/the-book-2nd.html

• Read this post by Prof. Dimitri Bertsekas. Check his webpage Dimitri Bertsekas Homepage . It contains many useful books and videolectures, for students who want to study more in detail the topics of this course (here for instance web.mit.edu/dimitrib/www/RLbook.html ). These are of course not mandatory to study.

Lesson 23 (14/11/2025): Two Examples on RL + Started Mathematical Modelling of RL Problems

We did two examples of simple RL problems, one in which we estimate the state value function V, and the other in which we estimate the action-value function Q. Then we started the formal mathematical modelling of RL problems, as Markov Decision Processes (MDP).

• Blackboard notes.

• Zoom recording of the lesson.

• Study up to formula (3.4) at pag 49 [Barto & Sutton].  You can skip section 2.8. Andrew Barto and Richard S. Sutton. Reinforcement Learning: An Introduction. Second edition. Available online here:  http://incompleteideas.net/book/the-book-2nd.htm.

Lesson 24 (17/11/2025): RL - General definitions and the Bellmann Equation

We provided the main definitions (policy, return, value functions, etc.). In particular, we defined the value functions v and q, and we introduced the Bellmann equations (which are two equations - one for v and one for 1 - that v and q satisfy). We defined the optimal policy and the associated optimal value functions and the associated version of the Bellmann equations (called Bellmann optimality equations). We then discussed Dynamic Programming, which is a way of using the Bellmann equation as an update rule to iteratively compute the value functions. We finally introduced the idea of policy improvement.

• Blackboard notes.

• Zoom recording of the lesson - Part 1. The recording was interrupted due to internet issues.

• Zoom recording of the lesson - Part 2.

• Study Chapter 3 and Chapter 4, up to Section 4.1 included of book Andrew Barto and Richard S. Sutton. Reinforcement Learning: An Introduction. Second edition. Available online here:  http://incompleteideas.net/book/the-book-2nd.htm.

Lesson 25 (20/11/2025): RL - Recap and Dynamic Programmin Methods

After a recap of the previous lesson, we discussed Dynamic Programming methods for policy evaluation and policy improvement. We defined the policy improvement procedure (repetition of the evaluation and improvement steps). We proved that when v stabilizes, than it means it is the optimal one. We discussed asyncronous dynamic programming and generalized policy iteration. We introduced the Monte Carlo methods for learning, the subject of next lesson.

• Blackboard notes.

• Zoom recording of the lesson.

• Study Chapter 4 of book Andrew Barto and Richard S. Sutton. Reinforcement Learning: An Introduction. Second edition. Available online here:  http://incompleteideas.net/book/the-book-2nd.htm.

Lesson 26 (21/11/2025): Monte Carlo RL Methods and Introduction to Temporal Difference Learning Methods

We discussed the Monte Carlo learning methods, which update the value function estimate after each episode (by computing the average observed discounted return after visiting the state).

• Blackboard notes.

• Zoom recording of the lesson.

• Study Chapter 5 of book Andrew Barto and Richard S. Sutton. Reinforcement Learning: An Introduction. Second edition. Available online here:  http://incompleteideas.net/book/the-book-2nd.htm. You can skip the parts we did not discuss in class. Always read the Summary at the end of the chapters.

Lesson 27 (24/11/2025): Temporal Difference Learning Methods and Introduction to More Recent Algorithms for Continuous Action and State spaces

We discussed TD algorithms and we introduced the discussion of recent RL algorithms that use neural networks as v,q approximators.

• Blackboard notes.

• Zoom recording of the lesson.

• Study Chapter 6, up to Section 6.5 included, of book Andrew Barto and Richard S. Sutton. Reinforcement Learning: An Introduction. Second edition. Available online here:  http://incompleteideas.net/book/the-book-2nd.htm. You can skip the parts we did not discuss in class. Always read the Summary at the end of the chapters.

• Read pag 142 and 143 of the book, on n-step methods.

Lesson 28 (27/11/2025): Finished RL + Introduction to identification lessons

We discussed the DDPG algorithm as an example of a more recent RL algorithm working with continuous action sapces. We then introduced to system/parameter identification.

• Blackboard notes.

• Zoom recording of the lesson.

• Study the DDPG algorithm for instance here Deep Deterministic Policy Gradient — Spinning Up documentation, or from the original paper [1509.02971] Continuous control with deep reinforcement learning.

References and Study Materials

For each lesson, I will indicate the exact study material to consult. In the following you will find a preliminary list. I will strive to use only study material that is free and openly accessible over the internet.

PROCESS CONTROL PART

Excellent books on MPC are:

• [Cannon_2016]. Kouvaritakis, Basil, and Mark Cannon. "Model predictive control." Switzerland: Springer International Publishing (2016): 38. Can be downloaded for free at this Link

• [Rawlings_2017]. Rawlings, James Blake, David Q. Mayne, and Moritz Diehl. Model predictive control: theory, computation, and design. Vol. 2. Madison, WI: Nob Hill Publishing, 2017.  Available online at: https://sites.engineering.ucsb.edu/~jbraw/mpc/MPC-book-2nd-edition-1st-printing.pdf

These are advanced books, we will use only specific sections, as I will mention during the course.

Slides on MPC part.

Reference book on (deep) reinforcement learning 

• [Barto & Sutton]. Andrew Barto and Richard S. Sutton. Reinforcement Learning: An Introduction. Second edition. Available online here:  http://incompleteideas.net/book/the-book-2nd.html

PROCESS IDENTIFICATION PART

• System identification: course notes prof. Philippe Dreesen, VRIJE Universiteit Brussel

• S. Bittanti, Model Identification and Data Analysis, John Wiley, 2019

• A.Garulli, A. Giannitrapani, Dispense di identificazione e analisi dati, 2011

• L. Ljung, System identification: Theory for the user. Prentice Hall, 1999

• A.De Santis, Metodi di identificazione

• J. Brownlee, How to Decompose Time Series Data into Trend and Seasonality, Time series, 2017,

• https://machinelearningmastery.com/decompose-time-series-data-trend-seasonality/

• Dario Radecic: https://towardsdatascience.com/time-series-from-scratch-decomposing-time-series-data-7b7ad0c30fe7

• https://online.stat.psu.edu/stat510/book/export/html/685

Some scientific articles we will consult.

The following ones are some excellent articles on MPC.

• [Bemporad_1999]. Bemporad , Alberto, and Manfred Morari . "Control of systems integrating logic, dynamics, and constraints." Automatica 35, no. 3 (1999): 407 427. 

https://www.sciencedirect.com/science/article/abs/pii/S0005109898001782

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.35.2690&rep=rep1&type=pdf

• [Mayne_2000]. Mayne, David Q., James B. Rawlings, Christopher V. Rao, and Pierre OM Scokaert. "Constrained model predictive control: Stability and optimality." Automatica 36, no. 6 (2000): 789-814. 

https://www.sciencedirect.com/science/article/pii/S0005109899002149

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.231.3109&rep=rep1&type=pdf

• [MIT_2003]. MIT Course “Optimization Methods in Management Science”. Available at this link.

See lecture notes at this link.

•I suggest checking in particular lessons L10 and L11.

•Read the IP reference guide (PDF) 

• [Hax_1977]. Bradley, Stephen P., Arnoldo C. Hax, and Thomas L. Magnanti. Applied mathematical programming. Addison-Wesley, 1977.

http://web.mit.edu/15.053/www/AMP.htm

•Consult Chapter 9.

• Another interesting presentation on logical constraints and integer programming is at this link.

Reference for the application of MPC to traffic management.

• [Liberati_2020]. Liberati, Francesco. "Model predictive control of a road junction." Smart Cities 3.3 (2020): 806-817.

https://www.mdpi.com/2624-6511/3/3/41

Reference for the application of MPC to assembly line control.

• [Liberati_2021]. Liberati, Francesco. "Task Execution Control in an Assembly Line via Shrinking Horizon Model Predictive Control”. Password –protected file in the shared Google drive folder. The password has been sent via message on Google Class. Ask in case. Note: this file is confidential, for personal study only, it cannot be shared in any way.

Additional references on MPC for building energy management. These are not mandatory, but I suggest to have at least a quick read. Skip the parts we did not do at lesson. See the simulation sections to see how the MPC controller manages the loads, the generation and the storage of energy in the building:

• [2014_Di Giorgio] Di Giorgio, Alessandro, and Francesco Liberati. "Near real time load shifting control for residential electricity prosumers under designed and market indexed pricing models." Applied Energy 128 (2014): 119-132.

https://www.sciencedirect.com/science/article/abs/pii/S0306261914003857

• [2019_Liberati] Liberati, Francesco, et al. "Joint model predictive control of electric and heating resources in a smart building." IEEE Transactions on Industry Applications 55.6 (2019): 7015-7027.

https://ieeexplore.ieee.org/abstract/document/8786121/

• [2018_Liberati] Liberati, Francesco, and Alessandro Di Giorgio. "Economic model predictive and feedback control of a smart grid prosumer node." Energies 11.1 (2018): 48.

https://www.mdpi.com/1996-1073/11/1/48

Past Module Exams

Previous exam on the process control part are in the following. For the previous exam on the process identification part, you can consult the webpage of Prof. Daniela Iacoviello.