Spring 2024-- 2025: ECE-698 ST Reinforcement Learning for Engineers
Details: This course will introduce the students to the basics of Markov Decision Process, and Reinforcement Learning (RL). The course is a little math-heavy, and intended for advanced MS, and Ph.D. students. The students will learn how to formulate problems as MDP, understand what algorithms work under what circumstances, differences among different algorithms, and how to implement those algorithms.
Course Schedule:
Topic 1: Markov Decision Process (MDP)
What is MDP? How to represent a decision-making problem as MDP? How to solve MDP (Dynamic Programming)? Value-Iteration and Policy iteration
Topic 2: Introduction to RL
Monte-carlo-based approaches, Q learning, TD learning, Policy gradient
Topic 3: Offline RL
Off-policy learning, batch Q learning
Topic 4: Function approximation
Actor-critic algorithm, advanced policy gradient
Topic 5: What are model-based and model-free RL algorithms
Convergence rate, Exploration and exploitation trade-off.
Topic 6: Deep RL
Neural-network based approximation, deep policy search
Topic 7: Fine-tuning RL and continual RL
LLMs, RLHF, meta-learning
Topic 8: Applications and Open Research Problems
The applications will be based on practical problems from a range of Engineering disciplines (Electrical, Mechanical, Computer, Civil, Bio-medical).
Fall 2023-2024: ECE 601 Linear Systems
This course provides the basic understanding of Linear Algebra, and its applications in the Dynamical Systems.