Reinforcement Learning

This course is designed to provide a comprehensive understanding of reinforcement learning (RL) and its applications in Production Control and Operations (P&OC). Participants will learn the fundamental concepts, algorithms, and techniques of RL, as well as advanced topics and real-world case studies. The course enables participants to develop and deploy RL-based solutions for various tasks relevant to production scheduling, inventory management, and supply chain optimization.

• Understand the principles, concepts, and mathematical foundations behind reinforcement learning

• Formulate P&OC problems as RL tasks and design appropriate reward functions

• Implement and apply classic RL algorithms, such as Q-learning, SARSA, and policy gradient methods

• Develop and train deep RL models, such as Deep Q-Networks (DQNs), Actor-Critic methods, and Proximal Policy Optimization (PPO)

• Apply advanced RL techniques, such as inverse RL, hierarchical RL, and multi-agent RL, to complex P&OC problems

• Deploy RL-based solutions for production scheduling, inventory management, supply chain optimization, and other real-world applications

1. Introduction to Reinforcement Learning

• Overview of RL and its applications in Production Control and Operations

• Markov Decision Processes (MDPs) and the Bellman equation

• Value functions, policies, and the exploration-exploitation trade-off

• Hands-on exercises: Implementing basic MDPs and solving them using dynamic programming

2. Classic Reinforcement Learning Algorithms

• Temporal Difference (TD) learning and the Q-learning algorithm

• SARSA (State-Action-Reward-State-Action) and its comparison with Q-learning

• Monte Carlo methods and their applications in RL

• Policy gradient methods and the REINFORCE algorithm

• Hands-on exercises: Implementing and applying Q-learning, SARSA, and policy gradient methods to production scheduling problems

3. Deep Reinforcement Learning

• Deep Q-Networks (DQNs) and their extensions (e.g., Double DQN, Dueling DQN)

• Actor-Critic methods and the Advantage Actor-Critic (A2C) algorithm

• Proximal Policy Optimization (PPO) and its advantages

• Hands-on exercises: Developing and training deep RL models for inventory management and supply chain optimization

4. Advanced Reinforcement Learning Techniques

• Inverse reinforcement learning and its potential for learning from expert demonstrations in production scheduling

• Hierarchical reinforcement learning for multi-timescale decision-making in production systems

• Multi-agent reinforcement learning and its applications in distributed production control and resource allocation

• Model-based reinforcement learning and its benefits for sample efficiency in P&OC simulations

• Hands-on exercises: Implementing advanced RL techniques for specific production control and operations use cases

5. Real-World Applications and Case Studies

• Production scheduling and resource allocation using RL

• Inventory management and replenishment with RL-based strategies

• Supply chain optimization and logistics planning using RL

• Predictive maintenance and fault detection in production systems with RL

• Hands-on exercises: Developing RL-based solutions for real-world P&OC problems

6. Deployment and Future Directions

• Deploying RL models in production environments and integrating them with existing P&OC systems

• Challenges and best practices for reward function design, hyperparameter tuning, and safety considerations in RL for production systems

• Monitoring and updating deployed RL models for continuous improvement

• Future research directions and open problems in RL for production control and operations

• Hands-on exercises: Deploying an RL model using a cloud platform (e.g., AWS, GCP) and integrating it with a simulated production system or supply chain environment

• Strong understanding of linear algebra, calculus, and probability theory

• Proficiency in programming with Python and deep learning frameworks (e.g., TensorFlow, PyTorch)

• Familiarity with basic machine learning concepts and techniques (e.g., supervised learning, neural networks)

• Knowledge of production control and operations management principles