The tutorial slides have now been uploaded here. Thank you for attending the tutorial!!!

Why is this tutorial being held at CPS-IoT week? 

AI-based approaches are increasingly being used for decision-making (both planning and control) in cyber-physical systems (CPS). However, CPS increasingly operate in environments that evolve over time due to drift in physical parameters, changing operating conditions, sensor degradation, and exogenous shocks. Yet most planning, control, and reinforcement learning methods rely on the assumption of stationary dynamics—an assumption that often breaks down in real-world deployments. This tutorial introduces a principled framework for decision-making under non-stationarity, equipping participants with the theoretical foundations, algorithmic tools, and practical experience needed to design adaptive decision-making systems that remain robust as the environment changes.

What will you learn? 

By the end of the tutorial, participants will be able to:

• Understand why stationarity assumptions fail in real-world CPS and IoT environments.

• Model evolving system dynamics using non-stationary MDPs, including abrupt changes, continuous drift, and periodic variation.

• Distinguish between known, detected, and hidden forms of non-stationarity.

• Evaluate why classical RL and online search-based methods struggle under non-stationarity and how adaptive planning can address these challenges.

• Design and experiment with non-stationary environments using NS-Gym, including CartPole, FrozenLake, CliffWalking, and Bridge environments.

• Apply adaptive decision-making techniques to CPS domains such as transportation, emergency response, and autonomous systems.

This work is based on NSF Award CNS-2238815, CNS-2531369 and the Assured Neuro Symbolic Learning and Reasoning (ANSR) project sponsored by the Defense Advanced Research Projects Agency (DARPA). Results presented in this tutorial were obtained using the Chameleon testbed supported by the National Science Foundation. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF or the DARPA.