Title: Robust Machine Learning, Reinforcement Learning and Autonomy: A Unifying Theory via Performance and Risk Tradeoff

Abstract: Robustness is a fundamental concept in systems science and engineering. It is a critical consideration in all inference and decision-making problems, both single agent and multi-agent ones. It has surfaced again in recent years in the context of machine learning (ML), reinforcement learning (RL) and artificial intelligence (AI). We describe a novel and unifying theory of robustness for all these problems emanating from the fundamental results we obtained in my research group some 25 year ago on robust output feedback control for general systems (including nonlinear, HMM and set-valued). We briefly summarize this theory and the universal solution it provides consisting of two coupled HJB equations. Our results rigorously established the equivalence of three seemingly unrelated problems: the robust output feedback control problem, a partially observed differential game, and a partially observed risk sensitive stochastic control problem. We first show that the “four block” view of the above results leads naturally to a similar formulation of the so-called robust (or adversarially robust) ML problem, and a rigorous path to analyze robustness and attack resiliency in ML. We show several examples. Then we describe a recent risk-sensitive approach, using an exponential criterion in deep learning, that explains the convergence of stochastic gradients despite over-parametrization. Finally, we describe our most recent results on robust and risk sensitive reinforcement learning (RL) for control, using exponential measures/rewards, that emerge from our earlier theory, with the important new extension that the models are now unknown. We show how all forms of regularized RL can be derived from our theory, including KL and Entropy regularization, relation to probabilistic graphical models, distributional robustness. All these new results emerge by systematic application of dynamic systems feedback control concepts to each of these learning problems. The deeper reason for this unification emerges: it is the fundamental tradeoff between performance and risk measures in decision making, via rigorous duality.   We close with open problems and future research directions.    

Biography: John S. Baras is a Distinguished University Professor, holding the Lockheed Martin Chair in Systems Engineering with the Institute for Systems Research (ISR) and the ECE Department at the University of Maryland College Park. He received his Ph.D. degree in Applied Mathematics from Harvard University, in 1973, and he has been with UMD since then. From 1985 to 1991, he was the Founding Director of the ISR. Since 1992, he has been the Director of the Maryland Center for Hybrid Networks (HYNET), which he co-founded. He is a Fellow of IEEE (Life), SIAM, AAAS, NAI, IFAC, AMS, AIAA, Member of the National Academy of Inventors (NAI) and a Foreign Member of the Royal Swedish Academy of Engineering Sciences (IVA). Major honors and awards include the 1980 George Axelby Award from the IEEE Control Systems Society, the 2006 Leonard Abraham Prize from the IEEE Communications Society, the 2017 IEEE Simon Ramo Medal, the 2017 AACC Richard E. Bellman Control Heritage Award, and the 2018 AIAA Aerospace Communications Award. In 2016 he was inducted in the University of Maryland  Clark School of Engineering Innovation Hall of Fame. In June 2018 he was awarded a Doctorate Honoris Causa by his alma mater the National Technical University of Athens, Greece. He has mentored 101 doctoral students, 135 MS students and 70 postdoctoral fellows, who have gone to excellent careers in industry, academia and government. His research interests include systems, control, optimization, autonomy, machine learning, artificial intelligence, communication networks, applied mathematics, signal processing and understanding, robotics, computing systems, formal methods and logic, network security and trust, systems biology, healthcare management, model-based systems engineering. He has been awarded nineteen patents, one software copyright, and honored with many awards world-wide, as innovator and leader of economic development. 

Title: Multi-scale Network Games: Modeling, Analysis, and Control 

Abstract: Strategic interactions among interconnected agents are commonly modeled using the formalism of network (or graphical) games, wherein the utility of an agent depends on not only its own actions but also those of its network neighbors. While a powerful modeling framework, network games in their basic form fail to capture a common phenomenon of organization among a large number of individual decision makers: groups/communities of varying sizes with potentially orthogonal, disjoint, or overlapping relationships, and larger groups composed of smaller groups. This is seen across human societies as well as more broadly in the biological world. The emergence of groups at multiple scales and their interactions can play an important role in (1) the agents' decision making and strategic interactions, whereby an agent could take on a group identity with considerable value associated with it, (2) how we conceptualize and design control and intervention policies, knowing we could aim to elicit an individual or a group response or both, and (3) how an adversary may attack the system, knowing actions against a group vs. individual agents could have very different costs and effects.  In this talk I will present a number of our recent studies on developing models for this type of multi-scale network games, and highlight the analytical, computational, and control advantages in doing so.  I will also briefly touch on the applicability of these models to the analysis of broad-spectrum threats and risk heterogeneity. 

Biography: Mingyan Liu received her Ph.D. Degree in electrical engineering from the University of Maryland, College Park, in 2000, and has been with the University of Michigan, Ann Arbor, ever since.  She is currently a professor with the Department of Electrical Engineering and Computer Science, the Alice L. Hunt Collegiate Professor of Engineering, and the Associate Dean for Academic Affairs. Her research interests are in optimal resource allocation, performance modeling, sequential decision and learning theory, game theory and incentive mechanisms, with applications to large-scale networked systems, cybersecurity and cyber risk quantification. She is the recipient of multiple best paper awards and numerous professional and institutional awards. She is a Fellow of the IEEE and a member of the ACM.

Title: Scalable Control for Distributed MIMO Communications

Abstract: Multi-antenna, or MIMO, technology has revolutionized wireless communication in terms of both power and spectral efficiency and is now a part of commercial wireless standards (e.g., WiFi and cellular). MIMO techniques today, however, are fundamentally constrained by form factor and carrier wavelength, both of which limit the number of antennas that can be accommodated on a transceiver. 

In recent years proposals have been made for distributed MIMO where groups of transceivers self-organize into virtual arrays that are indistinguishable in their functionality from a centralized antenna array and scale to an arbitrary number of nodes. Such distributed MIMO (DMIMO) concepts have been analyzed by theorists and dismissed by practitioners because of fundamental synchronization bottlenecks. Unlike centralized antennas, in distributed MIMO each node has its own oscillator. The frequency and phase of these oscillators undergo drift modeled by Brownian motion. Even expensive oscillators starting in perfect synchrony, become virtually incoherent within tens of milliseconds.   

This talk will focus on feedback assisted synchronization issues oftwo basic building blocks of DMIMO: distributed beamforming and distributed nullforming. In the former N-nodes collaborate to achieve a coherent beam whose power is N2-fold (as opposed to N-fold) higher than that due to a single node. Incoherence only results in an N-fold increase. In nullforming, nodes achieve a null at a prescribed location. It assists in achieving spatial-multiplexing and also secure communication. The key ingredient is scalable feedback by cooperating receivers. 

Biography: Soura Dasgupta, received the B.E. degree (Hons. I) in Electrical Engineering from the University of Queensland (Australia) in 1980, and the Ph.D. in Systems Engineering from the Australian National University, in 1985.  He is currently F. Wendell Miller University of Iowa Foundation Distinguished Professor in the Department of Electrical and Computer Engineering at the University of Iowa, U.S.A. In 1981, he was a Junior Research Fellow  at the Indian Statistical Institute, Calcutta. He has held visiting appointments at the University of Notre Dame, University of Iowa, Universite Catholique de Louvain-La-Neuve, Belgium, Tata Consulting Services, Hyderabad, the Australian National University,  Shandong Academy of Sciences and National ICT Australia.

He is a past Associate Editor of the IEEE TRANSACTIONS ON AUTOMATIC CONTROL, IEEE Control Systems Society Conference Editorial Board, and the IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS- II. He is a co-recipient of the Gullimen Cauer Award for the best paper published in the IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS in the calendar years of 1990 and 1991, a past Presidential Faculty Fellow (a predecessor of PECASE), a past subject editor for the International Journal of Adaptive Control and Signal Processing, and a member of the editorial board of the EURASIP Journal of Wireless Communications. In 2012 he was awarded the University Iowa Collegiate Teaching award. In the same year he was selected by the graduating class for excellence in teaching and commitment to student success. From 2016-18 he was a 1000 Talents Scholar in the People's Republic of China.

His research interests are in Controls, Signal Processing, Communications and Parkinson's Disease. He was elected a Fellow of the IEEE in 1998.

Title: Quickest detection of deception attacks in networked control systems with watermarking

Abstract: In this talk, we will present some of our recent results on quickest (sequential) detection of fake measurement attacks with the aid of an additive watermarking signal on the control input.  The objective of minimizing average detection delay subject to a constraint on the average performance loss measured via the increase in control cost due to the use of watermarking when there is no attack. We will show how the use of physical watermarking signal in the control input can be used to this end. We will also present some results on how to economize the use of watermarking signal so that the loss of control performance is minimized in case of no attacks. Some discussions will be presented on how to extend these algorithms to the case of unknown attacker signal parameters, by using joint system identification/learning and sequential detection techniques.

Biography: Subhrakanti Dey received the Bachelor in Technology and Master in Technology degrees from the Department of Electronics and Electrical Communication Engineering, Indian Institute of Technology, Kharagpur, in 1991 and 1993, respectively, and the Ph.D. degree from the Department of Systems Engineering, Research School of Information Sciences and Engineering, Australian National University, Canberra, in 1996. 

He is currently the Head of Division of Signals and Systems and Professor of Signal Processing at Uppsala University. Prior to this, he was with the National University of Ireland, Maynooth, Ireland, during 2018-2022, while being on leave of absence from Uppsala. He was also a Professor with the Department of Electrical and Electronic Engineering, University of Melbourne, Parkville, Australia, from 2000 until early 2013, when he moved to Uppsala University.  From September 1995 to September 1997, and September 1998 to February 2000, he was a Postdoctoral Research Fellow with the Department of Systems Engineering, Australian National University. From September 1997 to September 1998, he was a Postdoctoral Research Associate with the Institute for Systems Research, University of Maryland, College Park.

His current research interests include security and privacy in networked control systems, distributed optimization and learning over networks, and detection and estimation theory with energy harvesting sensor networks. Professor Dey currently serves on the Editorial Board of IEEE Control Systems Letters (Senior Editor), IEEE Transactions on Control of Network Systems, (Senior Editor) and IEEE Transactions on Wireless Communications (Editor), and Automatica (Assoc. Editor). He was also an Associate Editor for IEEE Transactions on Signal Processing, (2007-2010, 2014-2018), IEEE Transactions on Automatic Control, (2004-2007) and Elsevier Systems and Control Letters (2003-2013). 

Title: Stealthy Sensor Attacks: Characterization and Moving Target Defense

Abstract: Reports of cyber-attacks, such as Stuxnet, have shown their devastating consequences on digitally controlled systems supporting modern societies, and shed light on their modus operandi: First learn sensitive information about the system, then tamper the visible information so the attack is undetected, and meanwhile have significant impact on the physical system. In this talk, we focus on specific sensor attack scenarios and mitigation strategies currently being investigated in our research group. We show that an attacker with access to the sensor channel can perfectly estimate a linear controller’s state without error if, and only if, the controller has no unstable poles. An advanced attacker may exploit such a breach of confidentiality to design stealthy false data injection attacks (violations of sensor data integrity) resulting in large physical impact on the controlled plant. We illustrate some of the results using lab experiments, and discuss moving target defense mitigation strategies based on game theory.

Biography: Henrik Sandberg is Professor at the Division of Decision and Control Systems, KTH Royal Institute of Technology, Stockholm, Sweden. He received the M.Sc. degree in engineering physics and the Ph.D. degree in automatic control from Lund University, Lund, Sweden, in 1999 and 2004, respectively. From 2005 to 2007, he was a Post-Doctoral Scholar at the California Institute of Technology, Pasadena, USA. In 2013, he was a Visiting Scholar at the Laboratory for Information and Decision Systems (LIDS) at MIT, Cambridge, USA. He has also held visiting appointments at the Australian National University and the University of Melbourne, Australia. His current research interests include security of cyber-physical systems, power systems, model reduction, and fundamental limitations in control. Dr. Sandberg was a recipient of the Best Student Paper Award from the IEEE Conference on Decision and Control in 2004, an Ingvar Carlsson Award from the Swedish Foundation for Strategic Research in 2007, and a Consolidator Grant from the Swedish Research Council in 2016. He has served on the editorial boards of IEEE Transactions on Automatic Control and the IFAC Journal Automatica. He is Fellow of the IEEE. 

Title: Unveiling the Impact: Exploring Grounding Effects on Scalable Consensus Networks 

Abstract: This research focuses on investigating the effects of disruption via grounding in discrete-time consensus networks. Grounding refers to a phenomenon where a subset of agents in a network no longer respond to inputs from other agents, potentially modifying their dynamics. These grounded agents, which can be leaders or stubborn agents, may experience grounding due to internal faults, safety protocols, external malicious attacks, or deliberate design choices. In this talk, we explore the impact of grounding on expander graph families, which are known for their favorable scaling properties as the network size increases. Our analysis reveals that grounding leads to a decrease in the algebraic connectivity and eigenratio of the network, resulting in a deterioration of performance and scalability. The findings of this research contribute to a deeper understanding of the consequences of grounding in consensus networks and its implications for network design and resilience. 

Biography: Dr. Yamin Yan received the B.E. degree from Wuyuzhang Honors College in Automation, Sichuan University, Chengdu, China, in 2013, and the Ph.D. degree from the Department of Mechanical and Automation Engineering, the Chinese University of Hong Kong (CUHK), Hong Kong, in 2017. She worked as a Research Associate at the University of Newcastle, Australia, from 2018 to 2021. She is currently a Research Assistant Professor at the Department of Electronic and Computer Engineering/ the CKS Robotics Institute, the Hong Kong University of Science and Technology (HKUST), as well as an adjunct lecturer at the University of Newcastle. Her research interests include networked systems and control, cyber-physical systems, security control, and computational intelligence.

She was a selected participant in the 2022 Asian Deans' Forum-The Rising Stars Women in Engineering Workshop. She is also a recipient of the 2022 Royal Society International Kan Tong Po Visiting Fellowship, the 2019 Future Women Leaders Conference Award for women in STEM from Monash University, Australia, the 2019 IEEE International Conference on Control and Automation (ICCA) Best Paper Finalist, and 2017 CUHK Faculty Outstanding Tutors Award.   

Title: Mean field game among teams

Abstract: In this talk  we present a model of a game among teams. Each team consists of a homogeneous population of agents. Agents within a team are cooperative while the teams compete with other teams. The dynamics and the costs are coupled through the empirical distribution (or the mean field) of the state of agents in each team, which is assumed to be observed by all agents. Agents have asymmetric information (also called a non-classical information structure). We propose a mean-field based refinement of the Team-Nash equilibrium of the game, which we call mean-field Markov perfect equilibrium (MF-MPE). We identify a dynamic programming decomposition to characterize MF-MPE. We then consider the case where each team has a large number of players and present a mean-field approximation which approximates the game among large-population teams as a game among infinite-population teams. We show that MF-MPE of the game among teams of infinite population is easier to compute and is an $\varepsilon$-approximate MF-MPE of the game among teams of finite population.

Biography: Aditya Mahajan is Associate Professor of Electrical and Computer Engineering at McGill University, Montreal, Canada. He is a member of the McGill Center of Intelligent Machines (CIM), Montreal Institute of Learning Algorithms (MILA), and Groupe d'études et de recherche en analyse des décisions (GERAD). He received the B.Tech degree in Electrical Engineering from the Indian Institute of Technology, Kanpur, India and the MS and PhD degrees in Electrical Engineering and Computer Science from the University of Michigan, Ann Arbor, USA.

He is a senior member of the IEEE and member of Professional Engineers Ontario. He currently serves as Associate Editor of IEEE Transactions on Automatic Control, IEEE Control Systems Letters, and Springer Mathematics of Control, Signal, and Systems. He was an Associate Editor of the IEEE Control Systems Society Conference Editorial Board from 2014 to 2017.

He is the recipient of the 2015 George Axelby Outstanding Paper Award, the 2016 NSERC Discovery Accelerator Award, the 2014 CDC Best Student Paper Award (as supervisor), and the 2016 NecSys Best Student Paper Award (as supervisor). His principal research interests include decentralized stochastic control, team theory, reinforcement learning, multi-armed bandits and information theory.

Title: Deceptive Decision-Making: Inference, Strategies, and Environment Co-Design

Abstract: The success of an agent in a number of defense and intelligence scenarios rests on the agent's use of deception: a strategy that enables the agent to progress towards its objective while influencing the beliefs of the agent's adversary about the agent's intent. For instance, an adversarial intruder may wish to instill incorrect belief about its location, identity or objective, while at the same time progressing along its mission. The success of the agent's deception strategy naturally depends on its opponent's inference mechanism, as well as the characteristics of the environment in which deception is taking place. In this talk, I will introduce current avenues of work on formalizing the notions of deception in autonomous and human-machine systems. I will discuss design of optimal deceptive strategies as well as the design of optimal strategies to prevent or uncover deception. Finally, I will briefly describe recent work on counter-deceptive environment planning, attempting to rigorously ensure that the agents' operating environment layout helps expose potential deception.

Biography: Melkior Ornik is an assistant professor in the Department of Aerospace Engineering at the University of Illinois Urbana-Champaign in Urbana, USA, also affiliated with the Coordinated Science Laboratory, as well as the Discovery Partners Institute in Chicago. He received his Ph.D. degree from the University of Toronto in 2017. His research focuses on developing theory and algorithms for control, learning and task planning in autonomous systems that operate in uncertain, changing, or adversarial environments, as well as in scenarios where only limited knowledge of the system is available. He is a Senior Member of IEEE, his recent work has been extensively funded by NASA grants and Department of Defense programs, and he has been awarded the 2023 Air Force Young Investigator Program grant.

Title: Performance Guarantees for Learning-based Decision Making

Abstract: Decision-making methods based on machine learning have now demonstrated super-human performance in multiple domains, like playing strategic games (Weiqi, chess, poker, etc.). A mathematical framework underlying such approaches is to approximate the value function, founded on optimal control theory and Bellman's principle for Markov decision processes. But performance guarantees for such solutions have remained elusive, leaving the performance of learning-based decision making reliant on extensive experimentation. In this talk, we describe a method to provide guaranteed performance bounds. The method involves quantifying the curvature of the objective function in an optimization problem associated with a given decision scheme.

Biography: Edwin K. P. Chong received the B.E. degree with First Class Honors from the University of Adelaide, South Australia, in 1987 and the M.A. and Ph.D. degrees in 1989 and 1991, respectively, both from Princeton University, where he held an IBM Fellowship. He joined the School of Electrical and Computer Engineering (ECE) at Purdue University in 1991. Since August 2001, he has been a Professor of ECE and Professor of Mathematics at Colorado State University. He currently serves as Head of ECE. He coauthored the best-selling book, An Introduction to Optimization (5th Edition, Wiley-Interscience, 2023). He received the NSF CAREER Award in 1995 and the ASEE Frederick Emmons Terman Award in 1998. He was a co-recipient of the 2004 Best Paper Award for a paper in the journal Computer Networks. In 2010, he received the IEEE Control Systems Society Distinguished Member Award. 

Prof. Chong is a Fellow of IEEE and of AAAS. He was the founding chairman of the IEEE Control Systems Society Technical Committee on Discrete Event Systems and was an IEEE Control Systems Society Distinguished Lecturer. He was an inaugural Senior Editor of the IEEE Transactions on Automatic Control. He was the General Chair for the 2011 Joint 50th IEEE Conference on Decision and Control and European Control Conference. He served as President of the IEEE Control Systems Society in 2017.

Title: Adversarial Robustness Considerations in Black-Box Optimization

Abstract: We consider the problem of optimizing an unknown function via noisy black-box function samples.  In settings where these samples are expensive to obtain (e.g., automatic parameter tuning), Gaussian process (GP) methods have been highly popular due to their sample efficiency, with extensive ongoing algorithmic and theoretical developments.

In this talk, I will cover two notions of *adversarially robust* optimization under the GP framework: (i) the final (opimized) point may be perturbed by an adversary and we want the function value to remain as high as possible despite this perturbation; and (ii) the function samples themselves are subject to adversarial noise, rather than only random noise.  For each of these settings, I will present algorithms based on optimism under uncertainty and/or successive elimination, and briefly outline their theoretical performance guarantees.

Biography: Jonathan is an Assistant Professor in the Department of Computer Science and Department of Mathematics at the National University of Singapore (NUS).  His research interests are in information theory, machine learning, signal processing, and high-dimensional statistics, particularly their intersection.  He is a holder of the Singapore National Research Foundation (NRF) fellowship and the NUS Presidential Young Professorship.  Before joining NUS, he was a post-doctoral research fellow at EPFL, and completed his PhD at the University of Cambridge.

Title: Reputation-based information design for increasing prosocial behaviour

Abstract: We will discuss an information design problem that uses (1) norms and conventions in a society or network and (2) individuals' desires to have good reputation (in keeping with these norms and conventions), to increase the quantum of prosocial actions. A function of each individual's action is made public and considerations of the resulting reputation improves prosocial actions. The planner designs the function to make public and agents respond with their actions. We will study the equilibria that ensue and will highlight some interesting properties. The problem came out of a field experiment conducted in about 20,000 households in Kerala on how effective feedback can be in reducing residential energy consumption. The talk will be on joint work with Alexandre Reiffers-Masson.

Biography: Rajesh Sundaresan is a Professor of Electrical Communication Engineering, an Associate Faculty at the Robert Bosch Centre for Cyber-Physical Systems, and the current Dean of the Division of Electrical, Electronics, and Computer Sciences, at the Indian Institute of Science. His research interests include decision theory, communication, computation, and control over networks, cyber-social systems, and, more recently, data-driven decision frameworks for public health responses.