SIEGE: AI-CPS Safety Enhancement Framework

This website provides the supplementary materials for the paper "SIEGE: A Semantics-Guided Safety Enhancement Framework for AI-enabled Cyber-Physical Systems", which presents detailed research workflow and experiment results not shown in the paper due to the page limit.

The website is organized as follows:

• Home page: The motivation why an industrial-level AI-enabled CPS benchmark is urgently needed, which is followed by an illustration and an introduction of our research workflow.

• AI-CPS Construction: In this section, we introduce how we use the Reinforcement learning Toolbox in MATLAB to construct RL controllers as substitutions of traditional controllers. We also present the training procedures of Deep Reinforcement Learning (DRL) controllers, including environment setup, reward function design, termination logic, agent selection, and training configurations.

• Experimental System: This section contains all subject CPS we collected for benchmarking with their environment introductions, functionality descriptions, control system explanations, specifications, and image illustrations.

• Semantics Abstraction: In this section, we describe the steps to build the specification-oriented semantics abstraction model, and the refinement process. Also, we introduce the steps that how to use the abstract model to predict the semantics of the incoming states.

• Ensemble Strategies: This section introduces the classic ensemble methods that we used as one of the baselines. And four newly proposed ensemble strategies that leverage the predicted semantics information from the abstract model to enhance the decision-making in AI-CPS.

• Summary: We make a summarization of the discussions, challenges, and opportunities of this work.

• Replication Package: The replication pack helps other researchers reproduce our work and conduct further efforts on safety assurance for AI-CSP.

Abstract

• Cyber-Physical Systems (CPSs) have been widely adopted in various industry domains to support many important tasks that impact our daily lives, such as automotive vehicles, robotics manufacturing, and energy systems. As Artificial Intelligence (AI) has demonstrated its promising abilities in diverse tasks like decision-making, prediction, and optimization, a growing number of CPSs adopt AI components in the loop to further extend their efficiency and performance. However, these modern AI-enabled CPSs have to tackle pivotal problems that the AI-enabled control systems might need to compensate for the balance across multiple operation requirements and avoid possible defections in advance to safeguard human lives and properties.

• Modular redundancy and ensemble methods are two widely adopted solutions in the traditional CPSs and AI communities to enhance the functionality and flexibility of a system. Nevertheless, there is a lack of deep understanding of the effectiveness of such ensemble design on AI-CPSs across diverse industrial applications. Considering the complexity of AI-CPSs, existing ensemble methods fall short of handling such huge state space and sophisticated system dynamics. Furthermore, an ideal control solution should consider the multiple system specifications in real-time and avoid erroneous behaviors beforehand. Such that, a new specification-oriented ensemble control system is of an urgent need for AI-CPSs.

• In this paper, we propose SIEGE: a semantics-guided ensemble control framework to initiate an early exploratory study of ensemble methods on AI-CPSs and aim to construct an efficient, robust, and reliable control solution for multi-tasks AI-CPSs. We first utilize a semantic-based abstraction to decompose the large state space, capture the ongoing system status and predict future conditions in terms of the satisfaction of specifications. We propose a series of new semantics-aware ensemble strategies and an end-to-end DRL hierarchical ensemble method to improve the flexibility and reliability of the control systems.

• Our large-scale, comprehensive evaluations over five subject CPSs show that

  1. The semantics abstraction can efficiently narrow the large state space and predict the semantics of incoming states,

  2. Our semantics-guided methods outperform state-of-the-art individual controllers and traditional ensemble methods, and

  3. The DRL hierarchical ensemble approach shows promising capabilities to deliver a more robust, efficient, and safety-assured control system.

Research Workflow

• Our framework SIEGE initiates an early exploratory study of ensemble methods on the control systems of AI-CPSs. We leverage a novel semantics-based abstraction to decompose the large state space and transparently inspect the status of the system. Four new ensemble methods are designed with the utilization of semantics information from the abstract model. We evaluate the newly proposed methods with comparisons to individual controllers and two classical ensemble methods by four research questions (RQs).

• RQ1 and RQ2 investigate the performance of the abstraction method in terms of effectiveness and the ability of semantics prediction. RQ3 studies the efficacy of classical ensemble methods, and RQ4 further demonstrates the performance of newly proposed ensemble strategies from other controllers.

• The figure above depicts the overview of SIEGE. It takes the selected DRL training methods as input and outputs an ensemble control system. At a high level, SIEGE contains three components: DRL sub-controller training, the semantics abstraction, and the ensemble strategies. As sub-controllers play a foundation in the ensemble system, it is necessary to obtain reliable sub-controllers as constituents.

• In the context of multi-requirement control, we aim to train sub-controllers with diverse decision logic. To achieve this goal, we concentrate on training DRL agents with different DRL algorithms, as well as the reward function setting. We then perform model abstraction to empower the ensemble analysis. We construct the abstract model by semantics-based state abstraction, action abstraction, and transition abstraction. A refinement process is applied to minimize the semantics error between concrete states and abstract states. Finally, we use the ensemble strategies to aggregate the action of the low-level controllers. The ensemble strategies in our study are from three categories: classic strategy, semantics-based strategy and coordinator-based strategy.