Project Overview

The recent fast advance of artificial intelligence technologies enables various autonomous cyber-physical systems (e.g., autonomous vehicles and unmanned aerial vehicles) to accomplish sophisticated tasks in real-world environment. However, the cyber components of these systems face increasing threats from adversaries. This project will study both the internal and external attacks on sensor data, i.e., sensor data corruption by attackers who have obtained access to the system’s sensors and adversarial examples that can be implemented by introducing tiny changes to the system’s environment. The existing studies (this, this, and this) have shown the feasibility of such attacks on several deployed software systems for vehicle driver-assistance and drones. This project will design attack detection and thwarting approaches and develop their efficient implementations suitable for embedded hardware accelerators. Moreover, this project will construct a testbed to capture representative cyber components of autonomous cyber-physical systems. It facilitates the design, evaluation, and demonstration of the threats and countermeasures. The demonstration will reinforce the relevant industry’s awareness on the criticality of the studied threats.

Project Team

This project is a collaborative effort between School of Computer Science and Engineering (SCSE) of Nanyang Technological University (NTU), Advanced Digital Sciences Center (ADSC) and Coordinated Science Laboratory (CSL) of University of Illinois at Urbana-Champaign (UIUC).

• NTU/SCSE Research Team

  • PI Associate Professor Dr. Rui Tan

  • Research Associates: Dongfang Guo, Yuting Wu

  • Involved PhD students: Wenjie Luo (PhD candidate)

• UIUC/ADSC Research Team

  • Co-PI Senior Research Scientist Dr. Xin Lou (also Assistant Professor at SIT)

• UIUC/CSL Collaborators

  • Prof. Ravishankar K. Iyer

  • Prof. Zbigniew T. Kalbarczyk

• Project Alumni

  • Dr. Pengfei Zhou: Research Scientist at ADSC. Joined University of Pittsburgh as Assistant Professor in 2022.

  • Dr. Qun Song: PhD student at NTU who contributed to the project. Joint TU Delft as Assistant Professor in 2022.

  • Dr. Zhenyu Yan: PhD student and Research Fellow at NTU who contributed to the project. Joined CUHK as Research Assistant Professor in 2021.

  • Dr. Sridhar Adepu: Research Fellow at ADSC. Joined University of Bristol as Lecturer (PhD Supervisor) in 2021.

• Associated Undergraduate Students at NTU for Final Year Projects

  • Peizhu Zhao, Ryan Edward Tay Siang An (2022 fall batch)

  • Goh Ying Ting (2022 spring batch)

  • Chia Yi You (2021 fall batch)

  • Beh Chee Kwang Nicholas (2021 spring batch)

  • Ang Zhan Phung, Khoo Kai Siang, Milla Samuel, Toh Koo Fong (2020 fall batch)

Code Releases

1. Code for dynamic ensemble generated by HyperNetwork to counteract adversarial examples
   Related publication: Sardino at EWSN'22.

Publication

From project

1. Susceptibility of Autonomous Driving Agents to Learning-Based Action-Space Attacks.
   Yuting Wu, Xin Lou, Pengfei Zhou, Rui Tan, Zbigniew Kalbarczyk, Ravishankar K. Iyer.
   The 9th International Workshop on Safety and Security Intelligent Vehicle (SSIV) with DSN'23, June 27, 2023, Porto, Portugal.

2. Work-in-Progress: Effects of Learning-Based Action-Space Attacks on Autonomous Driving Agents.
   Yuting Wu, Xin Lou, Pengfei Zhou, Rui Tan, Zbigniew Kalbarczyk, Ravishankar K. Iyer.
   The 13th ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS) Work-in-Progress session, May 9-12, San Antonio, Texas, USA. CPS-IoT Week 2023.

3. Towards Efficient Personalized Driver Behavior Modeling with Machine Unlearning.
   Qun Song, Rui Tan, Jianping Wang.
   The 7th IEEE/ACM Workshop on the Internet of Safe Things (SafeThings), May 9, 2023, San Antonio, Texas, USA. CPS-IoT Week 2023. [preprint]

4. On Credibility of Adversarial Examples against Learning-Based Grid Voltage Stability Assessment.
   Qun Song, Rui Tan, Chao Ren, Yan Xu, Yang Lou, Jianping Wang, Hoay Beng Gooi.
   IEEE Transactions on Dependable and Secure Computing (TDSC). Special Issue on Reliability and Robustness in AI-Based Cybersecurity Solutions. In press. [preprint]

5. Adversarial Attacks and Mitigations on Scene Segmentation of Autonomous Vehicles.

Yuqing Zhu, Sridhar Adepu, Ying Yang, Kushagra Dixit, Xin Lou.

The 8th Workshop On The Security Of Industrial Control Systems & Of Cyber-Physical Systems (CyberICPS), in Conjunction With ESORICS, September 26-30, 2022, Copenhagen, Denmark.

6. Sardino: Ultra-Fast Dynamic Ensemble for Secure Visual Sensing at Mobile Edge.
   Qun Song, Zhenyu Yan, Wenjie Luo, Rui Tan.
   The 19th International Conference on Embedded Wireless Systems and Networks (EWSN), Oct 3-5, 2022, Linz, Austria. (Acceptance ratio: 14/46=30%) [preprint]

7. DeepMTD: Moving Target Defense for Deep Visual Sensing against Adversarial Examples.
   Qun Song, Zhenyu Yan, Rui Tan.
   ACM Transactions on Sensor Networks (TOSN). In press. [PDF]

8. Understanding Credibility of Adversarial Examples against Smart Grid: A Case Study for Voltage Stability Assessment.
   Qun Song, Rui Tan, Chao Ren, Yan Xu.
   The 12th ACM International Conference on Future Energy Systems (e-Energy), June 28 - July 2, 2021, Torino, Italy. (Acceptance ratio: 17/75=22%) [PDF]

Related work

1. Moving Target Defense for Embedded Deep Visual Sensing against Adversarial Examples.
   Qun Song, Zhenyu Yan, Rui Tan.
   The 17th ACM Conference on Embedded Networked Sensor Systems (SenSys), November 10-13, 2019, New York, NY, USA. [PDF]

2. Learning-Based Time Delay Attack Detection for Cyber-Physical Systems.
   Xin Lou, Cuong Tran, David Yau, Rui Tan, Hongwei Ng, Zhengjia Fu, and Marianne Winslett.
   IEEE SmartGridComm 2019 - Workshop: AI in Energy Systems, 2019. 

3. ML-Based Fault Injection for Autonomous Vehicles: A Case for Bayesian Fault Injection.
   Saurabh Jha, Subho S. Banerjee, Timothy Tsai, Siva K. S. Hari, Michael B. Sullivan, Zbigniew T. Kalbarczyk, Stephen W. Keckler and Ravishankar K. Iyer.
   The 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2019. [PDF]

Acknowledgement

This project is supported by the National Research Foundation, Singapore and National University of Singapore through its National Satellite of Excellence in Trustworthy Software Systems (NSOE-TSS) office under the Trustworthy Computing for Secure Smart Nation Grant (TCSSNG) award no. NSOE-TSS2020-01.

Disclaimer

Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore and National University of Singapore (including its National Satellite of Excellence in Trustworthy Software Systems (NSOE-TSS) office).