Cooperative AI Inference

This material is based upon work supported by the National Science Foundation under Grant Number 2128341.

Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

===

NSF Award #2128341

Collaborative Research: CPS: Medium: RUI: Cooperative AI Inference in Vehicular Edge Networks for Advanced Driver-Assistance Systems

Start Date: October 1, 2021

End Date: September 30, 2025 (One additional year of no-cost extension)

Former PI: Dr. Ning Wang

Current PI: Dr. Shen-Shyang Ho

PIs from other Institutions:

1. Temple University: Dr. Yu Wang (former PI, Dr. Jie Wu)

2. Stony Brook University: Dr. Haibin Ling

3. Kettering University: Dr Jungme Park (former PI, Dr. Yunsheng Wang)

Abstract

Artificial Intelligence (AI) has shown superior performance in enhancing driving safety in advanced driver-assistance systems (ADAS). State-of-the-art deep neural networks (DNNs) achieve high accuracy at the expense of increased model complexity, which raises the computation burden of onboard processing units of vehicles for ADAS inference tasks. The primary goal of this project is to develop innovative collaborative AI inference strategies with the emerging edge computing paradigm. The strategies can adaptively adjust cooperative inference techniques for best utilizing available computation and communication resources and ultimately enable high-accuracy and real-time inference. The project will inspire greater collaborations between experts in wireless communication, edge computing, computer vision, autonomous driving testbed development, and automotive manufacturing, and facilitate AI applications in a variety of IoT systems. The educational testbed developed from this project can be integrated into courses to provide hands-on experiences. This project will benefit undergraduate, master, and Ph.D. programs and increase under-represented groups' engagement by leveraging the existing diversity-related outreach efforts.

A multi-disciplinary team with complementary expertise from Rowan University, Temple University, Stony Brook University, and Kettering University is assembled to pursue a coordinated study of collaborative AI inference. The PIs explore integrative research to enable deep learning technologies in resource-constrained ADAS for high-accuracy and real-time inference. Theory-wise, the PIs plan to take advantage of the observation that DNNs can be decomposed into a set of fine-grained components to allow distributed AI inference on both the vehicle and edge server sides for inference acceleration. Application-wise, the PIs plan to design novel DNN models which are optimized for the cooperative AI inference paradigm. Testbed-wise, a vehicle edge computing platform with V2X communication and edge computing capability will be developed at Kettering University GM Mobility Research Center. The cooperative AI inference system will be implemented, and the research findings will be validated on realistic vehicular edge computing environments thoroughly. The data, software, and educational testbeds developed from this project will be widely disseminated. Domain experts in autonomous driving testbed development, intelligent transportation systems, and automotive manufacturing will be engaged in project-related issues to ensure relevant challenges in this project are impactful for real-world applications.

Participating Graduate/Undergraduate Students:

Former:

1. Asmika Boosarapu (10/1/2021-5/31/2023) MSCS (Thesis)

2. Jack Campanella - BS ECE

3. Matthew McBurney - BS ECE

4. Sarah Ely - BS ECE

5. Lauren Eckert - BS ECE

6. Stephen Piccolo - BS CS

7. Richard Brown - BS ME

8. Gopi Krishna Patapanchala - MSDS

9. Suraj Bitla - MSCS

Current: 

1. Nicholas Bovee - MSCS

2. Izhar Ali - BSCS

Source Code: https://github.com/nbovee/tracr

Publications (Rowan): 

1. Nicholas Bovee, Izhar Ali, Gopi Patapanchala, Suraj Bitla, and Shen Shyang Ho, SplitTracr: A Flexible Performance Evaluation Tool for Cooperative Inference and Split Computing, International Conference on Performance Engineering (ICPE), Toronto, Canada, May 5-9, 2025 

2. Shen-Shyang Ho, Paolo Rommel Sanchez, Nicholas Bovee, Suraj Bitla, Gopi Krishna Patapanchala  and Stephen Piccolo#, Poster: Computation Offloading for Precision Agriculture using Cooperative Inference, 8th IEEE International Conference on Fog and Edge Computing (ICFEC 2024), Philadelphia, PA, May 6-9, 2024

3. Nicholas Bovee, Stephen Piccolo, Suraj Bitla, Gopi Krishna Patapanchala and Shen-Shyang Ho, Poster: SplitTracer: A Cooperative Inference Evaluation Toolkit for Computation Offloading on the Edge, 8th IEEE International Conference on Fog and Edge Computing (ICFEC 2024), Philadelphia, PA, May 6-9, 2024

4. Nicholas Bovee, Stephen Piccolo, Shen Shyang Ho, and Ning Wang Experimental test-bed for Computation Offloading for Cooperative Inference on Edge Devices, EdgeComm: The Fourth Work-shop on Edge Computing and Communications (at ACM/IEEE Symposium on Edge Computing), December 9, 2023, Wilmington, DE

5. Purva Makarand Mhasakar, Kevin Doshi, Ning Wang, Shen Shyang Ho, and Haibin Ling, Distributed Tracking and Verifying: A Real-Time and High-Accuracy Visual Tracking Edge Computing Framework for Internet of Things, EdgeComm: The Fourth Workshop on Edge Computing and Communications (at ACM/IEEE Symposium on Edge Computing), December 9, 2023, Wilmington, DE

6. Shen-Shyang Ho, Paolo Rommel Sanchez, Nicholas Bovee, Suraj Bitla, Gopi Krishna Patapanchala and Stephen Piccolo, Computation Offloading for Precision Agriculture using Cooperative Inference, 8th IEEE International Conference on Fog and Edge Computing (ICFEC 2024), Philadelphia, PA, May 6-9, 2024

7. Nicholas Bovee, Stephen Piccolo#, Suraj Bitla, Gopi Krishna Patapanchala and Shen-Shyang Ho, SplitTracer: A Cooperative Inference Evaluation Toolkit for Computation Offloading on the Edge, 8th IEEE International Conference on Fog and Edge Computing (ICFEC 2024), Philadelphia, PA, May 6-9, 2024

Other Related Publications (Not funded by this project):

1. Lahari Soumya Voleti and Shen-Shyang Ho, Personalized Learning with Limited Data on Edge Devices using Federated Learning and Meta-Learning, EdgeComm: The Fourth Workshop on Edge Computing and Communications (at ACM/IEEE Symposium on Edge Computing), December 9, 2023, Wilmington, DE

Articles:

https://today.rowan.edu/news/2024/05/meet-shen-shyang-ho.html