Clinical Adaptive Performance Enhancement through Human-AI Teaming (CAPE-HAT)

This project aims to develop a cognitive assistant to support medical professionals in performing Acute Cardiac Life Support (ACLS) by leveraging, leveraging artificial intelligence (AI) techniques to analyze diverse data sources, including cognitive and emotional states, behavioral patterns, and environmental conditions. Error prediction will leverage AI to determine assistive interventions for HPs that will contribute to the avoidance or reduction of medical errors. SCEL is collaborating with Drs. Sekan Toy of VT-SOM and Dawei Zhou of CS, and Drs. Pappada and Cheng at University of Toledo to conduct this interdisciplinary research at the nexus of human performance, sensing technologies, and human-AI collaboration to identify cognitive and affective states, behavioral patterns, and contextual factors contributing to medical errors. The research will advance our understanding on  the impact of neurophysiological processes on medical error rates, (2) multi-modal representation learning to predict healthcare professionals' cognitive states and potential errors, and (3) adaptive AI interactions considering human readiness and cognitive resources.

Sponsors: National Science Foundation - Smart and Connected Health

Principal Investigator: Dr. Serkan Toy, VT SOM

Co-Principal Investigator(s): Drs. Lau & Zhou, VT; Drs. Pappada & Cheng,

Industrial Partner(s): Carilion Clinic 

SCEL member(s): David Du

Project start date: Oct 2024

Studying hospital soundscapes with the Tesseract

This project aims to study soundscapes in different areas of the hospital and how the soundscapes can impact clinician performance and patient health.  Besides field studies, the research team will also employ run control experiments with th using the Tesseract,  a portable, immersive multimodal simulation environment co-developed by Virginia Tech’s Institute for Creativity, Arts, and Technology (ICAT) and Carilion Clinic’s Center for Simulation, Research, and Patient Safety. 

Sponsors: Carilion Clinic and Virginia Tech Institute for Creativity, Arts, and Technology

Principal Investigator: Dr. Ojeifo

Co-Principal Investigator(s): Drs. Price & Parker

Industrial Partner(s): Carilion Clinic 

SCEL member(s): David Du

Project start date: Aug 2024

Leadership Emergence for Synchronous Human-Autonomy Teaming (LEESH)

This project aims to study the emergent nature of leadership and cognitive synchronization for integrating autonomy into multi-agent organizations. SCEL is collaborating with Dr. Williams of ECE and Dr. Foti of Psychology, to apply implicit leadership theory, which predicts that leading and following dynamics in human teams tend to result in granting leadership to the most competent and trustworthy member regardless of a pre-defined authority structure. To leverage this theory, we will develop methods to support cognitive synchronization across diverse tasks according to skill-, rule-, and knowledge-based behaviors that classify human cognition and underlie assessments of trust. Reinforcement learning approaches will be developed for selecting interaction methods and improving interaction choices to promote long-term, productive collaboration. These methods will be evaluated in computer simulations and field experiments with unmanned vehicles and human teammates in the context of search and rescue missions. The design principles and algorithms will be applicable to many domains involving command and control with humans in the loop, such as disaster response, air traffic control, and patient flow management.

Sponsor: National Science Foundation - Human Center Computing

Principal Investigator: Dr. Nathan Lau

Co-Principal Investigator(s): Drs. Ryan Williams & Roseanne Foti

Industrial Partner(s): Black Diamond Search & Rescue Council 

SCEL member(s): Anirudh More

Project start date: Jan 2023

Intelligent virtual coach for self-training manual skills in laparoscopy

This project aims to study how medical professionals improve their proficiency levels from self-training in laparoscopic surgery for developing the next generation of training simulator. SCEL is collaborating with Dr. Sarah Parker, Health Systems and Implementation Science at Virginia Tech, and Dr. Laura Barnes, Univeristy of Virginia, to employ machine learning techniques of multi-modal sensor data for developing artificial intelligence that can guide novices in self-training for immediate feedback, faster learning, and potentially better surgical skills. We will be collecting data and seeking inputs from medical students, residents and attendings at the Carilion Clinic.

Sponsor: National Institute of Health -  iTHRIV

Principal Investigator: Dr. Lau

Co-Principal Investigator(s): Drs. Sarah Parker, Laura Barnes & Shawn Safford

Industrial Partner(s): Carilion Clinic 

SCEL member(s): Shiyu Deng, Chaitanya Kulkarni & Tianzi Wang

Project start date: Jun 2019

Examining senior drivers adaptation to mixed-level automated vehicles: A naturalistic approach

This project aims to study the impact of common in-vehicle technology (e.g., blind spot alert) on driving behaviors of elders (aged higher than 65 years old). VACSE is collaborating with Dr. Antin of Virginia Tech Transportation Institute to investigate the attitudes and behaviors of seniors when driving vehicles on real roads while using common automated features including blind sport alert, lane assist, lane keep alert, and adaptive cruise control. We are taking a naturalistic approach in collecting our data from seniors driving in their daily lives. This research hopes to shed light on whether seniors can take advantage of various common automated features to promote their safety on the road.

Sponsor: Safe-D, National University Transportation Center

Principal Investigator: Dr. Jon Antin

Co-Principal Investigator(s): Dr. Nathan Lau

SCEL member(s): Dan Liang

Project start date: Jan 2018

Distributed situation awareness in patient flow management

This project aims to study situation awareness in patient flow management. SCEL is collaborating with Carilion Clinic to model how situation awareness is distributed amongst workers in transitioning patients through various stages of their medical care from admission to discharge at the Carilion Roanoke Memorial Hospital. We are adopting the distributing situation awareness framework for modeling how information is acquired, processed and transacted across stakeholders in the hospitals. This research hopes to innovate principles for designing a command and control center in optimizing patient flow, thereby improving efficiencies in healthcare delivery.

Sponsor: NA

Principal Investigator: Dr. Nathan Lau

Industrial Partner(s): Carilion Clinic 

SCEL member(s): Abdul Alhaider

Project start date: Jan 2018

Reinforcing competence in cybersecurity of critical infrastructures: a Norway-US partnership (RECYCIN)

This project aims to broaden the collaboration in research and education between research institutes in Norway and Virginia Tech. SCEL is working with Institute for Energy Technology, Østfold University College, and Norwegian University of Science and Technology to establish a collaboration network to share our education and research materials and pursue cybersecurity projects together in an international setting.

Sponsor: Research Council of Norway

Principal Investigator: Dr. Vikash Katta

Co-Principal Investigator(s): Drs. Nathan Lau

Industrial Partner(s): Institute For Energy Technology 

SCEL member(s): Hao Wang & Po-Yu (Frank) Chen

Project start date: Jun 2020 - Jun 2024

Development of a novel multi-modal in-situ detection system supporting human-machine collaboration in core monitoring and control of advanced reactors 

This project aims to develop an in-situ detection and monitoring system with a physics-based Machine Learning (ML) algorithm to infer nuclear reactor core physics data with high fidelity and facilitate human-machine interaction for next generation nuclear reactors. SCEL is collaborating with Drs. Alireza Haghighat, ME , and Jonathan Links, Physics at Virginia Tech to develop a software interface to couple the CHANDLER multi-modal detection system and the VRS-RAPID (Virtual Reality System for Real-time Analysis for Particle transport and In-situ Detection) neutronics code system, and expand on the exiting ML algorithm and virtual reality visualization framework to provide an effective means for the human-machine collaboration. This in-situ monitoring system, with its ML algorithm and interactive visualization, will significantly contribute to improved safety and efficacy by making system adjustments in response to the data generated by the proposed in-situ detection system. 

Sponsor: Nuclear Regulatory Commission 

Principal Investigator: Dr. Alireza Haghighat

Co-Principal Investigator(s): Drs. Jonathan Link & Nathan Lau

Industrial Partner(s): Dominion Power

Project Period: Oct 2021 - Sep 2024

Whole-body Exoskeletons for Advanced Vocational Enhancement (WEAVE)

This project aims to study natural user interface with powered, full-body exoskeletons for commercial industries (e.g., manufacturing and construction) and the impact of such technology on future workforce development. SCEL is collaborating with Drs. Nussbaum and Srinivasan of ISE, Dr Leonessa of ME, and Dr. Ge of Economics at Virginia Tech to develop control algorithms adaptive to individual differences of the users and unique characteristics of the situation for operating exoskeletons. Sarcos is our industrial partner providing and refining their powered exoskeletons for testing in support of our research. This research hopes to shed light on how to augment human physical capacity, thereby increasing productivity and lowering injury risks while preserving human skill for operating in dynamic, unstructured environments.

Sponsor: National Science Foundation - Future of Work: Human-Technology Frontier

Principal Investigator: Dr. Divya Srinivasan

Co-Principal Investigator(s): Drs. Maury Nussbaum, Nathan Lau, Alexander Leonessa, & Suqin Ge

Industrial Partner(s): Sarcos 

SCEL member(s): Hsiang-Wen (Gram) Hsing, Chaitanya Kulkarni & Mark Hollister

Project period: Sep 2018 - Aug 2023

Balancing collaboration and autonomy for multi-robot multi-human search and rescue

This project aims to study human-robot interaction and user interface design for extending the capabilities of search and rescue (SAR) workers. SCEL is collaborating with Dr. Williams of ECE, Dr. Abaid of BEAM, and Dr. McClure of ARC at Virginia Tech to enable unmanned aerial vehicles (UAVs) to search for lost person autonomously and collaborate with SAR workers on search mission. We are designing novel planning and exploration-exploitation algorithms that integrate human inputs for tasking SAR teams and optimizing their search paths. Further, we will be testing our innovative engineering solutions with professional SAR workers in full-scale mock searches with the support of the Black Diamond Search and Rescue Council. This research hopes to extend the autonomous capability and human-interactivity of UAVs for reducing search time of a lost person.

Sponsor: National Science Foundation - National Robotics Initiative 2.0

Principal Investigator: Dr. Ryan Williams

Co-Principal Investigator(s): Drs. Nicole Abaid, Nathan Lau, & James McClure

Industrial Partner(s): Black Diamond Search & Rescue Council; Robert J. Koester 

SCEL member(s): Tianzi Wang & Subhash Holla

Project period: Sep 2018 - Aug 2023

Developing AR-guided skill training to improve the performance and productivity of reach truck drivers

This project aims to study how reach truck drivers build their expertise for developing training programs that would maximize learning of novices. SCEL will be collaborating with Drs. Sol Lim, Sunwook Kim, and Maury Nussbaum, ISE, to compare performance between novice and expert drivers to understand factors driving performance in reach truck operations, and apply the findings for developing an AR-guided interactive tutorial. We will be working with the Raymond Corporation to use their simulators and recruit their instructors for our empirical study and tutorial design. This research hopes to improve the skill acquisition of reach truck drivers, thereby improving productivity and safety of warehouses. 

Sponsor: Raymond Corporation

Principal Investigator: Dr. Sol Lim

Co-Principal Investigator(s): Drs. Maury Nussbaum, Nathan Lau, Sunwook Kim

Industrial Partner(s): Raymond Corporation 

SCEL member(s): Hsiang-Wen (Gram) Hsing & Tianzi Wang

Project period: Jun 2022 - Jun 2023