The interaction and collaboration between humans and multiple robots represent a novel field of research known as human multi-robot systems.  Adequately designed systems within this field allow teams composed of both humans and robots to work together effectively on tasks such as monitoring, exploration, and search and rescue operations. This website presents a deep reinforcement learning-based affective workload allocation controller specifically for multi-human multi-robot teams. The proposed controller can dynamically reallocate workloads based on the performance of the operators during collaborative missions with multi-robot systems. The operators' performances are evaluated through the scores of a self-reported questionnaire (i.e., subjective measurement) and the results of a deep learning-based cognitive workload prediction algorithm that uses physiological and behavioral data (i.e., objective measurement). To evaluate the effectiveness of the proposed controller, we use a multi-human multi-robot CCTV monitoring task as an example and carry out comprehensive real-world experiments with 32 human subjects (e.g., 16 teams) for both quantitative measurement and qualitative analysis. Our results demonstrate the performance and effectiveness of the proposed controller and highlight the importance of incorporating both subjective and objective measurements of the operators' cognitive workload, as well as seeking consent for workload transitions, to enhance the performance of multi-human multi-robot teams.

1. Involving more human operators to the system makes overall system complicated and unstable

   • • human operator’s condition can cause human errors and wrong decision-making

2. To maximize the overall system performance, the MH-MR system should understand human condition and manage workloads based on their conditions

   • • Objective measurements (e.g., physiological signal and behavioral features) and subjective measurement (e.g., self-reporting)

3. However, there are still challenges for building the MH-MR team 

   • Standard frameworks for biosensors, limited dataset for human condition, and workload allocation algorithm for MH-MR teams

• We design a data-driven human performance model to estimate the human operator's mission performance from CWL measurements. It can be adapted to various applications by tuning parameters based on empirical experiments.

• We propose a DRL-based AWAC capable of adapting the distribution of workload in response to variations in human cognitive load and team performance.

• We design and conduct an extensive real-world user study in CCTV surveillance scenarios to validate the productivity and effectiveness of the proposed AWAC.

• We investigate and furnish insightful analysis of various workload allocation strategies for MH-MR teams.  

• Wonse Jo; Department of Computer and Information Technology, Purdue University, West Lafayette, IN, USA

• Ruiqi Wang; Department of Computer and Information Technology, Purdue University, West Lafayette, IN, USA 

• Dr. Baijian Yang; Department of Computer and Information Technology, Purdue University, West Lafayette, IN, USA 

• Dr. Dan Foti; the Department of Psychological Sciences, Purdue University, West Lafayette, IN, USA  47906. 

• Dr. Mo Rastgaar; School of Engineering Technology, Purdue University, West Lafayette, IN, 47906.

• Dr. Byung-Cheol Min; Department of Computer and Information Technology, Purdue University, West Lafayette, IN, USA

Please use the following citation:

• Jo, W., Wang, R., Yang, B., Foti, D., Rastgaar, M., & Min, B. C. (2024). Cognitive Load-based Affective workload allocation for multi-human multi-robot teams. IEEE Transactions on Human-Machine Systems (Accepcted). 

• Bibtex format: 

@article{jo2023affective,

  title={Cognitive Load-based Affective workload allocation for multi-human multi-robot teams},

  author={Jo, Wonse and Wang, Ruiqi and Yang, Baijian and Foti, Dan and Rastgaar, Mo and Min, Byung-Cheol},

  journal={IEEE Transactions on Human-Machine Systems},

  year={2024}

}

More details can be found at: 

• NSF CAREER Project Website: https://polytechnic.purdue.edu/ahmrs

• Team-based User Experiment Video: https://youtu.be/qrmAQqfdLZk

• Preprint Paper: https://arxiv.org/abs/2303.10465

• Published paper: (TBD)