NSF CAREER: Modeling, Measuring and Controlling Human Comfort in Human-Autonomous-Machine Interaction (HaMI)

Project PI: Yunyi Jia

Project Duration: 08/2019-08/2024

Project Description: 

The next generation of autonomous machines, including collaborative robots and autonomous vehicles, will interact closely with humans. Despite tremendous efforts to promote safety and efficiency of such machines, low user acceptance remains a critical roadblock to their use. This project addresses the problem by developing techniques to measure human comfort and to modify machine behavior to increase human comfort while interacting with autonomous machines. The resulting advances will facilitate successful deployment of next-generation collaborative robots and autonomous vehicles that humans can interact with comfortably. The theoretical, computational, experimental, and design principles devised will apply to other autonomous machines that human interact with closely, such as surgical robots, domestic robots, and autonomous aircraft. The integrated education plan of this award will empower students, workforces, and the public by engaging them in learning about and helping to design next-generation autonomous machines that interact closely with humans. 

The main objective of this project is to advance the underlying science of human comfort in Human--Autonomous-Machine Interaction (HaMI) and provide transformative solutions and guidelines to design autonomous machines that humans interact with comfortably. The project will advance knowledge of the factors that influence human comfort in HaMI and create new computational models that give quantitative predictions of human comfort. It will create a novel multi-modal measurement paradigm for human comfort that uses the computational models and physiological signals to accurately measure human comfort in real-time. The project will create a novel framework to optimize the behaviors of autonomous machines to improve human comfort. Finally, it will create and disseminate comfort datasets for collaborative robots and autonomous vehicles to facilitate further research. Research-integrated teaching will educate students to understand and design autonomous machines that interact closely with humans. Outreach to K-12 students will inspire youngsters to engage in STEM learning especially in autonomous machines. Integrated academic, industrial and public dissemination will create a consortium to collectively understand, work with, and accept next-generation autonomous machines that interact closely with humans.

Publications: 

• L. Guo and Y. Jia*,  "Bilateral Adaptation of Longitudinal Control of Automated Vehicles and Human Drivers," IEEE Transactions on Intelligent Transportation Systems, 2023.

• H. Su, J. Brooks and Y. Jia*, “Development and Evaluation of Comfort Assessment Approaches for Passengers in Autonomous Vehicles,” SAE Technical Paper, 2023. 

• R. Gangadharaiah, L. Mims, Y. Jia, and J. Brooks, “Opinions from users across the lifespan about autonomous and rideshare vehicles with associated features,” SAE Technical Paper, 2023. 

• L. Mims, R. Gangadharaiah, H. Su, Y. Jia, J. Jacobs, M. Sterling, and J. Brooks, “What makes passengers uncomfortable in vehicles today? An exploratory study of current factors that may influence user acceptance of future vehicles,” SAE Technical Paper, 2023.

• Y. Yan, H. Su and Y. Jia, "Designing Comfortable Robotic System with Human Comfort Analysis and Modeling in Human-Robot Collaboration (HRC)," IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 2023. (Poster)

• Y. Chen, D. Paulius, Y. Sun and Y. Jia*, "Robot Learning of Assembly Tasks from Non-expert Demonstrations using Functional Object-Oriented Network," IEEE Conference on Automation Science and Engineering (CASE), 2022.

• H. Su and Y. Jia*, "Computational Modeling of Human Comfort in Automated Vehicles from Maneuvering Behaviors," IEEE Intelligent Transportation Systems Conference (ITSC), 2022.

• J. Xiang, L. Guo and Y. Jia*, "Comfort Improvement for Autonomous Vehicles using Reinforcement Learning with In-situ Human Feedback," SAE Technical Paper, 2022.

• H. Su and Y. Jia*, "Study of Human Comfort on Autonomous Vehicles Using Wearable Sensors," IEEE Transactions on Intelligent Transportation Systems, pp. 1-15, 2021. 

• L. Guo and Y. Jia*, "Anticipative and Predictive Control of Automated Vehicles in Communication-Constrained Connected Mixed Traffic," IEEE Transactions on Intelligent Transportation Systems, pp. 1-14, 2021. 

• W. Wang, R. Li, Y. Chen, Y. Sun and Y. Jia*, "Predicting Human Intentions by Learning from Demonstrations in Human-Robot Hand-Over Tasks," IEEE Transactions on Automation Science and Engineering, pp. 1-15, 2021.

• L. Guo and Y. Jia*, "Inverse Model Predictive Control (IMPC) based Modeling and Prediction of Human-Driven Vehicles in Mixed Traffic," IEEE Transactions on Intelligent Vehicles, pp. 1-12, 2020.

• Y. Sun, W. Wang, Y. Chen and Y. Jia*, "Learn How to Assist Humans through Human Teaching and Robot Learning in Human-Robot Collaborative Assembly," IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2020.

• W. Wang, Y. Chen and Y. Jia*, "Evaluation and Optimization of Dual-Arm Robot Path Planning for Human-Robot Collaborative Tasks in Smart Manufacturing Contexts," ASME Letters in Dynamic Systems and Control, vol. 1, no. 1, pp. 1-7, 2020.

• L. Guo and Y. Jia*, "Predictive Control of Connected Mixed Traffic under Random Communication Constraints," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020.

• Y. Chen, W. Wang, V. Krovi and Y. Jia*, "Enabling Robot to Assist Human in Collaborative Assembly using Convolutional Neural Networks," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020.

• S. Zhang, Y. Chen, J. Zhang and Y. Jia*, "Real-Time Adaptive Assembly Scheduling in Multi-Robot-Human Collaboration According to Human Capability," IEEE International Conference on Robotics and Automation (ICRA), 2020.

• S. Zhang and Y. Jia*, "Capability-Driven Adaptive Task Distribution for Flexible Multi-Human-Multi-Robot (MH-MR) Manufacturing Systems," SAE Technical Paper, 2020.

• Y. Jia* and B. Ayalew, "Cyber-Human-Physical Heterogeneous Traffic Systems for Enhanced Safety," IEEE International Conference on Connected and Autonomous Driving, 2020.

• W. Wang, Y. Chen, R. Li and Y. Jia*, "Learning and Comfort in Human–Robot Interaction: A Review," Applied Sciences, vol. 9, no. 23, pp. 1-20, 2019.

• R. Li, W. Wang, Y. Chen, and Y. Jia*, "Natural Language and Gesture Perception based Robot Companion Teaching for Assisting Human Workers in Assembly Contexts," ASME Dynamic Systems and Control Conference (DSCC), 2019.

• L. Guo and Y. Jia*, "Modeling, Learning and Prediction of Longitudinal Behaviors of Human-Driven Vehicles by Incorporating Internal Human Decision-Making Process using Inverse Model Predictive Control," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019.

Dataset:

•  Clemson Comfort Dataset: http://cecas.clemson.edu/comfort/