Human Autonomous Vehicle Interaction

Back to Human-Robot Collaboration

Lionel P. Robert Jr.

Professor 

School of Information 

Core Faculty, Michigan Robotics

University of Michigan

Visit: MAVRIC: Michigan Autonomous Vehicle Research Intergroup Collaboration

Autonomous vehicles have the potential to help provide our society with safer and more efficient driving. Despite this, there are serious concerns about whether drivers will ever choose to fully rely on autonomous vehicles. The most central of these concerns involve issues related to trust, communication and acceptance. 

My research in this area explores highly significant questions related to human interactions with autonomous vehicles. This research has the potential to promote safety, productivity and sustainability. As autonomous vehicles like many other robots become increasingly central to our society it seems timely and important to understand how to promote their interactions with humans.

Magazines and Trade Publications 

Robert, L. P. (2019). AVS And Safety in Forum. Issues in Science and Technology, 35(4), (Summer 2019) pp. 5–19. https://issues.org/forum-43/

Robert, L. P. ( 2019). The Future of Pedestrian-Automated Vehicle Interactions, XRDS: Crossroads, 25(3), pp. 30-33. (pdf), DOI: https://doi.org/10.1145/3313115 article provided by author: http://hdl.handle.net/2027.42/148533 or http://arxiv.org/abs/1904.06417.

Robert, L. P. (2018). Safety and Self-Driving Cars, Medium, https://medium.com/@lprobert/safety-and-self-driving-cars-bd54fe620ee8

Journal Article Publications 

Zhang, Q., Yang, X. J., and Robert, L. P. (2025). Artificial Intelligence Voice Gender, Gender Role Congruity, and Trust in Automated Vehicles, Scientific Reports, forthcoming.

Molaei, S., Robert, L.P. and Banovic, N. (2025). What do people want to know about Artificial Intelligence (AI)? The Importance of Answering End-user Questions to Explain Autonomous Vehicle (AV) Decisions, Proceedings of the ACM on Human-Computer Interaction (PACM-HCI), #, CSCW, Article #,

Du, N., Zhou, F., Tilbury, D.M., Robert, L.P., Yang, X.J. (2024). Behavioral and Physiological Responses to Takeovers in Different Scenarios during Conditionally Automated Driving, TRF: Traffic Psychology and Behaviour, accepted, (pdf), open access: https://dx.doi.org/10.7302/22038.

Zhang, Q., Esterwood, C., Pradhan, A., Tilbury, D., Yang, X. J. and Robert, L. P. (2023). The Impact of Modality, Technology Suspicion, and NDRT Engagement on the Effectiveness of AV Explanations, IEEE Access, 11, 81981-81994 (pdf), open access https://doi.org/10.1109/ACCESS.2023.3302261 copy provided by the author: https://dx.doi.org/10.7302/8000. 

Du, N., Robert, L. P. and Yang, X. J. (2022). A Cross-cultural Investigation of the Effects of Explanations on Drivers’ Trust, Preference, and Anxiety in Highly Automated Vehicles, Transportation Research Record, (pdf), https://doi.org/10.1177/03611981221100528 link to the copy provided by the author https://dx.doi.org/10.7302/4381.

Zhang, Q., Yang, X. J. and Robert, L. P. (2021). What and When to Explain? A Survey of the Impact of Explanations on Attitudes Towards Adopting Automated Vehicles, IEEE Access, 9, 159533 -159540 (pdf), open access: https://doi.org/10.1109/ACCESS.2021.3130489. 

Zhang, Q., Yang, X. J. and Robert, L. P. (2022). Individual Differences and Expectations of Automated Vehicles, International Journal of Human-Computer Interaction, 38(9), 825-836 (pdf), https://doi.org/10.1080/10447318.2021.1970431.

Robert, L. P. (2021). Contextualizing Human‒Automated Vehicle Interactions: A Socio-ecological Framework, Robotics,10(3), 92 open access  https://doi.org/10.3390/robotics10030092.

Jayaraman, S., Robert, L. P., Yang, X.J., Tilbury, D. (2021). Multimodal Hybrid Pedestrian: A Hybrid Automaton Model of Urban Pedestrian Behavior for Automated Driving Applications, IEEE Access, (pdf), https://doi.org/10.1109/ACCESS.2021.3058307.

Zhang, Q., Yang, X. J. and Robert, L. P. (2021). Driver’s Age and Automated Vehicle Explanations, Sustainability, (pdf), link to the copy provided by the author http://hdl.handle.net/2027.42/166295.

Esterwood, C., Yang, X. J., Robert, L. P. (2021). Barriers to AV Bus Acceptance: A US National Survey and Research Agenda, International Journal of Human-Computer Interaction, (pdf), https://doi.org/10.1080/10447318.2021.1886485. link to the copy provided by the author here http://dx.doi.org/10.7302/21.

Azevedo-Sa, H., Zhao, H., Jayaraman, S., Esterwood, C., Yang, X.J., Tilbury, D. Robert, L. P. (2021). How Internal and External Risk Impacts the Relationships between Trust and Diver Behavior in Automated Driving Systems, Transportation Research Part C: Emerging Technologies, (pdf), 123, open access link: https://doi.org/10.1016/j.trc.2021.102973. link to the copy provided by the author: http://hdl.handle.net/2027.42/164966.

Azevedo-Sa, H., Jayaraman, S., Esterwood, C., Yang, X. J., Robert, L. P. and Tilbury, D. (2020). Context-Adaptive Management of Drivers’ Trust in Automated Vehicles, IEEE Robotics and Automation Letters (RA-L), (pdf), DOI:10.1109/LRA.2020.3025736 link to the copy provided by the author: http://hdl.handle.net/2027.42/162571.

Du, N., Zhou, F., Pulver, E., Tilbury, D., Robert, L. P., Pradhan, A. and Yang, X. J. (2020). Predicting Driver Takeover Performance in Conditionally Automated Driving, Accident Analysis and Prevention, (pdf), accepted, link to the copy provided by the author: http://hdl.handle.net/2027.42/156409. 

Azevedo-Sa, H., Jayaraman, S., Esterwood, C., Yang, X. J., Robert, L. P. and Tilbury, D. (2020). Real-Time Estimation of Drivers’ Trust in Automated Driving Systems, International Journal of Social Robotics, (pdf), 13(4), 1911–1927. https://doi.org/10.1007/s12369-020-00694-1 link to the copy provided by the author: http://hdl.handle.net/2027.42/162572.

Du, N., Zhou, F., Pulver, E., Tilbury, D., Robert, L. P., Pradhan, A. and Yang, X. J. (2020). Examining the Effects of Emotional Valence and Arousal on Takeover Performance in Conditionally Automated Driving. Transportation Research Part C: Emerging Technologies, (pdf), 112, pp. 78-87, open access link: DOI: https://doi.org/10.1016/j.trc.2020.01.006,  author's copy: http://hdl.handle.net/2027.42/152470  and  http://arxiv.org/abs/2001.04509.

Jayaraman, S.K., Chandler, C., Tilbury, D.M., Yang, X.J., Pradhan, A.K., Tsui, K.M. and Robert, L.P. (2019), Pedestrian Trust in Automated Vehicles: Role of Traffic Signal and AV Driving Behavior, Frontiers in Robotics and AI, (pdf), 6(117), open access link: DOI:10.3389/frobt.2019.00117, author's copy: http://hdl.handle.net/2027.42/151794.

Du, N., Haspiel, J., Zhang, Q., Tilbury, D., Pradhan, A., Yang, X. J. and Robert, L. P. (2019). Look Who’s Talking Now: Implications of AV’s Explanations on Driver’s Trust, AV Preference, Anxiety and Mental Workload, Transportation Research Part C: Emerging Technologies, (pdf), 104, pp. 428-442,open access link: https://doi.org/10.1016/j.trc.2019.05.025, link provided by the author: http://hdl.handle.net/2027.42/149154 and http://arxiv.org/abs/1905.08878.

Petersen, L., Robert, L. P., Yang, X. J. Tilbury, D. (2019). Situational Awareness, Driver’s Trust in Automated Driving Systems and Secondary Task Performance, SAE International Journal of Connected and Automated Vehicles, 2(2), (pdf), DOI:10.4271/12-02-02-0009, open access link: https://saemobilus.sae.org/content/12-02-02-0009/ and copy provided by the author: http://hdl.handle.net/2027.42/148141.

Robert, L. P. (2019). Are Automated Vehicles Safer than Manually Driven Cars?, AI & Society, (pdf), link to publisher's site: https://doi.org/10.1007/s00146-019-00894-y copy provided by the author: http://hdl.handle.net/2027.42/149146 and http://arxiv.org/abs/1905.07054.  

Conference Publications 

Han, D. W., Liu, J., Yang, X. J., Romo, A., Horrey, W., Tilbury, D., Zhou, F., Robert, L., & Molnar, L. (2024). Supporting Driver Attention Toward Potential Hazards During Takeover: A Preliminary Result. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, https://doi.org/10.1177/10711813241275917.

Han, D. W., Liu, J., Yang, X. J., Romo, A., Horrey, W., Tilbury, D., Zhou, F., Robert, L., & Molnar, L. (2024). Toward Integrated Takeover Performance Measurement: Validation of Fréchet Distance as a Takeover Performance Metric. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 0(0). https://doi.org/10.1177/10711813241266831.

Ali, A., Banerjee, R., Jo, W., Dubrow, S., Riegner, K., Smereka, J.M., Robert, L.P., Tilbury, D. (2024). Spot Report: An Open-Source and Real-Time Secondary Task For Human-Robot Interaction User Experiments, Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction (HRI 2024), March 11-15, 2024 in Boulder, Colorado, USA.

Esterwood, C., Ali, A., George, Z., Dubrow, S., Smereka, J., Riegner, K., Tilbury, D. and Robert, L.P. (2023). Promises and Trust Repair in UGVs, Proceedings of the HFES 67th Annual Meeting of the Human Factors and Ergonomics Society (HFES 2023), (pdf), October 23-27, 2023, Washington, DC. https://doi.org/10.1177/21695067231196235 Open access link: https://dx.doi.org/10.7302/7979.

 Zhang, Q., Yang, X. J. and Robert, L. P. (2023). Finding the Right Voice: Exploring the Impact of Gender Similarity Between Human and Automated Vehicle Voice and Gender-Role Congruity on the Efficacy of Automated Vehicle Explanations, Proceedings of the Fall 2023 AAAI Symposium, 2(1), (pdf), October 25-27, 2023, Arlington, VA, USA. https://doi.org/10.1609/aaaiss.v2i1.27675.

Du, N., Zhou, F., Tilbury, D., Robert, L. P. & Yang, X. J. (2021). Designing Alert Systems in Takeover Transitions: The Effects of Display Information and Modality, Proceedings of 13th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutoUI 2021), (Acceptance rate: typically average 43%) Sep. 9-14, 2021, Virtual Conference.

Robinson, K. P., Robert, L. P., Eglash, R. (2021). Extrapolating Significance of Text-based Autonomous Vehicle Scenarios to Multimedia Scenarios and Implications for User-centered Design, Proceedings of 30th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2021), (pdf), Aug 08 - Aug 12, 2021. Online Virtual Conference (originally in Vancouver, BC, Canada), https://doi.org/10.1109/RO-MAN50785.2021.9515372 open access link http://hdl.handle.net/2027.42/168412.

Yining Cao, Y., Zhou, F., Molnar, L., Robert,  L.P., Tilbury, D.M. and Yang. X. J. (2021). Towards Standardized Metrics for Measuring Takeover Performance in Conditionally Automated Driving: A  Literature Review, Proceedings of the 65st Annual Meeting of the Human Factors and Ergonomics Society (HFES 2021), October 4 - 7, 2021 at the Baltimore Marriott Waterfront Hotel.

Du, N., Kim, J., Zhou, F., Pulver, E., Tilbury, D., Robert, L. P., Pradhan, A., & Yang, X. J. (2020). Evaluating Effects of Cognitive Load, Takeover Request Lead Time, and Traffic Density on Drivers’ Takeover Performance in Conditionally Automated Driving, Proceedings of 12th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutoUI 2020), (pdf), September 21-22, 2020, Washington, DC, USA, link provide by the author: http://hdl.handle.net/2027.42/156045.

Du, N., Zhou, F., Pradhan, A., Tilbury, D., Robert, L. P., & Yang, X. J. (2020). Examining Effects of Scenario Type and Vehicle Speed on Takeover Readiness and Performance in Conditionally Automated Driving, Proceedings of the 64st Annual Meeting of the Human Factors and Ergonomics Society (HFES 2020), October 5-9, 2020, Chicago.

Jayaraman, S.K., Tilbury, D.M., Yang, X.J., Pradhan, A.K., and Robert, L.P. (2020). Analysis and Prediction of Pedestrian Crosswalk Behavior during Automated Vehicle Interactions, Proceedings of the International Conference on Robotics and Automation (ICRA 2020), (pdf), May 31-June 4, 2020, Paris, France, link provide by the author: http://hdl.handle.net/2027.42/154053.

Jayaraman, S., Robert, L.P., Yang, X.Y., Pradhan, A., Tilbury, D. Efficient Behavior-aware Control of Automated Vehicles at Crosswalks using Minimal Information Pedestrian Prediction Model, Proceedings of the American Control Conference (ACC 2020), July 1-3, 2020, Denver, CO, USA, (pdf), link provided by the author: http://hdl.handle.net/2027.42/154113.

Zhang, Q., Yang, X. J. and Robert, L. P. (2020). Expectations and Trust in Automated Vehicles, In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI 2020), (pdf), April 25-30, 2020, Honolulu, Hawaii, USA. http://dx.doi.org/10.1145/3334480.3382986,  Link provided by the author http://hdl.handle.net/2027.42/153795.

Du, N., Zhou, F., Pulver, E., Tilbury, D., Robert, L.P., Pradhan, A., and Yang, J.X. (2020). Predicting Takeover Performance in Conditionally Automated Driving, In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI 2020) (pdf), April 25-30, 2020, Honolulu, Hawaii, USA. https://doi.org/10.1145/3334480.3382963 and copy provided by the author: http://hdl.handle.net/2027.42/153789.

Azevedo-Sa, H., Jayaraman, S., Esterwood, C., Yang, X.J. and Robert, L. P. and Dawn M. Tilbury. 2020. Comparing the Effects of False Alarms and Misses on Humans’ Trust in (Semi)Autonomous Vehicles. In Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (HRI 2020), (pdf), March 23 26, 2020, Cambridge, United Kingdom..ACM, New York, NY, USA. https://doi.org/10.1145/3371382.3378371 and copy provided by the author http://hdl.handle.net/2027.42/153524.

Zhao, H., Azevedo-Sa, H., Esterwood, C., Yang, X. J., Robert, L. P., Tilbury, D.  2019. Error Type, Risk, Performance, and Trust: Investigating the Different Impacts of false alarms and misses on Trust and Performance. Proceedings of the Ground Vehicle Systems Engineering and Technology Symposium (GVSETS 2019), Novi, MI, (pdf), Aug. 13-15, 2019 author's copy: http://hdl.handle.net/2027.42/149648.

Du, N., Ayoub, J., Zhou, F., Pradhan, A., Robert, L. P., Tilbury, D., & Yang, X. J. (2019). Examining the Impacts of Drivers' Emotions on Takeover Readiness and Performance in Highly Automated Driving. Proceedings of the 63st Annual Meeting of the Human Factors and Ergonomics Society (HFES 2019), 63(1), pp. 2076-2077, October 28 - November 1, 2019 in Seattle, WA (pdf), https://doi.org/10.1177%2F1071181319631391 link provided by the author: http://hdl.handle.net/2027.42/149482.

Petersen, L., Zhao,  H., Tilbury, D., Yang, X. J. and Robert, L.P.  (2018). The Influence of Risk on Driver's Trust in Semi-Autonomous Driving, Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS 2018), Novi, MI (pdf) and link to the paper http://hdl.handle.net/2027.42/144567.

Jayaraman, S.K.,  Creech, C., Robert, L. P., Tilbury, D., Yang, X. J., Pradhan, A. and Tsui, K. (2018). Trust in AV: An Uncertainty Reduction Model of AV-Pedestrian Interactions, Proceedings of the Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction (HRI 2018), March 5–8, 2018, (pdf), Chicago, IL, USA, https://doi.org/10.1145/3173386.3177073 and the copy provided by the author: http://hdl.handle.net/2027.42/140747.

Haspiel, J., Du, N., Yang, X. J., Tilbury, D., Pradhan, A., Robert, L. P., (2018). Explanations and Expectations: Trust Building in Automated Vehicles, Proceedings of the Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction (HRI 2018), March 5–8, 2018, (pdf), Chicago, IL, USA, https://doi.org/10.1145/3173386.3177057 and the copy provided by the author: http://hdl.handle.net/2027.42/140746.

Petersen, L., Tilbury, D., Robert, L.P., (2017). Effects of Augmented Situational Awareness on Drive Trust in Semi-Autonomous Vehicle Operation, presented during the Autonomous Ground Systems (AGS) Technical Session of the 2017 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS 2017), Novi, MI (pdf) and Link to the paper http://hdl.handle.net/2027.42/137707.

Technical Reports

Han, D., Liu, J., Zhou, F., Tilbury, D., Robert, L.P., Molnar, L., and Yang, X. J. (2023). Measuring and Predicting Drivers’ Takeover Readiness and Supporting Takeover Transitions in Automated Driving, AAA Foundation for Traffic Safety Emerging Technologies Technical Report, Washington, DC. https://trid.trb.org/View/2259937.

Symposium, Workshop & Unpublished Papers

Ali, A. Azevedo-Sa, H., Tilbury, D. and Robert, L. P. (2022). Perceptions of Task Allocation Methods for Human-Robot Teams, IEEE International Conference on Robotics and Automation (ICRA 2022), Workshop on Collaborative Robots on Collaborative Robots and the Work of the Future, May 23, 2022, Philadelphia, PA. 

Zhang, Q., Yang, X. J. and Robert, L. P. (2021). From the Head or the Heart? The Impact of Explanation on Cognitive and Affective Trust, 2021 AAAI Fall Symposium on AI for HRI, Virtual, November 4-6, 2021. 

Azevedo-Sa, H., Yang, X. J., Robert, L.P. and Tilbury, D. (2021). Using Trust in Automation to Enhance Driver-(Semi)Autonomous Vehicle Interaction and Improve Team Performance, 2021 International Symposium on Transportation Data and Modelling, (pdf), Virtual, June 21-24. http://dx.doi.org/10.7302/1288.

Jayaraman, S., Robert, L. P., Yang, X.J., Tilbury, D. (2021). Automated Vehicle Behavior Design for Pedestrian Interactions at Unsignalized Crosswalks, 2021 International Symposium on Transportation Data and Modelling, (pdf), Virtual, June 21-24. http://dx.doi.org/10.7302/1287.

Azevedo-Sa, H., Yang, X.J. and Robert, L. P. and Dawn M. Tilbury. (2020). Using Trust in Automation to Enhance Driver-(Semi)Autonomous Vehicle Interaction and Improve Team Performance, International Symposium on Transportation Data and Modelling (ISTDM 2020), (pdf), June 24 - 26, 2020, Ann Arbor, Michigan, USA.

Jayaraman, S., Robert, L.P., Yang, X.Y., Tilbury, D. (2020). Automated Vehicle Behavior Design for Pedestrian Interactions at Unsignalized Crosswalks, International Symposium on Transportation Data and Modelling (ISTDM 2020), (pdf), June 24 - 26, 2020, Ann Arbor, Michigan, USA.

Na Du, N., Kim, J.Zhou, F., Tilbury, D., Robert, L. P., Pradhan, A.K. and Yang, X.J. Taking Over Control From Highly Automated Vehicles in Complex Road Situations: The Role of Drivers’ Cognitive Load to be presented at the 2019 Engineering Research Symposium, Ann Arbor, MI.

Zhang, Q., Esterwood, C., Yang, X. J. and Robert, L. P.  (2019). An Automated Vehicle (AV) like Me? The Impact of Personality Similarities and Differences between Humans and AVs presented at AAAI Fall Symposium on Artificial Intelligence for Human-Robot Interaction, (pdf), Arlington, Virginia USA, November 7-9, 2019. http://hdl.handle.net/2027.42/150653. 

Zhang, Q. et al. (2019). Communication Modality and Automated Vehicle Acceptance presented at the 3rd IAVSD Workshop on Dynamics of Road Vehicles: Connected and Automated Vehicles, April 28-29, 2019 in Ann Arbor, MI, USA. 

Du, N. et al. (2019). Effects of Driver's Emotion on Takeover Readiness and Performance in Highly Automated Driving presented at the 3rd IAVSD Workshop on Dynamics of Road Vehicles: Connected and Automated Vehicles, April 28-29, 2019 in Ann Arbor, MI, USA. 

Jayaraman, S.K., Tilbury, D., Yang, X. J., Pradhan, A. and Robert, L. P. (2019). A Hybrid Framework for Improved AV-Pedestrian Predictions presented at the 3rd IAVSD Workshop on Dynamics of Road Vehicles: Connected and Automated Vehicles, April. 28-29, 2019 in Ann Arbor, MI, USA. 

Zhao, H. et al. (2019). Trust and Risk in Autonomous Ground Systems presented at the 3rd IAVSD Workshop on Dynamics of Road Vehicles: Connected and Automated Vehicles, April 28-29, 2019 in Ann Arbor, MI, USA. 

Jayaraman, S.K.,  Creech, C., Tilbury, D., Yang, X. J., Pradhan, A. , Tsui, K. and Robert, L. P. (2018). Workload in Pedestrians Interacting with Autonomous Vehicles presented at the 2nd IFAC Conference on Cyber-Physical & Human-Systems, Dec. 14-15, 2018 in Miami, FL, USA. 

Zhang, Q. Na, D. Robert, L. P.,Yang, X. J. (2018). Trust in AVs: The Impact of Expectations and Individual Differences presented at the Conference on Autonomous Vehicles in Society: Building a Research Agenda, May 18-19 2018, East Lansing, MI, (pdf), Copy provided by the author: http://hdl.handle.net/2027.42/142567.

Na Du, Robert, L. P., Pradhan,, A., Tilbury, D. and Yang, X. J. (2018). A Cross-Cultural Study of Trust Building in Autonomous Vehicles presented at the Conference on Autonomous Vehicles in Society: Building a Research Agenda, May 18-19 2018, East Lansing, MI, (pdf), Copy provided by the author: http://hdl.handle.net/2027.42/142564.

Yang, X. J., Tilbury, D., Pradhan, A., Robert, L. P. (2018). Interacting with Autonomous Vehicles: An Interdisciplinary Approach presented at the Workshop on the Interacting with Autonomous Vehicles: Learning from other Domains  at 36rd ACM Conference on Human Factors in Computing Systems (CHI 2018) (pdf), April 21-26, 2015, Montreal, Canada, Copy provided by the author: http://hdl.handle.net/2027.42/142722.

Creech, C., Jayaraman, S.K., Robert, L. P., Tillbury, D., Yang, X. J., Pradhan, A. and Tsui, K. (2017). Trust and Control in Autonomous Vehicle Interactions, to be presented at the Morality and Social Trust in Autonomous Robots workshop at the 2017 Robotics: Science and Systems (RSS 2017), Cambridge, MA. (pdf) and (poster) Link to the paper and poster http://hdl.handle.net/2027.42/137661.