Traditional research on trust in automation often relies on signal detection theory (SDT) to model autonomy performance, which oversimplifies complex real-world scenarios into binary outcomes. By moving beyond the SDT to account for multi-class classifications, we uncovered previously unexplored error patterns. Notably, AI's false reassurance decreased performance and trust, while AI errors that inadvertently prompted safety checks paradoxically improved performance [J1]. This work has been selected as the Human Factors Journal editor's pick.
In a healthcare application, we investigated pharmacists' trust in pill-verification AI designed to reduce medication dispensing errors. This research demonstrated that pharmacists have a positive predisposition to trusting AI. Presenting AI uncertainty information increased trust when AI was correct; however, it amplified a "negativity bias" (greater drop in trust) following AI errors [J2].
REFEREED JOURNAL PUBLICATIONS
[J1] Kim, J.Y., Lester, C., & Yang, X.J. (2025). Beyond Binary Decisions: Evaluating the Effects of AI Error Type on Trust and Performance in AI-Assisted Tasks. Human Factors, 0(0). [DOI].
[J2] Kim, J.Y., Marshall, V., Rowell, B., Chen, Q., Zheng, Y., Lee, J.D., Kontar, R., Lester, C., & Yang, X.J. (2025). The Effects of Presenting AI Uncertainty Information on Pharmacists’ Trust in Automated Pill Recognition Technology: Exploratory Mixed Subjects Study. JMIR Hum Factors;12:e60273. [DOI].
[J3] Tsai, C., Kim, J.Y., Chen, Q., Rowell, B., Yang, X.J., Kontar, R., Whitaker, M., & Lester, C. (2025). The Effect of Artificial Intelligence Helpfulness and Uncertainty on Cognitive Interactions with Pharmacists: A Randomized Controlled Trial. J Med Internet Res;27:e59946. [DOI].
[J4] Lester, C., Rowell, B., Zheng, Y., Co, Z., Marshall, V., Kim, J.Y., Chen, Q., Kontar, R., & Yang, X. (2025). Effect of Uncertainty-Aware AI Models on Pharmacists’ Reaction Time and Decision-Making in a Web-Based Mock Medication Verification Task: Randomized Controlled Trial. JMIR Med Inform 2025;13:e64902. [DOI]
[J5] Zheng, Y., Rowell, B., Chen, Q., Kim, J.Y., Al Kontar, R., Yang, X. J., & Lester, C. A. (2023). Designing Human-Centered AI to Prevent Medication Dispensing Errors: Focus Group Study With Pharmacists. JMIR Formative Research, 7(1), e51921. [DOI].
REFEREED CONFERENCE PROCEEDINGS
[C1] Kim, J., Kautz, C., & Yang, X. J. (2024). Evaluating the Effects of Consequences of Error on Trust in AI-Aided Task: An Eye Tracking Study. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (HFES) 69th Annual Meeting. [DOI].
[C2] Kim, J., Richie, O., & Yang, X. J. (2024). Evaluating The Influence of Incorrect Reassurances on Trust in Imperfect Automated Decision Aid. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (HFES) 68th Annual Meeting. [DOI].
[C3] Kim, J., Chen, S. T., Hannan, J., Larson, H., Chung, H., Jain, T., Fields, M., Ulin, S., Stirling, L., & Yang, X. J. (2024). ErgoNomiCs and Human-Automation iNteracTion (ENCHANT) Summer Camp (Evaluation). In 2024 ASEE Annual Conference & Exposition. Portland, OR. June. [DOI].
[C4] Kim, J., Chen, S. T., Lester, C., & Yang, X. J. (2023). Exploring Trust and Performance in Human-Automation Interaction: Novel Perspectives on Incorrect Reassurances from Imperfect Automation. AHFE International Conference. [DOI].
[C5] Du, N., Kim, J., Zhou, F., Pulver E., Tilbury, D., Robert, L. P., Pradhan, A., & Yang, X. J. (2020). Evaluating Effects of Cognitive Load, Takeover Request Lead Time, and Traffic Density on Drivers’ Takeover Performance in Conditionally Automated Driving. 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ’20). September 21–22, 2020, Virtual Event, DC, US. [DOI].
[C6] Du, N., Kim, J., Zhou, F., Tilbury, D., Robert, L. P., Pradhan, A., & Yang, X. J. (2020). Examining effects of scenario type and vehicle speed on takeover readiness and performance in conditionally automated driving. In Proceedings of the 64th Annual Meeting of the Human Factors and Ergonomics Society (HFES). Chicago, IL. October. [DOI].