Prof. Jennifer Trueblood, November 4th
Title: Harnessing Human Uncertainty to Improve AI
Abstract:
Artificial intelligence (AI) holds great promise for complex decision-making tasks, such as interpreting medical images. However, human errors during AI development can introduce biases into models and create misalignment between machines and human users. Despite advances in unsupervised machine learning, most systems still rely on human-labeled data -- a massive industry powered by data annotation companies. These companies often aggregate labels from multiple annotators to improve accuracy, leveraging the “Wisdom of the Crowd.” In this talk, I examine how human annotators are subject to systematic biases. These biases can propagate from individuals to crowds to machine learning models. I’ll present cognitive-inspired data engineering methods that correct for these biases using well-established models of human subjective probability judgment. These approaches can improve model accuracy, calibration, and alignment with expert decision-makers. This work underscores the importance of understanding human cognition and decision-making in the training and development of AI systems.
Zoom link: https://us02web.zoom.us/j/89345898509
Prof. Shoham Choshen-Hillel, December 2nd
Title: Sex bias in pain management decisions
Abstract:
In the pursuit of mental and physical health, effective pain management stands as a cornerstone. Here, we examine a potential sex bias in pain management. Leveraging insights from psychological research showing that females’ pain is stereotypically judged as less intense than males’ pain, we hypothesize that there may be tangible differences in pain management decisions based on patients’ sex. Our investigation spans emergency department (ED) datasets from two countries, including discharge notes of patients arriving with pain complaints (N = 21,851). Across these datasets, a consistent sex disparity emerges. Female patients are less likely to be prescribed pain-relief medications compared to males, and this disparity persists even after adjusting for patients’ reported pain scores and numerous patient, physician, and ED variables. This disparity extends across medical practitioners, with both male and female physicians prescribing less pain-relief medications to females than to males. Additional analyses reveal that female patients’ pain scores are 10% less likely to be recorded by nurses, and female patients spend an additional 30 min in the ED compared to male patients. A controlled experiment employing clinical vignettes reinforces our hypothesis, showing that nurses (N = 109) judge pain of female patients to be less intense than that of males. We argue that the findings reflect an undertreatment of female patients’ pain. We discuss the troubling societal and medical implications of females’ pain being overlooked and call for policy interventions to ensure equal pain treatment.
Zoom link: https://us02web.zoom.us/j/85266835470