Schedule

Introduction to the Workshop

Keynote by Elena Glassman

Interfaces for Better Characterizing and Leveraging Large Language Models

Abstract: The behavior of a given large language model (LLM) is difficult to fully and globally characterize, due in large part to its large input and output spaces and enormous, often unspecified training data; their stochastic behavior can surprise users on previously seen and unseen inputs. And yet, even early on, their potential utility in data work was enticing. This talk will describe recent projects that help those wielding LLMs develop more local empirically grounded mental models of one or more LLMs' behavior. The talk will conclude with a discussion of interface design considerations and pitfalls to avoid when building LLM-powered features for assisting with data work.

Bio: Elena L. Glassman is currently an Assistant Professor of Computer Science at the Harvard Paulson School of Engineering & Applied Sciences, specializing in human-computer interaction. She was the Stanley A. Marks & William H. Marks Professor at the Radcliffe Institute for Advanced Study from 2018 - 2022, and more recently received a 2023 Sloan Research Fellowship. At MIT, she earned a PhD and MEng in Electrical Engineering and Computer Science and a BS in Electrical Science and Engineering, supported by the NSF Graduate Research Fellowship and the NDSEG Graduate Fellowship. Before joining Harvard, she was a postdoctoral scholar in Electrical Engineering and Computer Science at the University of California, Berkeley, where she received the Berkeley Institute for Data Science Moore/Sloan Data Science Fellowship.