Data Science Talk Series 

Abstract:

In recent times, many benchmarks have been designed to test whether machine learning models understand natural language. Deep neural nets based on large language models (eg. GPT) have shown impressive performance on these NLU benchmarks, even surpassing humans. DNNs however are inherently opaque, therefore, it is difficult to understand how exactly they make decisions. This has put the field of NLP in a conundrum -- do the models "really" understand language or do they rely on superficial cues to solve NLU tasks? Do the model parameters encode linguistic knowledge or social bias? My approach to this problem is to a) rigorously define the steps humans would take to solve certain NLU problems, b) develop methods to extract "faithful" explanations from DNNs, and c) test the models' alignment with human rationale. Using this framework, I have analyzed multiple LLM-based models on three important NLU tasks (question answering, natural language inference, and fact-checking). This analysis has shown that certain models are more aligned with humans than others, and this alignment is correlated with a model's performance and generalization abilities. My talk will focus on this explanation framework, and some parallel discoveries about how large language models encode linguistic information and bias.