Guangyuan Jiang1, *, Manjie Xu1, *, Song-Chun Zhu1, 2, Wenjuan Han3, ✉️, Chi Zhang2, ✉️, Yixin Zhu1, ✉️
*equal contribution ✉️corresponding authors
1Peking University 2Beijing Institute for General Artificial Intelligence 3Beijing Jiaotong University
NeurIPS 2023 Spotlight
Standardized and quantified evaluation of machine behaviors is a crux of understanding LLMs. In this study, we draw inspiration from psychometric studies by leveraging human personality theory as a tool for studying machine behaviors. Originating as a philosophical quest for human behaviors, the study of personality delves into how individuals differ in thinking, feeling, and behaving. Toward building and understanding human-like social machines, we are motivated to ask: Can we assess machine behaviors by leveraging human psychometric tests in a principled and quantitative manner? If so, can we induce a specific personality in LLMs?
To answer these questions, we introduce the Machine Personality Inventory (MPI) tool for studying machine behaviors; MPI follows standardized personality tests, built upon the Big Five Personality Factors (Big Five) theory and personality assessment inventories. By systematically evaluating LLMs with MPI, we provide the first piece of evidence demonstrating the efficacy of MPI in studying LLMs behaviors. We further devise a Personality Prompting (P2) method to induce LLMs with specific personalities in a controllable way, capable of producing diverse and verifiable behaviors. We hope this work sheds light on future studies by adopting personality as the essential indicator for various downstream tasks, and could further motivate research into equally intriguing human-like machine behaviors.
If you find our work useful, please cite us:
@inproceedings{jiang2023evaluating,
title={Evaluating and Inducing Personality in Pre-trained Language Models},
author={Jiang, Guangyuan and Xu, Manjie and Zhu, Song-Chun and Han, Wenjuan and Zhang, Chi and Zhu, Yixin},
booktitle={NeurIPS},
year={2023}
}