Social Simulation with LLMs
@ COLM 2025

Important Information

Workshop Date: October 10, 2025 

Workshop Room:  TBC
Workshop Time Zone: Montreal (UTC -4)

Contact Email: social-simulation@googlegroups.com 

Theme

In an era where digital interactions shape our social fabric, leveraging LLMs for social simulation offers unique lens to understand and influence complex societal dynamics. Numerous recent works have demonstrated the potential of LLMs in modeling complex decision-making processes [1, 5], emulating human-like interactions [3, 5],  and tracking social evolution [4, 7].  For instance, [1, 5, 8]  showcase scalable frameworks for simulating intricate electoral and societal dynamics. Complementary research, such as [4, 8], further illustrates how varying worldviews and cultural trajectories can be captured using state-of-the-art LLMs.  These advancements in social simulation—whether modeling entire societies or social media ecosystems—provide unprecedented opportunities to understand human collective behavior.  However, these capabilities also raise important ethical, social, and policy considerations such as social identity biases [2] and the risk of misrepresenting or flattening identity dynamics through LLM-driven simulations [6]. 

Our workshop invites researchers from  researchers, practitioners, and thought leaders to explore innovative social simulation techniques, serving as a meeting point for diverse communities spanning agent-based modeling, social psychology, and ethical technology design. Participants will share insights on emergent social behaviors, LLM interaction patterns, and ethical implications while collaboratively addressing the field's key challenges.

[1] Altera. AL, Andrew Ahn, Nic Becker, Stephanie Carroll, Nico Christie, Manuel Cortes, Arda Demirci, Melissa Du, Frankie Li, Shuying Luo, Peter Y Wang, Mathew Willows, Feitong Yang, and Guangyu Robert Yang. 2024. Project sid: Many-agent simulations toward ai civilization. Preprint, arXiv:2411.00114.
[2] Hu, T., Kyrychenko, Y., Rathje, S. et al. Generative language models exhibit social identity biases. Nat Comput Sci 5, 65–75 (2025). https://doi.org/10.1038/s43588-024-00741-1
[3] Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. Preprint, arXiv:2304.03442.
[4] Jérémy Perez, Corentin Léger, Marcela Ovando-Tellez, Chris Foulon, Joan Dussauld, Pierre-Yves Oudeyer, and Clément Moulin-Frier. 2024. Cultural evolution in populations of large language models. Preprint, arXiv:2403.08882.
[5] Maximilian Puelma Touzel, Sneheel Sarangi, Austin Welch, Gayatri Krishnakumar, Dan Zhao, Zachary Yang, Hao Yu, Ethan Kosak-Hine, Tom Gibbs, Andreea Musulan, Camille Thibault, Busra Tugce Gurbuz, Reihaneh Rabbany, Jean-François Godbout, and Kellin Pelrine. 2024. A simulation system towards solving societal-scale manipulation. Preprint, arXiv:2410.13915.
[6]  Wang, A., Morgenstern, J. & Dickerson, J.P. Large language models that replace human participants can harmfully misportray and flatten identity groups. Nat Mach Intell (2025). https://doi.org/10.1038/s42256-025-00986-z
[7] Xinnong Zhang, Jiayu Lin, Libo Sun, Weihong Qi, Yihang Yang, Yue Chen, Hanjia Lyu, Xinyi Mou, Siming Chen, Jiebo Luo, Xuanjing Huang, Shiping Tang, and Zhongyu Wei. 2024. Election sim: Massive population election simulation powered by large language model driven agents. Preprint, arXiv:2410.20746.
[8] Zhaoqian Xue, Mingyu Jin, Beichen Wang, Suiyuan Zhu, Kai Mei, Hua Tang, Wenyue Hua, Mengnan Du, and Yongfeng Zhang. 2025. What if llms have different world views: Simulating alien civilizations with llm-based agents. Preprint, arXiv:2402.13184.

Keynote Speakers

Organizers

Panelists

Shared Task: Social-Media Based Personas

Calling all students, researchers, academics, and industry practitioners to participate in our shared task on social-media based persona modeling, hosted on Kaggle! If you would like to participate, please sign up here: https://www.kaggle.com/competitions/social-sim-challenge-social-media-based-personas 

Important Dates:

• Competition Start: June 2nd, 2025 (10 PM EST)

• Competition End:   Aug 31st, 2025,  (8AM EST)    July 2nd, 2025, AoE

Task Description:

This task focuses on building realistic social-media personas from user activity and evaluating how well a model can predict their future actions.  Inspired by generative agent-style simulations papers, the goal is to explore the predictive limits of LLM-based agents when grounded in real-world behavioral data. Given a cluster of anonymized individuals and their historical activity on social platforms (e.g., BlueSky), your model must predict the most plausible next social media action the persona would take.  The challenge is to model subtle persona traits, habits, and social behavior grounded in real-world clusters while maintaining privacy and generalizability. 

Dataset

Participants will be provided with an anonymized dataset of processed clustered user activity logs (train, validation, and test splits). Each cluster contains sequences of actions (e.g., posts, replies, follows) made by individuals over time. 

• Train set: Ground truth provided

• Validation set: Ground truth hidden; evaluation shown on leaderboard

• Test set: Ground truth hidden; final evaluation only run once at the end of the competition

Participants will submit up to 3 result sets at the end of the competition, each consisting of predictions for both the validation and test sets. Only validation scores will be visible during the competition; the final ranking will be based on the average test performance across the three submitted sets. 

Evaluation Metrics:

1. F1-score (action space)

2. Cosine-similarity

Publication:

We encourage participants to submit a short (up to 4 pages) or long (up to 9 pages) paper describing their approach, insights, or analysis. Selected submissions will be invited to present their work at the workshop.

Questions:

Please don't hesitate to reach out if you have any questions, including uncertainties about the relevance of a particular topic. You can contact us at social-simulation@googlegroups.com.

Update

Shared Task Start (June 02): 

The shared task competition on Kaggle has started:  https://www.kaggle.com/competitions/social-sim-challenge-social-media-based-personas

🚧 Shared Task Delay (May 26): 

The shared task will be hosted on Kaggle, but we're still finalizing the setup — thank you for your patience!

In the meantime, to help you get started, the task will be based on a dataset similar to this:  🔗 BluePrint Dataset on Hugging Face
This version will be using the 25 user clusters and is derived from Bluesky data. The shared task will use a similar structure, but with an extended time window and updated content.

Thanks again for bearing with us!

Call for Papers

Social Sim'25 invites authors to submit papers about their work around social simulation .  Topics of interest include, but are not limited to:

• Agent-Based Simulation: Collective behavior and social dynamics using LLM-powered agents

• Emergence and Cultural Evolution: Capturing the development of norms, beliefs, and behaviors over time

• Persona Fidelity: Designing psychologically plausible, consistent, and evolving agent personas

• Novel Simulation Frameworks: Innovative approaches to simulate societal-scale phenomena with high fidelity

• Feedback Loops in Social Systems: Modeling reciprocal influence between individual actions and group-level outcomes

• Hybrid Modeling Approaches: Combining symbolic/abstract models with generative methods for expressive and high-fidelity simulation

• Simulation Realism and Privacy: Balancing detailed modeling with privacy-preserving practices

• Ethical and Social Implications: Navigating the responsibilities of deploying realistic simulations in sensitive domains

• Policy and Platform Design Applications: Using simulations to inform public policy, civic design, or platform governance

• Mitigating Manipulation Risks: Addressing the dual-use nature of persuasive or predictive simulations

Submission Types:

We invite the following types of submissions:

• Regular Papers: Submissions presenting completed work, including theoretical advances, empirical studies, or novel simulation methods. These can be up to 4 (four) pages for short papers or 9 (nine) pages for long papers, following COLM’s formatting guidelines.

• Early-stage Papers: Submissions outlining preliminary findings, prototype systems, or position pieces. These should focus on emerging ideas and directions, and may be up to 4 (four) pages in length.

• Demo Papers:  Submissions showcasing interactive systems or tools—particularly those that extend or run simulations on  Social Sandbox or similar frameworks.  These can be up to 4 (four) pages of content.

All submissions should include references (not counted toward the page limits) and follow COLM’s guidelines. All submissions will be non-archival.

Submission Portal: https://openreview.net/group?id=colmweb.org/COLM/2025/Workshop/Social_Sim 

Important Dates:

• Submission Deadline: June 23, 2025, AoE  June 27, 2025, 11:59 PM EST

• Accept/Reject Notification: July 24, 2025, AoE

• Camera Ready Deadline: July 31, 2025, AoE

• Workshop Date: October 10, 2025

Questions:

For questions regarding the call or the relevance of specific topics, please contact us at social-simulation@googlegroups.com. If you are interested in serving as a reviewer, please sign up here.