Research

A Game Design Framework for Understanding LLMAS

Advisor: Prof. Qianwen Wang

Institution: the Department of CompSci & Eng, UMNTC

Duration: September 2024 - May 2025

Link: [Paper] [Codes] [Demo] [Video]

Abstract

As multi-agent Large Language Model (LLM) systems become ubiquitous in various applications and daily activities, understanding their internal mechanisms has become critical for proper use. However, this understanding remains largely inaccessible, particularly for non-AI experts who increasingly create and use these systems. Inspired by the effectiveness of games in facilitating learning, we propose Agentopia, an interactive game framework for learning multi-agent system (MAS) concepts. In the game, players take on the challenge of mitigating hallucinations, learning through direct experimentation how different MAS configurations affect information flow, error propagation, and output quality. Our framework employs intuitive metaphors that map abstract MAS concepts into familiar game mechanics. We further implement our design approach into extensible configurations that can be easily customized and generalized. We instantiate the framework as a “newsroom” scenario in which various agents work together to compose a data story for a given dataset. To evaluate effectiveness, we conducted an in-lab observational study to assess the learning benefits of Agentopia and understand how users interact with the system.

Outcome

Y. Lu, S.Du, and Q. Wang, “The Agetopia Times: Understanding and Mitigating Hallucinations in Multi-Agent LLM Systems via Data Journalism Gameplay” - IEEE Visualization and Visual Analytics(VIS) 2025

Y. Lu, S.Du, and Q. Wang, “Understanding LLM Multi-agent Systems through a Game Design Framework” - Under Revision - Prepare for ACM UIST 2026 Submission

Others

This work was presented at the CS&E Graduate Student Orientation Poster Session(Fall 2025) at the University of Minnesota, Twin Cities. 

An Educational Visualization Tool for GNNs

Advisor: Prof. Qianwen Wang

Institution: the Department of CompSci & Eng, UMNTC

Duration: April 2024 - September 2024

Links: [Paper] [Workshop] [Codes] [Demo] [Slides] [Video]

Abstract

Graph Neural Networks (GNNs) have achieved significant success across various applications. However, their complex structures and inner workings can be challenging for non-AI experts to understand. To address this issue, we present \name, an educational visualization tool for interactive learning of GNNs. GNN 101 seamlessly integrates mathematical formulas with visualizations via multiple levels of abstraction, including a model overview, layer operations, and detailed animations for matrix calculations. Users can easily switch between two complementary views: a node-link view that offers an intuitive understanding of the graph data, and a matrix view that provides a space-efficient and comprehensive overview of all features and their transformations across layers. GNN 101 not only demystifies GNN computations in an engaging and intuitive way but also effectively illustrates what a GNN learns about graph nodes at each layer. To ensure broad educational access, GNN 101 is open-source and available directly in web browsers without requiring any installations.

Outcomes

Y. Lu, C. Chen, K. Huang, M. Zitnik, and Q. Wang, “GNN 101: Visual Learning of Graph Neural Networks in Your Web Browser,” arXiv preprint arXiv:2411.17849, Nov. 2025. - IEEE Transactions on Visualization and Computer Graphics

Y. Lu, C. Chen, M. Xu, and Q. Wang (2024). What Can a Node Learn from Its Neighbors in Graph Neural Networks? at the 7th VISxAI Workshop at IEEE VIS 2024, St. Pete Beach, Florida, October 2024. 

Others

This tool also featured in the CS224W: Machine Learning on Graphs at Stanford University. 

This tool also featured in the Visual Computing Group Film at the University of Minnesota, Twin Cities. 

This work was presented at the Visual Computing Seminar at the University of Minnesota, Twin Cities. 

gnn-exp: Visual Exploration and Explanation of Graph Neural Networks in Your Computational Notebooks

Advisor: Dr. Qianwen Wang

Institution: the Department of CompSci & Eng, UMNTC

Duration: November 2025 - Present

Link: [Codes] [Demo] [PyPl] [Paper] 

Abstract

gnn-exp is a Python package that integrated a diverse set for visualizing graph neural networks. The package has a graph visualizer, a graph editor, and a graph neural network visualizer. The package can perform visualizations and interactions such as hierachical structures and dual views to support better sensemaking and explainability to technical users. gnn-exp was adopted from our previous work, GNN101. 

Outcome

Y. Lu and Q. Wang (2025). gnn-exp: Visual Exploration and Explanation of Graph Neural Networks in Your Computational Notebooks [Computer Software]. Python Package Index. https://test.pypi.org/project/gnn-explorer/ 

Optimization Method for Tool Calling using In-context Learning

Advisor: Prof. Maria Gini

Institution: University of Minnesota and University of Hong Kong

Duration: June 2025 - August 2025

Link: [Paper]

Abstract

Recent advancements in large language models (LLMs) have significantly improved performance across various natural language processing (NLP) tasks and have sparked increasing interest in equipping LLMs with tool-calling capabilities to augment their practical intelligence. Tool calling allows LLMs to interact with external APIs or systems to solve complex real-world tasks, but it remains challenging due to its multi-stage nature and the difficulty of selecting suitable tool usage demonstrations. While prior work has focused on manual prompt engineering, we investigated how in-context learning (ICL) can be utilized to optimize tool calling without the need for parameter updates. To improve both accuracy and generalizability of tool calling, we propose a novel in-context tool calling (ITC) framework that integrates retrieved demonstrations and reranking mechanisms. Specifically, we construct a new dataset by repurposing existing benchmarks and introduce a fine-grained evaluation metric (FG-ACC) to assess tool-calling performance. According to experiments conducted in multiple LLM frameworks, ITC consistently outperforms baseline methods in both coarse- and fine-grained metrics. Our ablation studies further reveal that reranking demonstrations based on semantic alignment and empirical utility lead to significant gains. These findings suggest that retrieved in-context examples, when properly selected and ordered, can substantially enhance the reasoning and tool-use capabilities of LLMs.

Outcomes

J. You, M. Gini, W. Zhu, S. Chen, Y. Lu, Y. Mao, "Optimizing the Tool Calling Capabilities of Large Language Models Using In-Context Learning," In Submission to Association of Computational Linguistics.

A Study about Human-AI Cognitive Process Alignment in Scientific Writing

Advisor: Prof. Qianwen Wang

Institution: the Department of CompSci & Eng, UMNTC

Duration: May 2025 - June 2025

Links: [Code] [Demo]

Abstract

In this study, we investigate the alignment of human-AI cognitive processes in scientific writing to explore the strengths and weaknesses of current large language models. We utilize both statistical and semantic methods to measure and analyze cognitive process alignment. Software aspect, we developed a simulation pipeline for LLM scientific iterative writing with various configurations and hyperparameters, a computational data pipeline to analyze the alignment of human-AI cognitive processes, and a visual analytical tool to explore the iterative scientific writing process. This work also motivates us further to explore the design opportunities in the human-AI co-writing interface. 

Outcome

Y. Lu, S.Du, and Q. Wang (2025). ScholaWrite Visualization[Computer Software]. Github Repository. https://github.com/Visual-Intelligence-UMN/scholawrite-vis 

Outcomes

R. Strausbaugh, D. Warshofsky, P. Kelly, M. S. S. Gill, A. Toscano, Y. Lu, S. Leggio, and A. Kamenshikova, "GRB231115A: TURBO Pre-Burst Optical Upper Limits," GRB Coordinates Network, vol. 35296, p. 1, Dec. 2023.