Enhancing Critical Thinking in Human-AI Interactions with Metacognitive Prompts 

With the rise of Generative AI (GenAI) search tools, searching for information is becoming increasingly convenient. However, increased reliance on these tools raises concerns regarding how they impact users' thinking and learning.

This work explores how metacognitive prompts—that encourage learners to pause, reflect, and think more deeply—can help them engage more actively with information generated by GenAI tools. Based on a formative study with 40 learners comparing their behaviors with vs without metacognitive prompts, we developed an adaptive system that provides personalized metacognitive prompts to guide learners in becoming more thoughtful and intentional in their search process.

Paper to be presented at ASIS&T'25 

Student-AI Collaborative Hint Generation

Through two field studies (N=72 and 97), we explored whether data science students, in guided collaboration with LLMs, can generate helpful hints for incorrect programming assignments while learning deeply through the hint-writing process. We compared three designs engaging students in writing hints and evaluated them based on their impact on student learning, engagement, and the quality of generated hints. One design included no AI support and the other two involved student-AI collaboration with different types of exposure to LLM-generated hints. 

We found that deferring AI assistance by requiring students to write a hint on their own first is most effective for producing high-quality hints and may result in better learning outcomes compared to on-demand AI assistance. This work highlights the importance of student-AI interaction designs that promote active student engagement with AI tools.

Paper presented at LAK'24

Understanding Learners' Programming Difficulties in Introductory Data Science

To better support learners in introductory data science courses, we qualitatively analyzed 47 students' incorrect assignment submissions in a data manipulation course (covering pandas, NumPy, etc.), conducted a log analysis of student interactions in Jupyterlab, and interviewed data science instructors. 

We categorized student mistakes into the categories shown below. For each mistake category, we provided actionable pedagogical recommendations and insights to develop scalable assessment and feedback generation tools. For instance, we suggested the features that can be extracted from data science code to detect student mistakes and provide feedback using supervised or semi-supervised machine learning methods. Overall, this work provides implications for instructors and instructional designers to teach data science at scale.

Paper presented at SIGCSE'24

Learnersourcing Questions in a MOOC: Understanding Student Motivations

Learnersourcing is a pedagogically supported form of crowdsourcing in which learners collectively generate useful content for future learners while engaging in a meaningful learning experience themselves. Two examples of learnersourcing are students creating multiple-choice questions or hints. This work takes a student-centered approach to understanding how we can design learnersourcing systems where students value the learnersourcing task, learn deeply, and generate high-quality output. 

We conducted a field experiment over 4 months with 3,661 students in the Coursera MOOC “Introduction to Data Science in Python”. The goal was to study the effects of learnersourcing on students who create Multiple Choice Questions (MCQs) and their motivations for engaging in question generation. Based on our insights, we proposed choice-based learnersourcing as a scalable personalized learning design for MOOCs. Further, we identified factors influencing student motivation to engage in learnersourcing and provided insights for designing learnersourcing activities that promote student agency.

Check out our paper here.
We also wrote a paper synthesizing prior work on learnersourcing and describing a student-centric design framework of learnersourcing.