Current Projects 





Attention-aware Cyberlearning to Detect and Combat Inattentiveness during Learning

Working with Jim Brockmole, we aim to blend basic research focused on why, when, and how minds wander with advances in eye tracking, mental state estimation, and conversational learning technologies to advance a new genre of attention-aware learning technologies that automatically detect and combat wandering minds. Our exemplary technological innovation leverages emerging consumer-grade eye tracking  and recent advances in mental state estimation to add an attention-aware computing layer to Guru, a cyberlearning technology for high-school biology.  The research will be conducted in 9th grade biology classrooms in Northern Indiana.

[This project is funded by the National Science Foundation]


Boredom and Mind wandering during Reading

The foundational research question is how engagement emerges from complex three-way interactions among the learners themselves (i.e., individual differences), the instructional materials (i.e., text difficulty), and the learning activities (i.e., task control and task value). A distinctive goal is to track the dynamics of emergent engagement trajectories via state-of-the-art technologies and methods from affective computing, eye tracking, and nonlinear dynamical systems. The possibility of promoting engagement and learning will also be considered by developing predictive software that selects activities and materials in a manner that is sensitive to the traits, needs, and styles of individual learners.

[This project is funded by the National Science Foundation]

Automating the Measurement and Assessment of Classroom Discourse

For over a century, research has documented the dominant configuration of lecture, recitation, and seatwork in American schools. Recent research looking at the role of classroom discourse, i.e., interactions between teachers and students, has confirmed, as an alternative to this configuration, the importance of open discussions prompted by open-ended teacher questions ("authentic teacher questions") in reading and literature instruction. The goal of our project, using cutting-edge research in speech recognition, discourse classification, and natural language understanding (NLU), is to develop CLASS 5.0, a computer program that will autonomously code classroom interactions between teachers and their students. Collaborators include Martin Nystrand (University of Wisconsin-Madison), Andrew Olney and Art Graesser (University of Memphis), and Sean Kelly (University of Pittsburgh).  Visit project page here.

[This project is funded by the Institute of Education Sciences]

An Online Performance Measure of Academic Diligence

We develop and validate a suite of performance tasks of academic diligence in middle school children. Our approach is grounded in the deliberate practice framework, which posits that skill is the consequence of sustained effort on challenging practice activities, repeated over time, with feedback and guidance. We hypothesize that diligence may distinguish students who persist in tedious tasks from those who quickly disengage by switching to less beneficial but more enjoyable alternatives. The primary outcome of this project is computer software for the psychometrically validated behavioral measure of diligence. The software will be designed to be scalable to run on any computer or mobile device, and extensible to allow other researchers to customize it as needed. This project is in collaboration with Angela Duckworth from the University of Pennsylvania.

[This project is funded by the John Templeton Foundation] 

Understanding and Increasing College Persistence

The overall goal of this project led by  Angela Duckworth from the University of Pennsylvania is to provide new insight into student factors that predict college persistence and develop strategies to cultivate them via school-based interventions. The project entails three complementary components: (1) a longitudinal study of urban high school seniors through their first year of college; (2) an in-depth, multi-method study of urban high school seniors who have demonstrated exceptional ­learning trajectories; (3) a series of double-blind randomized intervention experiments with urban high school seniors aimed at improving their mindsets about their academic potential as well as the intellectual and social meaning of critical feedback from college professors. 

[This project is funded by the Bill & Melinda Gates Foundation] 

Increasing Agency by Promoting a Purpose for Learning

In collaboration with David Yeager and Marlone Henderson at UT Austin, this project aims to (1) create and validate three behavioral measures of “academic perseverance” and  (2) develop and experimentally test in urban public middle schools a) a student-targeted “purpose” intervention designed to encourage adolescents to tie academic pursuits to higher-order goals, imbuing them with a sense of purpose around academic work and b) teacher practices designed to further reinforce in students a sense that their academic work serves a larger purpose.  Each intervention in (2) above is predicted to improve outcomes as assessed by the perseverance measures described in (1) above, in addition to raising overall grades.

[This project is funded by the Raikes Foundation and the John Templeton Foundation] 

Some Past Projects




Emotions while Students Learn from Newton's Playground

In collaboration with Ryan Baker from Teacher's College Columbia and Valerie Shute from Florida State University, this project focuses on building automated detectors of students emotions while they learn physics by playing a fun and engaging educational game called Newton's Playground. The project combines multimodal assessment of student affect and engagement from automated facial feature analysis and interaction patterns with stealth assessment of conscientiousness and conceptual physics understanding. The goal is to figure out the ways that specific affective states disengaged behaviors, and conscientiousness interact and ultimately influence learning. 

[This project is funded by the 
Bill & Melinda Gates Foundation]

Intelligent tutoring system with EEG-based instructional strategy optimization

We are collaborating with QUASAR USA to develop a computerized tutoring platform that adapts its teaching strategy to students in real-time by monitoring their brain activity. As part of this program, we will conduct trials with high school students using its dry electrode headsets to measure EEG in a classroom environment. EEG has the potential to enhance instruction by giving the tutoring system unique insight into the student's cognitive workload and engagement. 

[This project is funded by the National Science Foundation]

GuruTutor: A Computer Tutor That Models Expert Human Tutors

This project, in collaboration with Andrew Olney, investigates expert tutoring mechanisms at multiple levels including models, modes, and moves. We are developing broad computational models of expert tutors that encompasses their pedagogical and motivational strategies, dialogue, language, affective responses, and gestures.  The overall goal of the project is to develop a computer tutor for high school biology based on strategies and dialogue of expert human tutors. The tutor could have a big impact on Memphis City Schools because it seeks to improve educational outcomes on the Tennessee Gateway Science Test, which high school students must pass in order to receive a diploma.[Read More]

[This project is funded by the 
Institute of Education Sciences] 

Confusion and Cognitive Disequilibrium during Learning

This project focuses on the affective state of confusion with an emphasis on the following research questions: (1) What are the appraisals that lead to confusion? (2) How is confusion expressed in the face, speech, body, physiology, language, and context? (3) What are the temporal dynamics of confusion,? (4) How is confusion effectively regulated? (5) When is confusion beneficial for learning? We have developed computerized interventions that induce, track, and regulate confusion to test the hypothesis that there might be some benefits to productively confusing learners. 

[This project is funded by the National Science Foundation]

Monitoring Emotions while Student Learn with AutoTutor

The goal of this research is to build and test learning environments that coordinate complex learning and learner emotions. The project augments an existing intelligent tutoring system (AutoTutor) that helps learners construct explanations by interacting with them in natural language and helping them use simulation environments. The tutorial dialogue of AutoTutor will be enhanced in the proposed research by incorporating signal processing algorithms and sensing devices that classify various facial patterns and affective states of learners. 

[This project is funded by the National Science Foundation]

Robust Automated Knowledge Capture

Working in collaboration with researchers at Sandia National Laboratories, the University of Memphis, and the University of Notre Dame, we attempt to identify skills that may differentially affect performance of individuals in cognitive tasks relevant to flying airplanes and communicating with team members. The project attempts to identify or develop measures to quantify individual ability with respect to each identified skill, particularly the ability to flexibly switch strategies in response to dynamically changing task constraints.

[This project is funded by Sandia National Laboratories]

Cognitive Computing Research Group

I work with Stan Franklin on the conceptual and computational development of the LIDA model. LIDA is a cognitive architecture that aspires to model several facets of human and animal cognition. LIDA incorporates sophisticated action selection, a centrally important attention mechanism, and multimodal instructionalist and selectionist learning mechanisms. Empirically grounded in cognitive science and neuroscience, the architecture is strictly neither symbolic nor connectionist, but blends crucial features of each. LIDA is a successor of IDA, an agent that helps the Navy by assigning sailors to jobs. IDA is a very complex agent that perceives e-mails from sailors, deliberates on the right jobs for the sailor and negotiates with the sailor in the context of sailor's preferences and Navy's policies. [Read More]

Memphis Intelligent Kiosk Initiative (MIKI)

MIKI is a three-dimensional directory assistance-type digital persona displayed on a prominently-positioned 50 inch plasma unit housed at the FedEx Institute of Technology at the University of Memphis. MIKI, which stands for Memphis Intelligent Kiosk Initiative, guides students, faculty and visitors through the Institute’s maze of classrooms, labs, lecture halls and offices through graphically-rich, multidimensional, interactive, touch and voice sensitive digital content. MIKI differs from other intelligent kiosk systems by its advanced natural language understanding capabilities that provide it with the ability to answer informal verbal queries without the need for rigorous phraseology. 

[This project is funded by the FedEx Institute of Technology]

Radio Frequency Identification Consortium

This project focused on characterizing the performance of RFID tags in a GHz Transverse Electromagnetic (GTEM) cell. Performance of four commercially available RFID tags manufactured by different vendors was characterized on the basis of horizontal directivity,vertical directivity, sensitivity, and frequency characteristics.

With these baseline characteristics determined, we moved two of the four tags through a real world environment in three dimensions using an industrial robotic system to determine the effect of asset position in relation to the reader on tag readability.