background

Algebra is foundational to STEM success, but many students struggle with notations, structuring, and flexible thinking. Traditional tools focus on correctness, ignoring the process. Gamified environments such as FH2T offer low-stakes exploration, allowing multiple solution paths and promoting deeper understanding.

Yet few systems are set up to log the students' entire problem-solving process, constraining the opportunity to identify variability in strategies, misconceptions, or trajectories of learning.

project goals

The main goal of this project is to design technologies that provide ample opportunities for high-quality self-paced instruction with immediate feedback to students while providing teachers with meaningful real-time information about student strategy and performance that can inform subsequent instruction.

We seek to achieve this larger goal by working on five interrelated sub-projects:

Project 1: 

Examining Mechanisms of Mathematical Performance. 

• • Systematically identify and explore how perceptual, mathematical, and behavioral indicators of problem-solving, both within and between students, lead to improved math performance.

Project 2: 

Developing and Validating Real-Time Detectors of Strategy.

• • Create automated detectors of students’ strategy, perceptual noticing, pauses, productive failure, and errors by mining clickstream data.

  • Establish the Predictive Validity of the real-time detectors.

Project 3: 

Teacher-Centered Design Of Orchestration Tool.

• • Co-design, share prototypes, and solicit feedback from teachers (15 teachers each year).

Project 4: 

Development of the Orchestration Tool.

• • Develop prototype interface, i.e., embed and overlay detectors into Graspable Math Activities to make alerts and visualizations appear in real-time.

  • Conduct PDs, cognitive task analysis, and think-aloud with teachers to solicit ongoing feedback

Project 5: 

Establish the Promise of the Orchestration Tool.

• • Establish the Usability and Feasibility of Real-Time Orchestration in the Classroom

Note: The NSF CAREER project narrative discusses five "sub-projects", two of which are expressed in a more robust way as research studies. Here is how they are defined:

 The other three sub-projects (Projects 3-5) have a less of a focus on data-driven experimental study, and more on tool design, development, and implementation with teachers. So, to summarize:

Yes, there are two formal studies described (Study 1 and Study 2).

The other projects are design/development/evaluation-focused.

Study 1 - Mechanisms of Mathematical Performance

Participants 

This study included 800 sixth-grade (Year 7) students and their 6th-grade teachers, conducted between October 2023 and May 2024. 

Key analyses and findings:

Analyses used over 10,000 hours of logged data to:

1. Classify productive vs. unproductive strategies

2. Identify behavioral indicators such as pauses, resets, and conceptual errors

3. Evaluate the impact of perceptual features (e.g., spatial layout or visual grouping) on problem-solving

 Overall, we found that students' strategy choices and efficiency were impacted significantly by close proximity and visual arrangement of problem elements, underscoring the significance of perceptual design in digital math environments.

Study 2 - Real-Time Strategy Detection with AI

Participants 

We analyzed process-level log data from thousands of middle school students who engaged with algebraic problem-solving tasks on the Graspable Math platform.

Study 2 Design and procedure

The study was guided by the development and validation of a new tool, MathFlowLens (MFL), which employs graph-based algorithms to follow and classify students’ step-by-step algebraic strategies:

Data Collection

Detailed clickstream and gesture-action logs were extracted from their problems involving algebraic transformations.

Path Classification

We classified each student’s problem-solving path (main pathway) into one of four pathways, using algorithms including Dijkstra’s and A*:

Optimal paths – used a correct and efficient solution

Sub-optimal paths – correct solution but longer strategies

Incomplete paths – incomplete attempts

Dead-end paths – strategies will not lead to a productive resolution, signal failure & abandonment

Visualization and Analysis

MFL produced interactive network visualizations of these pathways, granting much deeper insights into student strategy use and variability to researchers and educators, respectively.

Key Analyses and Findings

Those students who explored dead-end and sub-optimal paths tended to have a better conceptual understanding and procedural flexibility than those students who only explored optimal paths, and as a result had lower flexibility scores.

The results from this research highlight the benefits of not discouraging exploratory behavior as it relates to building mathematical understanding and help inform the design of environments that do not devalue such pursuits.