Clients & Case Studies

Discover how my consulting services have transformed educational solutions, utilizing AI and related techniques to improve learning outcomes.

Measuring Learning Through Innovative Methods

Uncovering a deeper understanding of learning & engagement through innovative research methodologies, digital tools, & data analysis. Throughout my research career, I've been fascinated by the intricate interplay between human cognition, technology, and learning. From analyzing emotions in educational games to designing unobtrusive assessments, my projects explore how we can leverage innovative methodologies to understand and optimize the learning experience.

Designing and Developing Tools for Impact

Designing, developing, implementing, and evaluating digital solutions for education, worker protection, litigation, and more. Throughout my career, I've been driven by the belief that technology can be a powerful catalyst for learning and innovation. From analyzing data to inform educational policy to designing engaging curriculum and evaluating the effectiveness of digital tools, I've worked across diverse sectors to bridge the gap between theory and practice, optimizing educational experiences for learners of all ages and backgrounds.

Redefining and Investigating Learning Models

Championing and investigating innovative school models, personalized learning, & competency-based approaches to empower every learner. My passion lies in exploring innovative school models that empower students to take ownership of their learning. From evaluating large-scale personalized learning initiatives to designing engaging STEM experiences for middle schoolers, I've worked across diverse contexts to identify effective practices and support educators in implementing sustainable change.

Field Building and Policy Shaping

Shaping policy and spearheading advancements in fields like education data science and the humanities to build a brighter learning landscape. Throughout my career, I've been driven by the belief that effective policies and well-developed fields are crucial for translating research into tangible benefits for society. From influencing national research ethics guidelines to shaping the nascent field of education data science, I've worked at the intersection of research and practice, advocating for evidence-based approaches and innovative technologies that empower learners and researchers alike.

Representative Projects Associated with Research Methodologies

As a Senior Research Officer at the Bill & Melinda Gates Foundation

Log File and Valence Studies of Engagement. Machine learning and data mining investigations with Principal Investigator (PI) Ryan Baker on the ways that (a) affective states (e.g., boredom, frustration, engaged concentration, and delight), (b) behaviors indicating engagement and disengagement, and (c) conscientiousness (i.e., the motivation to work hard despite challenging conditions) interact and ultimately influence learning.
Stealth Assessments of Cognitive and Non-Cognitive Factors. Application and advancement of evidence-centered design methods with PI Val Shute to design, implement, and evaluate “stealth assessments,” which are assessments embedded deeply within games to measure unobtrusively, accurately, and dynamically how learners progress relative to targeted competencies.
Studies of Grit and Academic Tenacity, University of Pennsylvania. Multi-year investigation with PI Angela Duckworth to provide new insight into both cognitive and non-cognitive student factors that predict college persistence and to develop school-based interventions to cultivate them.

As a Researcher at SRI International

Co-Principal Investigator, Virtual Environment Real User Study (Verus). An international study identifying behavioral indicators in Virtual Worlds (VWs) and Massive Multiplayer Online Games (MMOGs) from logfiles that are predictive of real-world characteristics of the users. Sponsored by Air Force Research Labs at >$5M over three years.

Representative Projects Associated with Digital Tools

As an Executive Director at ETS

EF+Math. As an Executive Director at ETS, engineered the contract for ETS researchers to provide support on measurement and methods to the EF+Math Program team and their grantees. The EF+Math initiative aims to improve math outcomes for students in grades 3–8 by focusing on foundational skills necessary for learning: executive functions. EF+Math, the Advanced Education Research and Development Fund’s inaugural program, focuses on students who have been traditionally underserved. The program supports projects that focus on mathematics and executive functions that provide students with agency over their attention, emotions, and behavior to achieve their goals.
Ireland Department of Education. In the spring of 2020, when Irish schools closed due to the pandemic, the usual school-leaving examinations needed for university admissions were canceled. Faced with this challenge, Ireland's Department of Education sought researchers from ETS to implement a complex statistical procedure for standardizing teacher-assigned grades based on historical data. This work provided valuable insights for ETS R&D into alternative scoring systems. It also opened the door to further collaboration with Ireland, who continued to rely on "Accredited Grades" (standardized teacher grades) in 2021 and 2022, with ETS R&D providing development and support.

As a Director at Summit Consulting

Study Director, Residential Mortgage-Backed Securities Litigation. In the run-up to the commercial bank crisis of the late 2000s, investment banks pooled mortgages into residential mortgage-backed securities (RMBS). The downturn in the housing market bankrupted the investment banks that held >$500B in RMBS. The federal government bailed out the banks by purchasing significant amounts of RMBS. When the securities did not perform as expected, the federal government took 19 commercial banks to court. Oversight of more than 30 legal engagements concerning RMBS for both public and private entities. Managerial responsibilities include complex communication and contract management among experts, multiple consulting organizations, counsel, and counsel’s clients. Organization of personnel into functional teams to rapidly develop skills and create efficiencies. Once staff masters the knowledge and skills of one functional team, they are rotated to gain new knowledge and develop new skills. This practice ensures all team members understand cases holistically, avoids single-person dependences, and creates opportunities to deepen and transfer knowledge and skills.

Technical responsibilities include oversight of (a) data management, the preparation, blending, and storage of massive amounts of sensitive quantitative data (~1 TB per case); (b) study design, the creation of complex quantitative study designs that align and inform legal strategies; (c) evidence generation, the proper application of statistical methods using transparent and reproducible code (R, Stata, SAS, and Python) to quantify outcomes, differences, and damages; and (d) evidence presentation for fact-finders (i.e., juries and judges), the presentation of evidence in graphs, tables, visualizations, and compelling stories.

Study Director, Department of Labor (DOL), Chief Evaluation Office (CEO), Mine Safety and Health Administration (MSHA) Early Detection Project. Managed a team of five to develop predictive analytic models and user-friendly tools for MSHA administrators and inspectors. The model uses administrative and commercial data to determine which mine operators and contractors will likely fail to pay their fines from inspection violations.
Study Director, DOL, CEO, Wage and Hour Division (WHD), Enterprise-Wide Enforcement Pilot Project. Managed a team of five in the conducting of industry analysis of the hotel and motel industry, calculating baseline estimates of wage compliance in the hotel and motel industry, and analyzing predictors, correlates, and impacts of wage compliance. Afterward, the team developed and piloted a predictive model that determines the likelihood that hotels and motels are likely to violate wage laws.
Study Director, DOL, CEO, Site-Specific Targeting Impact Evaluation. An impact evaluation on the Occupational Safety and Health Administration’s (OSHA) Site-Specific Targeting (SST) program, which aims to maximize safety and health by focusing OSHA’s enforcement actions on worksites with the highest rates of injury and illness. The study includes the simultaneous application of a randomized control trial and regression discontinuity design to estimate the impact of receiving a “high-rate letter” which indicates that the worksite had a higher-than-average rate of injuries. Each site in the treatment group received a high-rate letter, while the control group sites did not. All the sites received follow-up inspections to check for compliance with OSHA standards.
Study Director, DOL, CEO, Federal Agency Targeting (FEDTARG) Program Study. A descriptive, quantitative study of OSHA's FEDTARG program, which targets federal worksites for inspections based on the number of lost-time cases (LTCs) the worksite experienced in the previous fiscal year. The results will provide details on the characteristics of the inspected sites, inspection information including the violations cited during inspections, and the injury/illness counts following inspections.

As a Senior Research Officer at the Bill & Melinda Gates Foundation

Research and Evaluation on Khan Academy. A multi-year implementation study with PI Bob Murphy to generate information for school systems, school leaders, and teachers on how Khan Academy, and by implication, other similar digital learning tools and resources, could be used to support personalized math learning. Participants included 20 public, charter, and independent schools; more than 70 teachers; and approximately 2,000 students in each study year.
Encouraging Student Success in Online Learning. A multi-year investigation with PI Marianne Bakia and Co-PI Jessica Mislevy to determine to what extent, when, and how six large-scale online Algebra 1 courses work or do not work for different student populations, especially those that are historically disadvantaged or underserved.
Game-Based Formative Assessment Designs and Practices, University of Michigan. Investigation with PI Barry Fishman to understand how teachers’ formative assessment knowledge and practices interact with the formative assessment features of a range of educational games intended to support student learning of core content in mathematics, science, English language arts, and social studies and history.
GlassLab Research. Multi-year, mixed methods research with PI Geneva Haertel, Co-PI Britte Haugan Cheng, and others on the qualities, features, internal validity, reliability, and effectiveness of the assessments that were embedded within the GlassLab game products. GlassLab, a $10M investment by the Gates Foundation and the MacArthur Foundation, supported the creation of assessments built into commercial video games, such as the Sims.

As a Researcher at SRI International

Co-Principal Investigator, Evaluation of NASA’s “Astronaut: Moon, Mars, and Beyond.” This effort extended current models for evaluating educational games by describing the roles, responsibilities, and communication networks of the public-private partnership formed to build NASA’s MMOG, “Astronaut: Moon, Mars, and Beyond.” In addition, the evaluation investigated the impact of gameplay on students’ understanding of STEM content and their STEM career aspirations in close collaboration with other researchers affiliated with the game. Funded by NASA at approximately $500K over two years.
Researcher, Evaluation of Evidence-Based Practices in Online Learning. A study sponsored by the U.S. Department of Education to provide educators, policymakers, and researchers with evidence-based strategies for implementing online learning in secondary education.
Principal Investigator, U.S. Army Persistent Learning Capability Training Effectiveness Analysis. This evaluation included the verification and validation—and formative and summative evaluation—of a Multi-user Online Virtual Exercise (MOVE) that soldiers used to train using field tools collaboratively before applying them in theater. Funded by the Department of Defense at >$1M over two years.

As a Project-based Curriculum Developer

Curriculum Developer, The National Parks. Curriculum developer for “Bridging the Watershed,” in which National Park Rangers and teachers from Prince George’s County, Maryland, and the District of Columbia collaborated to develop hands-on science lessons and activities for secondary students. A collaborating author and editor of the “Watershed Watchdogs” module. Brought groups of science students each academic semester to Washington area national parks to complete modules and analyze findings with students.
Curriculum Developer, Maryland Public Television. Authored and presented learning modules for, “the Maryland Public Television 2000 Maryland Tech Consortium Institute for English, Social Studies, and Technology Mentor Teacher Program,” “the Maryland Public Television and Johns Hopkins University BioHealth Link Questions of Cancer Mentor Teacher Program,” “the Maryland Public Television 1999 Maryland Tech Consortium Institute for Math, Science, and Technology Mentor Teacher Program,” and “the Maryland Public Television National Teacher Training Institute for Math, Science, and Technology.”

Learn more about Residential Mortgage-Backed Securities from Khan Academy

Representative Projects Associated with School Models

As a Senior Research Officer at the Bill & Melinda Gates Foundation

Research and Evaluation on Personalized Learning School Models. Multi-year investigation with PI John Pane, Co-PI Laura Hamilton, and others of more than 100 personalized learning school models in the United States over time to determine which models and the instructional strategies they apply consistently overperform traditional school models.
Research and Evaluation on Personalized Learning Pathways. Process and outcome evaluation with PI Matt Lewis and Co-PI Jennifer Steele of three pilot personalized learning pathways interventions in Philadelphia Public Schools, Adams 50, and China Society. The effort included the creation of a vision document and draft metrics for the field regarding personalized learning models in K-12 settings.

As a Researcher at SRI International

Evaluator, CISE Pathways to Revitalized Undergraduate Computing Education (CPATH). Evaluation of CPATH, an NSF-sponsored initiative with the goal of transforming undergraduate computing education on a national scale.
Researcher, Pre-Engineering Partnership Design. Pre-Engineering Partnerships (PEP) is a new program that extends the National Defense Education Program’s (NDEP) impact on middle schools. Through PEP, Department of Defense scientists and engineers visit middle school classrooms and conduct instructional activities with the students and their teachers. SRI-designed activities that connect teachers’ needs, scientific and engineering resources, student motivation, and learning materials and technologies to be used in PEP.

Representative Projects Associated with Field Building and Policy Shaping

As a Senior Research Officer at the Bill & Melinda Gates Foundation

Common Rule Study. Support of a National Academy of Sciences consensus study and resulting report to communicate the position of the behavioral, social, and educational research community on revising the Common Rule, which governs human subject research conducted or supported by federal departments and agencies.
How People Learn II: Learners, Contexts, and Cultures. Support of a National Academy of Sciences consensus study and resulting report that expands on the foundation laid out in the 2000 report and takes an in-depth look at the constellation of influences that affect individual learning. It provides a much-needed update incorporating insights gained from the investigation of neurological processes involved in learning, individual and cultural variability related to learning, and educational technologies. In addition to expanding scientific understanding of the mechanisms of learning and how the brain adapts throughout the lifespan, authors document the important discoveries about influences on learning, particularly sociocultural factors and the structure of learning environments.
Learning Analytic Workgroup, Stanford University. Support of PI Roy Pea and a group of approximately 30 experts from diverse disciplines and sectors charged to prepare and deliver a public-facing report in summer 2014 that (a) provides a conceptual framework for building the field of education data scientists, (b) defines critical questions for understanding how to build the field of education data scientists, (c) articulates and prioritizes new tools, approaches, policies, markets, and programs of study associated with the field of education data scientists, (d) determines resources needed to address priorities, and (e) maps how to implement the field-building strategy and how to evaluate progress.

As a Researcher at SRI International

Researcher, National Education Technology Plan. Supported fellow SRI International researchers and a Technical Working Group in the development of the U.S. Department of Education’s fourth National Educational Technology Plan. The plan, “Transforming American Education: Learning Powered by Technology,” calls for applying the advanced technologies used personally and professionally to our entire education system to improve student learning, accelerate, and scale up the adoption of effective practices, and use data and information for continuous improvement.
Principal Investigator, Harnessing Virtual Worlds for Arts and Humanities Scholarship. Investigation of the potential of virtual worlds to support new forms of humanities scholarship and research. Funded by the Andrew Mellon Foundation at over $500K over one year.

Page updated

Google Sites

Report abuse