Engagement vs. Performance: What a Learning-Analytics Intervention Really Moves
Gizem Uzuner, NUU
This study examines whether a structured learning-analytics intervention can improve student engagement and academic performance across three undergraduate economics courses at New Uzbekistan University in Tashkent, Uzbekistan. Conducted as part of the QAA International Collaborative Enhancement Project, the research involved 163 students across three courses and two semesters.
The intervention combined weekly quizzes with conditional access to subsequent learning materials, engagement monitoring, and prompts designed to encourage students to remain on track. Using a cluster-randomised treatment and control design, the study compared student engagement, formative assessment participation and academic outcomes, alongside survey evidence on students’ perceptions of learning analytics.
The findings show substantial and statistically significant increases in engagement and formative assessment participation among students receiving the intervention. Academic outcomes, including module marks and examination results, were also higher in the intervention group, but these differences were not statistically significant. Further regression and mediation analyses suggest that increased engagement provides a positive pathway to improved academic performance, although this benefit appears to be offset by a separate negative direct effect associated with the intervention.
Student feedback highlighted the value of regular quizzes and learning analytics in helping learners monitor their progress, identify areas for revision and maintain consistent study habits. The findings demonstrate that improving engagement does not automatically translate into higher grades. They also highlight the importance of intervention design, implementation costs and the wider learning context when using analytics to support student success in higher education.
Key findings
Substantial improvements in engagement: Students receiving the intervention recorded significantly higher engagement scores than the control group. The effect was large, with Cohen’s d = 1.23. Participation in experiential learning activities and formative assessments also increased substantially, with effect sizes of 1.33 and 1.24, respectively.
Positive but statistically inconclusive academic outcomes: Average module marks were 60.60 in the intervention group, compared with 56.01 in the control group. Final examination marks averaged 56.86 and 51.09, respectively. Although these differences favoured the intervention group, they did not reach statistical significance at the 5% level.
Engagement was associated with academic performance: Engagement had the strongest observed correlation with module marks among the variables examined (r = 0.45). Regression analysis also indicated a positive relationship between engagement and academic results.
A complex relationship between engagement and grades: Mediation analysis identified a positive indirect pathway from the intervention to academic performance through increased engagement. However, a competing negative direct effect meant that the overall effect on grades was small and statistically inconclusive in the pooled sample. This suggests that increasing engagement alone may not be sufficient to improve academic outcomes.
Positive student feedback: Students reported that weekly quizzes helped them take greater responsibility for their learning, understand course concepts, identify topics requiring revision and avoid falling behind. Feedback also indicated that engagement monitoring was generally perceived as professional and respectful of privacy.
Limitations and implications
The study was conducted across only three course-level clusters, limiting the statistical power to identify differences between courses. Survey response rates were lower in the control group, and the relationships between some survey scales raise questions about whether they measure sufficiently distinct constructs. In addition, the intervention required considerable technical development and manual work to administer quizzes and provide feedback.
Future research should investigate which elements of learning-analytics interventions most effectively support academic performance, how implementation can be made more efficient, and whether the results hold across a wider range of courses and student cohorts.
Overall conclusion
The study provides strong evidence that structured learning-analytics interventions can increase student engagement and participation in formative assessment. However, the relationship between engagement and academic performance is more complex: higher engagement offers a promising route to better results, but it does not guarantee them. Effective implementation therefore requires attention not only to whether students engage, but also to how that engagement translates into learning and achievement.