Embedded Files
To deepen the analysis shown in Case Study 02, it is essential to explore the relationship between usage frequency and actual learning progress on the platform. Case Study 03 focuses on analyzing the correlation between logins, interpreted as weekly attendance, and academic progress, measured through completed lessons. The analysis is segmented by grade level (K–12 system) and by product, considering the different courses or content packages assigned to students.
I simulated >8000 enrolments for this case study.
Context
Context
In this context, schools contract products consisting of courses delivered through a learning platform, which can be assigned to one, multiple, or all grade levels. The EdTech offers a portfolio of six courses, organized into two sequential tracks: three Junior courses (J1, J2, and J3) and three Senior courses (S1, S2, and S3). Each track is designed with progressive levels, introducing different expectations in terms of usage, progression, and frequency depending on the course and the grade level to which it is assigned.
As a first step, it is possible to compute simple aggregated indicators, such as the average number of logins per student per year, the average number of completed lessons, and an estimated pace of progress measured as lessons per session. While these metrics provide a high-level overview of platform usage, they are insufficient to explain meaningful differences across student groups.
Performance by Product
Performance by Product
To deepen the analysis, it is necessary to examine how performance varies by product and grade level. By correlating weekly logins with completed lessons, a bubble chart by product was built, where each point represents average values per course, and bubble size reflects the number of students enrolled in each content package.
An initial reading of the chart shows that:
Courses J3 and S3 have a significantly smaller number of enrolled students, which is expected given that they are the most advanced levels in each learning track.
The largest groups are found in J1 and S1, which act as the main academic entry points to the edtech offering.
When analyzing the correlation between usage and progress, the following patterns emerge:
In general, higher login frequency is associated with a higher number of completed lessons.
Two relevant outliers can be identified:
Students enrolled in J2 show unusually low progress relative to their accumulated number of logins.
Students enrolled in S3 achieve strong progress despite using the platform less frequently.
To fully understand these patterns, the next step of the analysis will assess whether these differences are driven primarily by student age and grade level, or by content-related and product design factors.
Performance by"Year K-12" (school year)
Performance by"Year K-12" (school year)
When analyzing the results by grade level, the following patterns emerge:
Year 12 students tend to accumulate a relatively low number of logins, which can be explained by the fact that:
it is the final year of secondary school, typically involving graduation trips, end-of-year activities, and increased extracurricular demands;
students may work more independently and in more intensive study sessions, achieving relatively good progress outcomes despite lower access frequency.
Year 07 students, corresponding to the first year of secondary school, show lower overall progress.
It is assumed that these students are enrolled in the J1 course, as they have no prior experience on the platform.Â
This raises the question: is the limited progress related to the J1 course itself, or to its suitability for the age and academic readiness of Y07 students?
Year 08 students display low progress despite having a high number of logins throughout the year.
At this grade level, it is expected that some students have started the J2 course, while in certain schools a few others are enrolled in J1.
What is happening with Y08? Is this a characteristic of the grade level itself? Are there typical school activities at this age that may distort course progression, such as international foreign language exams? Or is the issue related to the structure or demands of the J2 course in accordance to the age of these students?
Performance by cross-group Year-Product
Performance by cross-group Year-Product
A cross-analysis is conducted. Each analysis group (each bubble) is defined by the intersection of Product + Year (K–12), meaning students enrolled in a specific course at a specific grade level. Taking J1 in Y07 is not equivalent to taking J1 in Y09, as students’ cognitive maturity and complementary knowledge differ substantially.
We observe that the correlation between logins and completed lessons is generally consistent and expected across most combinations, with the following exceptions:
S1, S2, and S3 in Y12 show strong progress despite a relatively low number of logins. This supports the previously stated hypothesis:
regardless of the course, older students are able to progress autonomously and through more intensive study sessions;
additionally, this effect appears to be particularly pronounced for S3 in Y12,
as these students have likely completed other learning paths on the platform in previous years,
and this prior experience facilitates engagement with the edtech materials and workflows.
J1 in Y07 shows lower-than-expected progress when compared to the general trend observed in other groups:
this behavior is expected, as it corresponds to the first year of secondary school and represents a broader academic and methodological adaptation period;
the issue does not seem to lie in the J1 content itself, since Y08 students enrolled in J1 display strong performance relative to their number of logins.
J2 in Y08, the case that raised the most concern in the earlier analysis:
students taking J2 in Y09 do not exhibit performance issues;
Y08 students enrolled in J1 also show no significant difficulties;
this suggests that the issue is not driven solely by characteristics of the J2 course or the Y08 grade level in isolation, but rather by the alignment between course content, student age, and grade level.
Some Conclusions
Some Conclusions
Es importante tener en cuenta en el diseño de productos que:
Y12 students tend to work in a more sporadic but intensive manner.
Special care should be taken with products aimed at Y07 students, given their constrained performance. Would it be advisable to design age-adaptive content?
A more in-depth analysis of the suitability of J2 for Y08 is required. Are there prerequisite knowledge areas from other subjects that students have not yet acquired? Are there skills that are still under development?
Links
Links
Page updated
Google Sites
Report abuse