Traditional studies of intelligent tutoring systems have focused on their use in the classroom. Few have explored the advantage of using ITS as a web-based homework (WBH) system, providing correctness-only feedback to students. A second underappreciated aspect of WBH is that teachers can use the data to more efficiently review homework. Universities across the world are employing these WBH systems but there are no known comparisons of this in K12. In this work we randomly assigned 63 thirteen and fourteen year olds to either a traditional homework condition (TH) involving practice without feedback or a WBH condition that added correctness feedback at the end of a problem and the ability to try again. All students used ASSISTments, an ITS, to do their homework but we ablated all of the intelligent tutoring aspects of hints, feedback messages, and mastery learning as appropriate to the two practice conditions. We found that students learned reliably more in the WBH condition with an effect size of 0.56. Additionally, teacher use of the homework data lead to a more robust and systematic review of the homework. While the resulting increase in learning was not significantly different than the TH review, the combination of immediate feedback and teacher use of the data provided by WBH resulted in increased learning compared to traditional homework practices. Future work will further examine modifications to WBH to further improve learning from homework and the role of WBH in formative assessment.
Keywords: intelligent tutoring systems, immediate feedback, homework, effect size
WBH (Experimental Condition) Video
TH (Traditional Homework Condition) Video
Kelly, K., Heffernan, N., Heffernan, C., Goldman, S., Pellegrino, J., & Soffer-Goldstein, D. (2014). Improving student learning in math through web-based homework review. In Liljedahl, P., Nicol, C., Oesterle, S., & Allan, D. (Eds.). (2014). Proceedings of the Joint Meeting of PME 38 and PME-NA 36 (Vol. 3). Vancouver, Canada: PME, 417-424.
Kim Kelly, Neil Heffernan, Cristina Heffernan, Susan Goldman, James Pellegrino, Deena Soffer Goldstein (2013). Estimating the Effect of Web-Based Homework. In Lane, Yacef, Motow & Pavlik (Eds) The Artificial Intelligence in Education Conference. Springer-Verlag. pp. 824-827. (Archived material is at http://web.cs.wpi.edu/~nth/PublicScienceArchive/Kelly.htm).
Kim Kelly, Neil Heffernan, Cristina Heffernan, Susan Goldman, James Pellegrino, Deena Soffer Goldstein (2013). Estimating the Effect ofWeb-Based Homework. In Formative Feedback in Interactive Learning Environments (FFILE) Workshop. Held at The Artificial Intelligence in Education Conference, Memphis.
This paper was accepted as a poster presentation at AIED and as a workshop talk. Below find both the full workshop paper and the short paper version.