Estimating the effect of Web-based Homework

This study compared traditional "business as usual" homework (with no feedback until the next day) with ASSISTments. For this study the ASSISTments condition provided correctness only feedback and allowed student to try as many times as they want. If a student could not guess the answer, there was a button they could hit to get told the correct answer. There was a reliable 10% improvement in student learning. The effect size was big (.56) across the whole sample (n=67), but for the students who scored the worst on the pretest (n=46) the effect size was .94 showing that ASSISTments is particularly good for students that come to school with less background. So it closes achievement gaps. Not too surprising the highest knowledge kids did not need the help.

Control

Students did not receive immediate feedback when completing their homework. The teacher read answers to students the next day. Asked students which questions they wanted to see.

In the image you can see that the student is not told correctness and has to wait until the next day to find out the answers.

Experiment

Students entered answers into ASSISTments at home and were given immediate feedback as to whether the answers were correct or incorrect. The teacher used the data in the item report to know which problems to go over on the homework.

In the image you can see that there are 20 problems in the homework. The teacher built the problem set in triples, three similar problems in a row. This student got the first one wrong but figured it out because of the feedback and go the next three right. The student is struggling with the next triple.

Click image to enlarge

Kelly, K., Heffernan, N., Heffernan, C., Goldman, S., Pellegrino, G. & Soffer, D. (2013). Estimating the Effect of Web-Based Homework. In Lane, Yacef, Motow & Pavlik (Eds) The Artificial Intelligence in Education Conference. Springer-Verlag. pp. 824-827. (More Information )

I longer version of this appeared later and is here:

Kelly, K., Heffernan, N., Heffernan, C., Goldman, S., Pellegrino, J., & Soffer-Goldstein, D. (2014). Improving student learning in math through web-based homework review . In Liljedahl, P., Nicol, C., Oesterle, S., & Allan, D. (Eds.). (2014). Proceedings of the Joint Meeting of PME 38 and PME-NA 36 (Vol. 3). Vancouver, Canada: PME. Pages 417-424.

The additional analysis on the lower knowledge student is stored here