Gates Foundation NGLC Report
When ASSISTments was given a $500,000 grants from the Gates Foundation, they used SRI as an outside evaluator to study the effectiveness of the 17 different products that the Gates Foundation Funded. For an assessment tool, SRI choose to use the NWEA's computer-adaptive test called Measures of Academic Progresses(MAPs). One of the nice things about MAPs is thousands of schools a year are using this test at least twice a year. So all students will take the test at the beginning of the year and then again at the end of the years. (Since there are 17,000 question in the question bank, and its computer adaptive, the test is never the same questions). This enables a growth score to be computed for each child that takes into account where they started the year. The Schools that adopted ASSISTments has a reliable improvement in their growth scores of .03 standards deviations. To translate that into more meaningful terms, we note the average growth on MAPs at seventh grade is about .3 standards deviations so that is about a 10% improvement in learning.
Quasi-Control
The control condition were schools that NWEA selected through their "virtual control schools" service that SRI asked them to do. WPI had no influence on this step at all. NWEA used its huge database of tens of thousands of schools, to find "similar" schools.
In the image you can see that the student is not told correctness and has to wait until the next day to find out the answers.
Click image to enlarge
Experiment
Students entered answers into ASSISTments at home and were given immediate feedback as to whether the answers were correct or incorrect. The teacher used the data in the item report to know which problems to go over on the homework.
In the image you can see that there are 20 problems in the homework. The child gets feedback as they go.
Miller, G. I, Zheng, Y., Means, B. & Van Brunt, J. (2013). Next Generation Learning Challenges Wave II Final Evaluation Report. SRI Project Report # P20778. Paid for by the Gates Foundation. Retrieved on May 20, 2014, from https://drive.google.com/file/d/0B2X0QD6q79ZJUl9Kd2JuVTN0VWhTYVRhX254QV85Njdqc1Vj/edit?usp=sharing
(See Table 3, listing WPI at the bottom showing a effect size of .036.)