Dr. Heffernan will refuse to run any study that is not minimally disruptive. A disruptive study, for example is a 20 item test where you give no feedback. That will be disruptive to learning. We want to see students get feedback as quickly as possible and that usually means immediately.
There is a natural tension between researchers who want long pre- and posttests and teachers who want their students to be learning not just taking tests. ASSISTments has a good reputation as a useful site for schools. We understand that researchers will want longer pre and posttest as they will provide a more sensitive measuring system more likely to detect reliable differences between conditions. It will also reduce the standard deviation of their measure, and inflate effect size estimates (as effect size is the difference in means divided by the standard deviation of the sample). So of course researchers want longer pre- and posttests.
But if Dr. Heffernan started doing that teachers would flee from using ASSISTments.
You will need to give feedback on your posttest. Of course this means that students will learn on the posttest and that will start to "wash out the effects of your condition" but if your effects are so minor that they can be washed away by getting feedback on your posttest, then you need to find a stronger research question or write better content. However there is a silver lining, by giving feedback you get additional information. This paper will give you detailed information on how you can use this extra information (e.g., # of attempts, # of hints requests, student response time, etc.) collected on your posttest to develop a partial credit metric.
Bottom line
What all this means, is that teachers and students don't think of ASSISTments as a TESTing site, but they think of it as LEARNing site.
What you, the researchers, get in return is more subjects, an authentic learning situation, and highly detailed information.