Enhancing the Efficiency and Reliability of Group Differentiation through Partial Credit
There is a gap that must be bridged between the Educational Data Mining (EDM) community, which seeks to model student performance, and educational researchers who use intelligent tutoring systems with the goal of designing interventions that improve learning gains. The present work begins to bridge this gap via two studies on the analytical benefits of using rich system features to better define student performance. We argue that partial credit, defined as an algorithmic combination of binary correctness, hint usage, and attempt count, can benefit assessment. Using both experimental and non-experimental data sourced from ASSISTments, an adaptive mathematics tutor, a resampling technique is applied to compare partial credit with binary scoring in the context of significant group differentiation. Consistent benefits were observed for partial credit, with the most pronounced comparison resulting in a 39% reduction in the sample size required to reliably differentiate between user groups.
Data and code used in this paper can be found here.