Logistic Knowledge Tracing (LKT)

The educational data mining / intelligent tutoring system research space is very interested in tracking student learning over time. Frequently datasets are fit with "learner models", which can be used retrospectively to help improve knowledge component models or evaluate system efficacy. They can also be used to make predictions in real-time (my favorite), and have those predictions used to make pedagogical decisions ("model says A is learned, but not B, let's practice B").

These models mostly track the number of attempts over time (sometimes differentiating between successes and failures). Many of them exist, but they are highly similar. Typical formulation (a and b are coefficients):

probability = a(count of successes) + b(count of failures) + (intercepts for student and item characteristics)

Some models will sound very different, like Bayesian Knowledge Tracing, but they are not logically very different and typically do worse than logistic models anyway. Don't worry about bayesian vs logistic, instead worry about whether the predictors (e.g., count of success) are built to account for principles of human learning.

The big problem is that most learner models do not have features to account for the effects of memory decay, spacing, or other well-known phenomena that impact learning. For instance, see that count of successes? Shouldn't it matter when the practice occurred? Of course it should! And the model can be built to do exactly that.

Dr. Pavlik and I made an R package to help researchers fit better models that incorporate more relevant features (such as when practice happened).

R Package:

https://cran.r-project.org/web/packages/LKT/index.html

Preprint describing the package and showing how it can lead to improved models:

https://arxiv.org/ftp/arxiv/papers/2005/2005.00869.pdf