Shared Task - EdNet

Shared Task Description

Overview

Knowledge Tracing (KT), the act of tracing a student's knowledge from their learning history, is one of the most important problems in the field of Artificial Intelligence in Education (AIEd). Through KT, an Intelligent Tutoring System (ITS) can understand each student's learning behavior and provide learning experience adapted to all individuals.

Accordingly, a variety of methods including Bayesian Knowledge Tracing (BKT), Deep Knowledge Tracing (DKT) and many more has been developed. In this context, the debate over which methods are most effective for KT has emerged. On modern ITSs with millions of users, however, the problem remains largely unsettled due to a lack of public, large-scale student interaction dataset for a fair comparison of different KT models.

To this end, we propose a KT task with EdNet, a large-scale ITS dataset with different student activities ranging from question solving to lecture watching activities. Specifically, the dataset is the largest of its kind in the world, containing 123M interactions coming from more than 1M users. For examples of the dataset, we refer to the below Tables 1 and 2.

Dataset Description

The EdNet dataset contains complete lists of interactions students made on SANTA®, an ITS specifically developed for preparing students for the TOEIC® exam. For each student, the interactions are sorted in a chronological order as a comprehensive csv table (see Table 1), allowing researchers to track different activities over time. Additionally, the metadata of learning items (namely questions, explanations and lectures) are provided in a separate table (see Table 2). Since the table includes educational tags of each learning item, methods like BKT can effectively make use of pedagogical properties to estimate a student's knowledge state.

Proposed Tasks

For this Shared Task, we invite participants to present their novel research work on the EdNet data. Participants are free to explore various tasks of educational values including, but not limited to, the following tasks with the EdNet data:

1. Correctness Prediction

• Input: Each user's history of learning behaviors (question responses and lecture watching activities) given in chronological order

• Output: Expected correctness probability (correct/incorrect) for each newly encountered question

• Goal: To predict a student's response correctness (correct/incorrect) to newly encountered multiple-choice questions assessing certain parts of their English skills.

• Metrics: Accuracy, ROC-AUC
  

2. Response Prediction

• Input: Each user's history of learning behaviors (question responses and lecture watching activities) given in chronological order

• Output: Expected response (option a/b/c/d) for each newly encountered question

• Goal: To predict a student's response to newly encountered multiple-choice questions assessing certain parts of their English skills.

• Metrics: Accuracy, ROC-AUC
  

3. Dropout Prediction

• Input: Each user's history of learning behaviors (question responses and lecture watching activities) given in chronological order

• Output: Expected dropout probability for each newly encountered activity (question, lecture)

• Goal: To predict a student's likelihood for dropout (large gap in learning activities) during learning activities.

• Metrics: Accuracy, ROC-AUC

Click the below button to check out specific details and download the data:

Citation

When submitting a Shared Task track paper, please cite the resource with the following BibTex:

@inproceedings{choi2020ednet,
title={Ednet: A large-scale hierarchical dataset in education},
author={Choi, Youngduck and Lee, Youngnam and Shin, Dongmin and Cho, Junghyun and Park,
Seoyon and Lee, Seewoo and Baek, Jineon and Bae, Chan and Kim, Byungsoo and Heo, Jaewe},
booktitle={International Conference on Artificial Intelligence in Education},
pages={69--73},
year={2020},
organization={Springer}
}