Simple and performant implementations of learner performance prediction algorithms:

• Performance Factors Analysis (PFA)
• DAS3H
• Deep Knowledge Tracing (DKT)
• Self-Attentive Knowledge Tracing (SAKT)

Create a new conda environment, install PyTorch and the remaining requirements:

conda create python==3.7 -n learner-performance-prediction
conda activate learner-performance-prediction
pip install -r requirements.txt
conda install pytorch==1.2.0 torchvision==0.4.0 -c pytorch

The code supports the following datasets:

• ASSISTments 2009-2010 (assistments09)
• ASSISTments 2012-2013 (assistments12)
• ASSISTments 2015 (assistments15)
• ASSISTments Challenge 2017 (assistments17)
• Bridge to Algebra 2006-2007 (bridge_algebra06)
• Algebra I 2005-2006 (algebra05)
• Spanish (spanish)
• Statics (statics)

For your convenience, the preprocessed data sets are in the data/ folder. You do NOT need to preprocess data sets yourself.

If you want to reproduce the preprocessing, download the data from one of the links above and:

• place the main file under data/<dataset codename>/data.csv for an ASSISTments dataset
• place the main file under data/<dataset codename>/data.txt for a KDDCup dataset
• place the two data files under data/<dataset codename>/{filename} for the Spanish dataset

python prepare_data.py --dataset <dataset codename> --remove_nan_skills

To encode a sparse feature matrix with specified features:

• Item Response Theory (IRT): -i
• PFA: -s -sc -w -a
• DAS3H: -i -s -sc -w -a -tw
• Best logistic regression features (Best-LR): -i -s -ic -sc -tc -w -a

python encode.py --dataset <dataset codename> <feature flags>

To train a logistic regression model with a sparse feature matrix encoded through encode.py:

python train_lr.py --X_file data/<dataset codename>/X-<feature suffix>.npz --dataset <dataset codename>

To train a DKT model:

python train_dkt2.py --dataset <dataset codename>

To train a SAKT model:

python train_sakt.py --dataset <dataset codename>

Sample behavioral test is in behavior_test.py. To test a model:

python behavior_test.py --dataset <dataset codename> --load_dir <load_directory> --filename <filename>

This sample code examines first 1000 test data. For each input data, we change all responses to 1(correct) and check if the predicted correctness probability increases. Similarly, we change all responses to 0(incorrect) and check if the predicted correctness probability decreases.