• The code can be downloaded from [here].

• The code to generate subject models can be downloaded from [here]. 

• Datasets for user study can be downloaded from the following list:

  • Task1

  • Task2

Importance Analyses of dynamic features

Error Resistance

Noise samples

Mis-predicted samples in the wild

User Study

Users' Demography

Identifying Root Causes in Task 1: Methods and Procedures

Our tool implementation is capable of evaluating all these root causes. The root causes and their identification methods are as follows:

• noise testing, training samples 

  • examining them on the canvas or our feedback recommendation method

• inexpressive model architecture, inappropriate learning rate, insufficient training epochs 

  • observing progressive accuracy and training dynamics in other settings by switching to the corresponding visualization tab provided during the user study

• gradient disappears

  • checking for frozen accuracy and frozen visualization in the early stage

• unbalanced training samples

  • directly observing the number of training samples across different classes

Post Survey

Typical User behaviors

Experiment Group

Control Group