• More case studies can be found here.

• The complexity analysis can be found here.

• Code implementation and replicable experiments can be found here.

• For generalized experiments on EMNIST, we have qualitative analysis (the section of "Additional results on EMNIST dataset") and quantitative analysis.

1. Description

Understanding how the predictions of deep learning models are formed during the training process is crucial to improve model performance and fix model defects, especially when we need to investigate nontrivial training strategies such as active learning and track the root cause of unexpected training results such as performance degeneration.

In this work, we propose a time-traveling visual solution DeepVisualInsight (DVI), aiming to manifest the spatio-temporal causality while training a deep learning image classifier. The spatio-temporal causality demonstrates how the gradient-descent algorithm and various training data sampling techniques can influence and reshape the layout of learned input representation and the classification boundaries in consecutive epochs. Such causality allows us to observe and analyze the whole learning process in the visible low dimensional space. Technically, we propose four spatial and temporal properties and design our visualization solution to satisfy them. These properties preserve the most important information when (inverse-)projecting input samples between the visible low-dimensional and the invisible high-dimensional space, for causal analyses. Our extensive experiments show that compared to baseline approaches, we achieve the best visualization performance regarding the spatial/temporal properties and visualization efficiency. Moreover, our case study shows that our visual solution can well reflect the characteristics of various training scenarios, showing the good potential of DVI as a debugging tool for analyzing deep learning training processes.

2 Motivate Example

The following figures show our visualization of an adversarial training process on CIFAR-10 dataset. Each point represents a sample and each color represents a class. The colors of points represent the labels of samples, and the color of a region represent a predicted class. For example, a point in red (class cat) located in brown (class dog) territory indicates that it is labelled as cat but classified as dog. Moreover, the color shade indicates the confidence of prediction, unconfident regions (i.e. classification boundaries) are visualized as white regions. Overall, the classification region and boundaries form the classification landscape. Here, the model fitting process is visualized by the process of (1) classification boundary being reshaped and (2) those data points being pulled towards the territory of the corresponding colors.

Those figures show that DVI manifests

(1) the boundary reshaping process when the model is adapting new adversarial and training samples

(2) the process of trade-off being made between adversarial robustness and testing accuracy.

During adversarial training,

(1) the adversarial points are gradually pulled to their color aligned territory, while

(2) the testing point is also gradually “pulled” away from its color-aligned territory to the territory of its adversarial neighbours.

Such trade-off is formed gradually.

We further show such a trade-off exists by visualizing the dynamics of overall data points. DVI tool can further faithfully visualize the process as animation. In addition, it supports samples and iteration queries for users to observe the dynamics of interested samples and iterations, gaining deep insights into the model training process.