Metric Learning for Motion Data

Description:

In general, a recorded motion data sequence has the general form of a multivariate time-series. Therefore, each dimension of the data is a 1-dimensional sequence containing the temporal states of the given motion in that specific dimension. In a skeleton-based motion, the dimensions typically correspond to the angular values of the body joints. Therefore, any interpretation about the dimensions of a specific action also reveals the semantic roles of the body joints in that motion.

In practice, it is common to observe correlations among different dimensions in a motion sequence. This observation comes from the fact that in a majority of skeleton-based motions, the movement of a significant portion of the involved joints is synchronized or correlated. Therefore, the desired goal is to eliminate those dimensions in the data which do not add any significant information to the given task and to emphasize the significant dimensions.

Accordingly, in a classification setting, metric learning is the idea of finding an efficient scaling of the input space such that different data classes can be better separated from each other based on their Euclidean distances in the scaled space. This adaptation leads to learning a task-specific distance metric which applies a weighting scheme on the dimensions of the data, and it can be used to measure the relevance of the data dimensions to the given classification task.

Despite the great success of metric learning in many applications, its state-of-the-art algorithms are mainly designed for vectorial data and face difficulties for implementation on temporal representations such as motion data. To tackle this limitation, I develop dissimilarity/kernel-based metric learning algorithms which rely on pairwise distances between data points.

In particular, I modify the popular large margin nearest neighbor algorithm (LMNN) to apply it on the multi-dimensional representation of motion data effectively. The resulted algorithms scale the relational/feature space to make the relational representation of the data more discriminative respect to the class labels and also focus on finding an efficient small set of dimensions to provide that.

Details of the developed algorithms are provided in the following related publications.