Detecting Common Actions in Motion Capture Data and Videos

We present a novel solution to the problem of detecting common actions in time series of motion capture data and videos. Given two action sequences, our method discovers all pairs of similar subsequences, i.e. subsequences that represent the same action.

This is achieved in a completely unsupervised manner, i.e., without any prior knowledge of the type of actions, their number and their duration. These common subsequences (commonalities) may be located anywhere in the original sequences, may differ in duration and may be performed under different conditions e.g., by a different actor.

The proposed method performs a very efficient graph-based search on the matrix of pairwise distances of frames of the two sequences. This search is supported by an objective function that captures the trade off between the similarity of the common subsequences and their lengths.

To discover all commonalities of two videos A, B MUCOS operates as follows:

1. Compare pairwise all frames of A, B and estimate their distance matrix D
2. Compute the sets of potential commonality end points and midpoints.
3. Define a graph G whose nodes are the end points and midpoints.
4. Compute all shortest paths in G.
5. Associate shortest paths with commonalities and discard those that don’t meet certain criteria.
6. Employ an objective function to evaluate and accept/reject the remaining commonalities.

Figure 2: Summary of the obtained results in all datasets..

• You can download the matlab code of MUCOS and SMUCOS methods proposed in [1]
• You can download the distance matrices and ground truth of MHAD101s (.rar) and MHAD101v datasets (.rar)
• You can download the videos of MHAD101v dataset (.rar)
• The rest dataset sets (CMU86-91 and 80-Pair) can be downloaded from evaco webpage [2]: (http://www.ics.forth.gr/cvrl/evaco/). In order to test the method for the datasets other than MHAD101-s and MHAD101-v, one needs to build the distance matrix from given sequences and feed it to the proposed method.
  • You can download the experimental results of MUCOS - SMUCOS and EVACO variants [2], presented in [1] (.rar).
  • The .rar files are protected with the following password: pr2017!@a
  • See the corresponding readme.txt files for more details.

Related Publications

[1] C. Panagiotakis, K. Papoutsakis and A.A. Argyros, A Graph-based Approach for Detecting Common Actions in

Motion Capture Data and Videos, Pattern Recognition, Pattern Recognition, Elsevier, vol. 79, pp. 1-11, July 2018.

[2] K. Papoutsakis, C. Panagiotakis and A.A. Argyros, "Temporal Action Co-Segmentation in 3D Motion Capture Data and Videos", In IEEE Computer Vision and Pattern Recognition (CVPR 2017), IEEE, Honolulu, Hawaii, USA, July 2017.