Dataset

Download Dataset and Code

Description

The dataset includes 20 cooking actions, involving either a single or both arms of the volunteer, some of them including tools which may require different forces. Three different view-points have been considered for the acquisitions, i.e. lateral, egocentric, and frontal. For each action a training and a test sequence is available, each containing, on average, 25 repetitions of the action. Furthermore, acquisitions of more structured activities are included, in which the actions are performed in sequence for a final, more complex goal.

An annotation is available, which includes the segmentation of single action instances in terms of time instants in the MoCap reference frame. A function then allows to map the time instants on the corresponding frame in the video sequences. In addition, functionalities to load, segment, and visualize the data are also provided in Python and Matlab.

List of the included actions:

Cutting the bread
Shredding a carrot
Cleaning a dish
Eating
Beating eggs
Squeezing a lemon
Mincing with a mezzaluna
Mixing in a bowl
Opening a bottle
Turning the frittata in a pan
Pestling
Pouring water in multiple containers
Pouring water in a mug
Reaching an object
Rolling the dough
Washing the salad
Salting
Spreading cheese on a slice of bread
Cleaning the table
Transporting an object

Test Activities:

Scene #1: The actor mixes ingredients in a bowl, then adds salt and pours some water. Finally the ingredients are mixed again.

Scene #2: The actor reaches a slices of cheese, grabs it and shreds it. Then, the actor moves the cheese back to its original position.

Scene #3: The actor reaches a bottle, moves it and removes the cap. The actor pours the water, then puts the bottle in the previous position. The actor mixes the ingredients in the bowl.

Scene #4: The actor cuts a slice of bread, spreads some nuts cream on it, then eats it.

Scene #5: The actor reaches a lemon and squeezes it. Then, all the objects are moved away and the actor cleans the table

Citation

Authors using this code in their pubblications should cite this paper:

"The MoCA dataset, kinematic and multi-view visual streams of fine-grained cooking actions"

E. Nicora, G. Goyal, N. Noceti, A. Vignolo, A. Sciutti, F. Odone Scientific Data 7 (1), 1-15