Dataset

Download Dataset and Code

Description

The dataset includes 20 cooking actions, involving either a single or both arms of the volunteer, some of them including tools which may require different forces. Three different view-points have been considered for the acquisitions, i.e. lateral, egocentric, and frontal. For each action a training and a test sequence is available, each containing, on average, 25 repetitions of the action. Furthermore, acquisitions of more structured activities are included, in which the actions are performed in sequence for a final, more complex goal.

An annotation is available, which includes the segmentation of single action instances in terms of time instants in the MoCap reference frame. A function then allows to map the time instants on the corresponding frame in the video sequences. In addition, functionalities to load, segment, and visualize the data are also provided in Python and Matlab.

List of the included actions:

  1. Cutting the bread

  2. Shredding a carrot

  3. Cleaning a dish

  4. Eating

  5. Beating eggs

  6. Squeezing a lemon

  7. Mincing with a mezzaluna

  8. Mixing in a bowl

  9. Opening a bottle

  10. Turning the frittata in a pan

  11. Pestling

  12. Pouring water in multiple containers

  13. Pouring water in a mug

  14. Reaching an object

  15. Rolling the dough

  16. Washing the salad

  17. Salting

  18. Spreading cheese on a slice of bread

  19. Cleaning the table

  20. Transporting an object

Test Activities:

Scene #1: The actor mixes ingredients in a bowl, then adds salt and pours some water. Finally the ingredients are mixed again.


Scene #2: The actor reaches a slices of cheese, grabs it and shreds it. Then, the actor moves the cheese back to its original position.


Scene #3: The actor reaches a bottle, moves it and removes the cap. The actor pours the water, then puts the bottle in the previous position. The actor mixes the ingredients in the bowl.


Scene #4: The actor cuts a slice of bread, spreads some nuts cream on it, then eats it.


Scene #5: The actor reaches a lemon and squeezes it. Then, all the objects are moved away and the actor cleans the table

Citation

Authors using this code in their pubblications should cite this paper:


"The MoCA dataset, kinematic and multi-view visual streams of fine-grained cooking actions"

E. Nicora, G. Goyal, N. Noceti, A. Vignolo, A. Sciutti, F. Odone Scientific Data 7 (1), 1-15