This site hosts action-image dataset which can be used to evaluate action retrieval in images.

The data are used in the following CVPR'2015 paper (downloaded below).
"Learning semantic relationships for better action retrieval in images",
by Vignesh Ramanathan, Congcong Li, Jia Deng, Wei Han, Zhen Li, Kunlong Gu, Yang Song, Samy Bengio, Charles Rosenberg, Li Fei-Fei