Fine-Grained Sports, Yoga, and Dance Postures
Recognition: A Benchmark Analysis
Introducing Sports-102 and Dance-12 Image datasets
Asish Bera, Mita Nasipuri, Ondrej Krejcar, and Debotosh Bhattacharjee
Abstract
Human body-pose estimation is a complex problem in computer vision. Recent research interests have been widened specifically on the Sports, Yoga, and Dance (SYD) postures for maintaining health conditions. The SYD pose categories is regarded as a fine-grained image classification task due to complex movement of body-parts. Convolutional Neural Networks (CNNs) have attained significant performance improvement to solve various human body-pose estimation problems. Though, decent progress has been achieved in yoga postures recognition using deep learning techniques, yet, fine-grained sport and dance actions recognition necessitates ample research attention. However, no benchmark public image dataset with sufficient variations is available yet to address sports and dance postures classification. In this work, we have proposed two image datasets, one for 102 sports categories, and another for 12 dance styles. For yoga postures, two public datasets, Yoga-82 (contains 82 classes) and Yoga-107 (represents 107 classes) are collected. These four SYD datasets are experimented with the proposed deep model, namely SYD-Net that integrates a patch-based attention (PbA) mechanism on the top of standard backbone CNNs. The PbA module leverages the self-attention mechanism that learns contextual information from uniform and multi-scale patches, and emphasizes discriminative features to understand the semantic correlation among the patches. The random erasing data augmentation is applied for improving the performance. The proposed SYD-Net has achieved state-of-the-art accuracy on Yoga-82 using five standard base CNNs. Also, SYD-Net’s accuracy on other datasets are remarkable implying its efficiency.
This paper has been accepted in IEEE Transactions on Instrumentation and Measurement, June 2023. Download the preprint: SYD-Net
The paper can also be downloaded from IEEE. Link
Please cite the paper if you use the dataset.
@article{bera2023fine,
title={Fine-Grained Sports, Yoga, and Dance Postures Recognition: A Benchmark Analysis},
author={Bera, Asish and Nasipuri, Mita and Krejcar, Ondrej and Bhattacharjee, Debotosh},
journal={IEEE Transactions on Instrumentation and Measurement},
year={2023},
publisher={IEEE}
}
The contributions of this work are:
• A patch-based attention mechanism that summarizes the discriminativeness of partial feature descriptors for sports, yoga, and dance poses recognition.
• A new image-dataset for sports with 102 classes, and another dataset representing 12 dance styles are proposed for posture classification avoiding part-based/skeletal-joint/bounding-box information.
• The proposed SYD-Net approach achieves state-of-the-art accuracy on the Yoga-82 and Yoga-107 datasets.
Summary of the proposed datasets and top-1 Accuracy (%) of SYD-Net using two backbone CNNs trained with ImageNet weights:
Dataset #Train #Test # Class #Accuracy Xception (ImageNet) #Accuracy MobileNet-v2 (ImageNet)
Dance-12 3129 1694 12 97.98 97.41
Sports-102 9278 4315 102 98.86 98.70
The datasets can be downloaded using the following link:
https://data.mendeley.com/datasets/dxcdv8652s/1
Dance-12 Dataset Link: Dance-12
Sports-102 Dataset Link: Sports-102
Sample images from the Dance-12 dataset are illustrated below:
Ballet
Bharatnatyam
Chhau
Dandya
Dhunuchi
Hip-Hop
Kalbelia
Kathak
Manipuri
Pole
Salsa
Samba
Sample images from the Sports-102 dataset are illustrated below:
Cricket
Hockey
Horse Jumping
Javelin
Kabaddi
Skating
Table Tennis
WWE
Please Note: All images of these datasets are collected from the Internet which are not property of Author's Organizations. All contributing authors and their organizations are neither responsible for the content nor the meaning of these images. These are used for research purpose only. No commercial issues are involved. We provide web links of images used in making of this dataset along with train and test splits.
Terms and Conditions:
The Dance-12 and Sports-102 datasets are available for non-commercial research and educational purposes only.
If researcher reproduce images in electronic or print media, kindly check individual image copyright for reproduction.
Researcher is allowed to redistribute the dataset (in original form only that available on this webpage), if third party agrees to these conditions.
If Researcher is employed by a for-profit, commercial entity, Researcher's employer shall also be bound by these terms and conditions, and Researcher hereby represents that he or she is fully authorized to enter into this agreement on behalf of such employer.
To access the dataset for research purpose only, please send a detailed email to the contact person, given below.
Questions?
Contact [asish.bera@pilani.bits-pilani.ac.in] to get more information about SYD-Net