Automatic Group Level Affect Analysis

Group Level Emotion:

Abstract: Understanding of emotion of a group of people in an image or video is an important problem. For details on the Group-level emotion recognition, a talk by Prof. Goecke is available on video lectures at: http://videolectures.net/fgconference2015_goecke_people_images/

EmotiW Subchallenges:

1. Sixth EmotiW Challenge was organised at ACM ICMI 2018, Boulder. [EmotiW 2018]

2. Fifth EmotiW Challenge was organised at ACM ICMI 2017, Glasgow. [EmotiW 2017]

3. Fourth EmotiW Challenge was organised at ACM ICMI 2016, Tokyo. [EmotiW 2016]

Publications:

1. Abhinav Dhall and Roland Goecke, Group expression intensity estimation in videos via Gaussian Processes, International Conference on Pattern Recognition 2012. [PDF]

2. Abhinav Dhall, Jyoti Joshi, Ibrahim Radwan, Roland Goecke , Finding happiest moments in a social context, Asian Conference on Computer Vision 2012. [PDF]

3. Abhinav Dhall, Jyoti Joshi, Karan Sikka, Roland Goecke and Nicu Sebe, The More the Merrier: Analysing the Affect of a Group of People In Images, IEEE Automatic Faces & Gesture Recognition (FG) 2015. [PDF]

4. Abhinav Dhall, Roland Goecke and Tom Gedeon, Automatic Group Happiness Intensity Analysis, IEEE Transaction on Affective Computing 2015. [PDF]

5. Xiaohua Huang, Abhinav Dhall, Guoying Zhao, Roland Goecke, Matti Pietikäinen, Riesz-based Volume Local Binary Pattern and A Novel Group Expression Model for Group Happiness Intensity Analysis, British Machine Vision Conference 2015. [PDF]

6. Abhinav Dhall, Roland Goecke, Shreya Ghosh, Jyoti Joshi, Jesse Hoey, Tom Gedeon. From individual to group-level emotion recognition: emotiw 5.0. In ACM-ICMI 2017. [PDF] [Github]

7. Xiaohua Huang, Abhinav Dhall, Xin Liu, Guoying Zhao, Jingang Shi, Roland Goecke, Matti Pietikainen, Analyzing the Affect of a Group of People Using Multi-modal Framework, IEEE Transactions on Multimedia 2018. [PDF]

8. Shreya Ghosh, Abhinav Dhall, Nicu Sebe. Automatic group affect analysis in images via visual attribute and feature networks. IEEE-ICIP 2018. [PDF] [Github]

9. Garima Sharma, Shreya Ghosh and Abhinav Dhall. Automatic Group Level Affect and Cohesion Prediction in Videos. ACII, 2019. [PDF]

Publications in EmotiW Subchallenges:

EmotiW 2018:

1. Group-Level Emotion Recognition using Hybrid Deep Models based on Faces, Scenes, Skeletons and Visual Attentions - X Guo, B Zhu, LF Polanía, C Boncelet .

2. Cascade Attention Networks For Group Emotion Recognition with Face, Body and Image Cues - K Wang, X Zeng, J Yang, D Meng, K Zhang .

3. Group-Level Emotion Recognition using Deep Models with A Four-stream Hybrid Network - AS Khan, Z Li, J Cai, Z Meng, J O'Reilly .

4. An Attention Model for group-level emotion recognition - A Gupta, D Agrawal, H Chauhan, J Dolz .

EmotiW 2017:

1. Group Emotion Recognition with Individual Facial Emotion CNNs and Global Image Based CNNs - L Tan, K Zhang, K Wang, X Zeng, X Peng and Y Qiao.

2. Group-Level Emotion Recognition using Deep Models on Image Scene, Faces, and Skeletons - X Guo, L Polania and K Barner.

3. A New Deep-Learning Framework for Group Emotion Recognition - Q Wei, Y Zhao, Q Xu, L Li, J He, Ln Yu and B Sun.

EmotiW 2016:

1. Happiness Level Prediction with Sequential Inputs via Multiple Regressions - J. Li, S. Roy, J. Feng, and T. Sim.

2. Group happiness assessment using geometric features and dataset balancing - V. Vonikakis, Y. Yazici, V. Dung Nguyen, and S. Winkler.

3. LSTM for Dynamic Emotion and Group Emotion Recognition in the Wild - B. Sun, Q. Wei, L. Li, Q. Xu, J. He, and L. Yu.

Group Level Cohesion:

Abstract: Cohesiveness of a group is an essential indicator of emotional state, structure and success of a group of people. We study the factors that influence the perception of group level cohesion and propose methods for estimating the human-perceived cohesion on the Group Cohesiveness Scale (GCS). Based on the Group Affect database, we add GCS and propose the 'GAF-Cohesion database'. It is interesting to note that GCS as an attribute, when jointly trained for group level emotion prediction, helps in increasing the performance for the later task. This suggests that group level emotion and GCS are correlated.

EmotiW Subchallenge:

Seventh EmotiW Challenge will be organised at ACM ICMI 2019. [EmotiW 2019]

Publications:

1. Shreya Ghosh, Abhinav Dhall, Nicu Sebe, Tom Gedeon (2019). Predicting Group Cohesiveness in Images. In IJCNN 2019 [PDF]

2. Shreya Ghosh, Abhinav Dhall, Nicu Sebe, Tom Gedeon. Automatic Prediction of Group Cohesiveness in Images. In IEEE Transactions on Affective Computing 2020. [Link]

Publications in EmotiW Subchallenges:

EmotiW 2019:

1. Exploring Regularizations with Face, Body and Image Cues for Group Cohesion Prediction - Guo, D., Wang, K., Yang, J., Zhang, K., Peng, X. and Qiao, Y.

2. Group-Level Cohesion Prediction Using Deep Learning Models with A Multi-Stream Hybrid Network-Xuan Dang, T., Kim, S.H., Yang, H.J., Lee, G.S. and Vo, T.H.

3. Automatic group cohesiveness detection with multi-modal features - Zhu, B., Guo, X., Barner, K. and Boncelet, C.

Most Influential Person:

Abstract: Group affect analysis is an important cue for predicting various group traits. Generally, the estimation of the group affect, emotional responses, eye gaze and position of people in images are the important cues to identify an important person from a group of people. The main focus of this study is to explore the importance of group affect in finding the representative of a group. We call that person the" Most Influential Person"(for the first impression) or" leader" of a group. In order to identify the main visual cues for" Most Influential Person", we conducted a user survey. Based on the survey statistics, we annotate the" influential persons" in 1000 images of Group AFfect database (GAF 2.0) via LabelMe toolbox and propose the" GAF-personage database".

Publication:

1. Shreya Ghosh, Abhinav Dhall (2018). Role of group level affect to find the most influential person in images. In ECCV workshop (HBUGEN 2018)[PDF] [Github]

Contact:

• Abhinav Dhall (abhinav.dhall@monash.edu)

• For accessing any of the databases above, please email at abhinav[DOT]dhall[at]monash[DOT]edu.

Contributors:

• Abhinav Dhall

• Roland Goecke

• Tom Gedeon

• Nicu Sebe

• Xiaohua Huang

• Shreya Ghosh

• Garima Sharma