AVEC 2018 introduces major novelties this year with three separated sub-challenges:
- Bipolar Disorder Sub-challenge (BDS) — participants have to classify patients suffering from bipolar disorder into remission, hypo-mania and mania, as defined by the young mania rating scale, from audio-visual recordings of structured interviews (BD corpus); performance is measured by the unweighted average recall over the three classes.
- Cross-cultural Emotion Sub-challenge (CES) — participants have to predict the level of three emotional dimensions (arousal, valence, and likability) time-continuously in a cross-cultural setup (German => Hungarian) from audio-visual recordings of dyadic interactions (SEWA corpus); performance is the concordance correlation coefficient (CCC) averaged over the dimensions.
- Gold-standard Emotion Sub-Challenge (GES) — participants have to generate a reliable gold-standard (i.e., a single time series preserving variance of emotion labels) from individual ratings of emotional dimensions (arousal, valence) that will be evaluated by a baseline multimodal (audio, video, physiology) emotion recognition system from recordings of dyadic interactions (RECOLA corpus); performance is the concordance correlation coefficient (CCC) averaged over the dimensions, and a condition on the unexplained variance between the generated gold-standard and the (original) individual annotations is used to ensure that sufficient variability is preserved in the labels.
The AVEC 2018 BDS is a new first of its kind task in the scope of mental health analysis. Whereas the topic of depression analysis was featured in the previous editions of AVEC, we introduce this year for the first time in a challenge the analysis of bipolar disorder, which rank among the top-ten mental disorder for adults according to the World Health Organisation. The dataset used for the AVEC 2018 BDS includes audiovisual recordings of structured interviews performed by 47 Turkish speaking subjects aged 18-53 (BD corpus). All those subjects suffered from bipolar disorder and were recruited from a mental health service hospital where they were diagnosed by clinicians following DSM-5’s inclusion criteria.
The AVEC 2018 CES is a major extension of the Emotion Sub-Challenge previously run in AVEC 2017 and based on the SEWA dataset (German culture). This dataset includes online video chats – recorded “in-the-wild”, i.e., with standard webcams and at home/work place – between 32 pairs of participants aged 18-60+ and discussing about advertisements they watched. This dataset has been extended and now includes data collected from 32 new pairs of participants with Hungarian culture and in the same age range as for the German culture. For the AVEC 2018 CES, the extended version of the SEWA dataset is used in the context of the first ever cross-cultural emotion recognition competition task, in order to quantify how emotion knowledge of a culture can be transferred to another: participants have data from one culture (German) to train and optimise an audio-visual emotion recognition system, and data from the second culture (Hungarian) is used solely for evaluation purposes (labels unknown).
The AVEC 2018 GES is a new task focusing on the generation of dimensional emotion labels that are used both for training and testing emotion recognition systems. The task consists in creating a single time-series of emotion labels, usually referred as “gold-standard”, from a pool of time-continuous annotations of dimensional emotions provided by several annotators. Many difficulties arise in the generation of this gold-standard, as inconsistencies appear in the reported annotation values – even for a single evaluator –, and a delay is present between the emotional event expressed in the data and the corresponding annotation value reported (time-continuously) by the annotator. Moreover, some annotators might be more reliable than others according to their familiarity with the expressed behaviours. In this Sub-Challenge, participants have to generate a reliable gold-standard from individual ratings of dimensional emotions that will be then used to train and evaluate a baseline multimodal emotion recognition system on the RECOLA dataset, which includes audio-visual and physiological recordings of dyadic interactions from 27 French speaking subjects aged 18-25. By reliable, we mean that the produced gold-standard should not only maximise the performance of the baseline emotion recognition system, but it also needs to preserve a sufficient amount of the variance found in the original individual annotations to be validated.