Challenge guidelines
Challenge guidelines
The organisers provide the following data for each Sub-challenge:
Only training and development partitions are available; the test partition will be made available few weeks before the end of the Challenge.
Requests for data access can be send to:
A common framework based on open-source toolboxes was exploited to extract baseline features from the audiovisual recordings. It spans three levels of representation: functionals of low-level-descriptors (hand-crafted), bag-of-words, and unsupervised deep representations.
Three well balanced (in terms of age, gender, and labels) speaker-disjoint partitions of the datasets were created for each Sub-challenge: training, development, and test, with about 40% of instances in the training partition and 15% in the development and test partitions. Participants have to stick to the definition of training, development, and test sets as given.
Whereas recordings are provided for all partitions, metadata and labels are not available for the test partition and must be inferred automatically. No manual intervention of any kind on the test partitions is permitted. Test results must be solely the result of a fully automatic process without any human intervention.
Baseline features can be reproduced using the scripts from the Github repository
The number of submissions of test results is limited to five trials per team and Sub-challenge. Test results must be submitted by email to fabien.ringeval@imag.fr and nicholas.cummins@informatik.uni-augsburg.de with the following object: "AVEC 2019 Test Result".
Formatting of the predicted labels on the test sets should follow the original format provided in the training and development sets of the respective Sub-challenges, and be named as follow: sub-challenge_teamname_submission; with sub-challenge being either CES, DDS, or SoMS, teamname the name of the team without accent, space and special character, and submission the number of the submission (1-5); e.g., CES_AVECbaseline_1 is the first submission of test results of the team "AVECbaseline" for the CES.
For the State-of-Mind Sub-challenge (SoMS), a single csv must be submitted and contains at least two columns: File (filename), Valence (predicted label). The level of arousal can be additionally provided as another column.
For the Detecting Depression with AI Sub-challenge (DDS), a single csv must be submitted and contains at least two columns: Participant_ID (filename), PHQ_Score (predicted label). Others information provided in the metadata (PHQ_Binary, PCL-C (PTSD), and PTSD Severity) can also be reported as additional columns; corresponding results will be additionally reported.
For the Cross-cultural Emotion Sub-challenge (CES), a single zip file must be submitted and contains predictions of the test subjects (German, Hungarian, Chinese) as csv files. Each csv file containing the predicted labels must be strictly formatted as the original files; i.e., with five columns for filename, timestamp, arousal, valence, and liking.