Code Submission

Each participant team should submit the following documents: 

1. Python-based source code; 

2. Checkpoint of the final model; 

3. A detailed and accurate README file; 

4. Results achieved on validation and test sets.

Submission via email to: reactmultimodalchallenge@gmail.com

Submission deadline: March 15th 2024 (Anywhere of Earth)

=======================================================================================

1.           Source code 

Please include all parts of the system, including training scripts, evaluation scripts, models, pre-processing scripts, etc.

We encourage participants to keep their codes having similar environment and style as our baseline code provided at https://github.com/reactmultimodalchallenge/baseline_react2024 . Alternatively, participants should submit a docker file including the code and the corresponding specific environment.

The data file directory must be consistent with the baseline code (please refer to the file ‘data.py’ provided by the baseline).

We encourage participants to use 3DMM and PIRender-based strategy for facial reaction visualization. Please provide the details of the visualization if other strategy is applied.

2.           Model and checkpoint

Please provide the following model definition python file and the corresponding checkpoints:

• Facial reaction attributes generation model

• Facial reaction 2D/3D image generation models

3.  README file

• Indicating the model type: offline facial reaction generation model or online facial reaction generation model.

• Detailed description of the coding environment and dependencies.

• Detailed description of the input and output (including facial attributes and 2D/3D facial images) of the model.

• Detailed description of dataloader, pre-processing and training settings (including training steps, hyper-parameters, etc.)

• Detailed description of visualization settings, including how to generate 2D/3D facial reaction images.

• Indicating the data directory. The organizer will directly replace the ‘data directory’ in the submitted code with our data directory, and thus the data file directory must be consistent with the baseline code.

• A ‘.txt’ file clearly listed several ‘command lines’, where these command lines should allow organizer to: (1) conduct model training; (2) load pre-trained weights and evaluate it on validation/test set; (3) generate 25D facial reaction attributes and 2D/3D facial reaction frames.

4. Results achieved on validation and test sets

• Each participant team should list seven metrics (FRCorr, FRDist, FRDiv, FRVar, FRDvs, FRRea, FRSyn) achieved by their models on validation and test set, separately.

• Each participant team should submit visualization of 4 facial reaction videos (.mp4 or .avi file, where each has at least 720 frames at fps=30), which should be corresponding to the following speaker videos:

1.                   NoXI/001_2016-03-17_Paris/Expert_video/21,

2.                    NoXI/023_2016-04-25_Paris/Expert_video/25,

3.                   NoXI/019_2016-04-20_Paris/Expert_video/13，

4.                   RECOLA/group-2/P41/2