The dataset used for this challenge was collected as part of the FacePsy study. It contains facial behavior and head gesture data aimed at detecting depressive episodes.
The dataset is available for download through this link
Examples of Feature Visualization Collected from FacePsy
The dataset will be published in the following directory structure.
dataset/
├── data/
│ ├── P08.json
│ ├── P10.json
│ ├── ...
│ └── P38.json
├── groundtruth/
│ └── phq9.csv
Structure Description
data/: Contains participant data in JSON format, capturing facial features and behavior during the study. Each entry represents unique facial expressions and orientations captured during specific user events.
Feature Description
Action Units (AU)
Description: Discrete facial movements representing specific facial muscle contractions based on the Facial Action Coding System (FACS).
Features:
AU01, AU02, AU04, AU06, AU07, AU10, AU12, AU14, AU15, AU17, AU23, AU24: Intensity values for each Action Unit.
BoundingBox
Description: Coordinates defining the face region within the image.
Features:
boundingBox: Format is x y width height (e.g., “75 1170 933 1907”).
Classification
Description: Probabilities indicating the state of facial features such as eyes and mouth.
Features:
leftEyeOpenProbability: Probability the left eye is open.
rightEyeOpenProbability: Probability the right eye is open.
smilingProbability: Probability of a smiling expression.
Contours
Description: Points outlining detailed facial features.
Features:
contours: Array of {x, y} coordinates.
File Information
Description: Metadata about the data storage file.
Features:
fileName: Path to the data file.
gameId: Identifier for the cognitive game session. FacePsy also implements cognitive games like Stroop and Visual Spatial Memory Flower Task.
Head Euler Angle
Description: Three-dimensional head orientation.
Features:
X: Pitch
Y: Yaw
Z: Roll
Landmarks
Description: Key points on the face used for tracking and analysis.
Features:
Array of landmarks with types and {x, y} coordinates.
Metadata
Description: Additional details about the data collection process.
Features:
appVersion: Application version used for data collection.
seq_id: Unique sequence identifier. NOTE: seq_id is unique for each data collection session. The FacePsy app collects data about the person when they lock/unlock or use some specific app for the first 10 seconds. Therefore, one seq_id signifies a group of fames processed during the 10-second period. Please see our paper FacePsy for information on data collection session and how to process it.
timestamp: Time when the data was captured.
triggerName: Event or condition that triggered data recording. NOTE: triggerName could also suggest on which app initiated data collection. For example, the values of triggerName are "unlock," "com.whatsapp," and others. triggerName could be used to identify which app the user is using.
PID
Description: Participant identifier.
Features:
pid: Unique identifier for the participant (e.g., “P24”).
groundtruth/: Includes `phq9.csv`, which provides the PHQ-9 scores used as ground truth for depressive episode labeling. The `phq9.csv` ground truth has columns as
pid: Participant ID of the the participants. For each participant, we have one JSON file in the `data/` folder.
start_ts: Start of oberservation period. An observation period has a length of 2 weeks.
end_ts: End of the observation period. NOTE: Some participants may have reported their PHQ-9 a few days after the 2-week period. For those observations, the length may be a few days longer.
start_phq9: PHQ-9 score at the start of observation period.
end_phq9: PHQ-9 score at the end of the observation period.
depression_episode: Depression label. A participant observation period (i.e., 2 weeks) is labeled as having a depressive episode(label=1) if and only if the participant reported PHQ-9 >= 5 both at the start and end of the observation period; otherwise, it is labeled a non-depressive (label=0).
How FacePsy Collects Data?
FacePsy runs on user's device in the background once installed
Triggers data collection when users unlock phones or use apps.
Data is collected for only for first 10 seconds when the trigger initiated
Unobtrusively captures facial behavior primitives during natural smartphone interactions
Features Collected: 12 AU, 1 Smile, 2 Eye open, 3 Pose, and 133 landmark points
Published papers about the dataset
If you use this dataset, please cite the FacePsy paper:
Rahul Islam and Sang Won Bae. 2024. FacePsy: An Open-Source Affective Mobile Sensing System - Analyzing Facial Behavior and Head Gesture for Depression Detection in Naturalistic Settings. The Proceedings of the ACM on Human Computer Interaction. 8, MobileHCI, Article 260 (September 2024), 32 pages. https://doi.org/10.1145/3676505
