Atrial Fibrillation Detection from face video

Detecting atrial fibrillation risk without biomedical acquisition device?

Atrial Fibrillation Detection from Face Video by Fusing Subtle Variations. Submitted to IEEE Trans. on Circuits and Systems for Video Technology.

[Main code] [pdf]

An illustration for the proposed remote AF detection method. The traditional AF detection approach is presented in the left part of the picture, which requires to put specific sensors on the body and utilize biomedical devices for capturing the ECG signal. As shown in the right part of the picture, the proposed remote AF detection model aims to detect AF from face video in a contactless manner, which is very convenient to monitor the AF risk in daily life since it only needs a common RGB camera.

Atrial Fibrillation (AF) is one of the most common cardiac arrhythmia, which particularly occurs in the elderly and individuals with heart disease. Though AF is often asymptomatic in normal activities, it has huge potential risks for stroke and other severe diseases. Thus, early detection of AF has great importance in the field of public health. Currently, electrocardiography (ECG) is the commonly used measure for diagnosis of AF, which presents irregular rhythm of waveform for AF patients. However, the measurement of ECG signal requires special medical acquisition device, which is not comfortable for practical monitoring in daily life. In this paper, we explore a very promising algorithm to detect AF from remote face video by analysing the color variations of face skin. The main challenge is that current remote photoplethysmography (rPPG) technique is rather immature, which causes the difficulty to extract accurate pulse signals for describing cardiac rhythm. To solve the above problem, we first utilize various rPPG algorithms to capture pulse rhythm from different regions on the face video. We then investigate biomedical statistical methods to extract suitable features from each pulse signal. Due to the imprecision of video-extracted pulse signals, some traditional physiological features may lose the usage since they were originally proposed for ECG signals. Furthermore, some of them are very susceptible to the influence of noise. Thus, we propose a feature fusion algorithm to select and combine reasonable information from multiple physiological features, which aims to preserve the discriminability to detect AF from the disturbance of noises and outliers. Experimental results on a real-world database demonstrate the effectiveness of the proposed method to provide useful information for AF detection.

Pipeline for atrial fibrillation detection


Training phase:

1 Track the 68 landmarks by Openface [Link]

2 Detect the 21 ROIs by the connection of specific landmarks

3 Utilize three pulse extraction methods [1-3] to generate the heart beat rhythm from each ROI. Thus, 63 pulse rhythms are obtained totally.

4 Perform peak detection on each pulse rhythm to compute the RRI signal and further extract the HRV features.

5 Concatenate all the 63 feature vectors. Conduct the proposed feature fusion and selection algorithm on the training set to obtain the projection matrix P.

6 Train the SVM model.


Testing phase:

1 Track the 68 landmarks by Openface [Link]

2 Detect the 21 ROIs by the connection of specific landmarks

3 Utilize three pulse extraction methods [1-3] to generate the heart beat rhythm from each ROI. Thus, 63 pulse rhythms are obtained totally.

4 Perform peak detection on each pulse rhythm to compute the RRI signal and further extract the HRV features.

5 Concatenate all the 63 feature vectors. Utilize the pretrained projection matrix P to obtain the discriminative feature for classification.

6 Predict the final result (healthy/AF) by SVM.

[1] X. Li, I. Alikhani, J. Shi, T. Seppanen, J. Junttila, K. Majamaa- Voltti, M. Tulppo, and G. Zhao, “The OBF database: A large face video database for remote physiological signal measurement and atrial fibrillation detection,” in IEEE FG 2018. pp. 242–249.[2]L. Feng, L.-M. Po, X. Xu, Y. Li, and R. Ma, “Motion-resistant remote imaging photoplethysmography based on the optical properties of skin,” IEEE Trans. on Circuits and Systems for Video Technology, vol. 25, no. 5, pp. 879–891, 2015.[3]W. Wang, S. Stuijk, and G. De Haan, “A novel algorithm for remote photoplethysmography: Spatial subspace rotation.” IEEE Trans. Biomedical Engineering, vol. 63, no. 9, pp. 1974–1984, 2016.

Database

The experiments are performed on the Oulu Bio-Face (OBF) database, which was captured at the University of Oulu for healthy participants and Oulu University Hospital for clinical patients. In the experiments, we utilize the resting-state recordings of healthy individuals and prior-treatment recordings of patients in the database, which respectively indicate the healthy and AF samples. At the time of preparing this paper, the database contains 100 healthy individuals and 35 AF patients. The protocols of experiments are described in the original paper. Ethical application was submitted and approved by the hospital ethical council. The consent (regulated by the rules of research conducted on patients at hospital) was signed by each patient before the data collection. The patients were accompanied and handled by professional cardiologists and nurses through the whole procedure.

Composition of the database

General statistical information of the participants are summarized in follows:



Age (y)


Gender

Race



Healthy individual

70 people (under 35), 25 people (from 35 to 50)

and 5 people (above 50)

61% M , 39% F

Caucasian: 32%, Asian: 37%,Others: 31%


AF Patient

4 people (under 50), 21 people (from 50 to 70)

and 10 people (above 70)

85.7% M , 14.3 % F

Caucasian: 100%


Samples in the database (Left: healthy individual. Right: AF patient)