CVPR 2021 Tutorial
Computer Vision for Physiological Measurement and Heath care
Embedded Files
Date and time: June 19, 2021 (fully virtual/online)
Date and time: June 19, 2021 (fully virtual/online)
Location: the Zoom link can be found on CVPR, you need to be registered to access the tutorials
Location: the Zoom link can be found on CVPR, you need to be registered to access the tutorials
Abstract
Abstract
Understanding people and extracting health-related metrics is an emerging research topic in computer vision (CV) that has grown rapidly recently. Without the need of any physical contact of the human body, cameras have been used to measure vital signs remotely (e.g. heart rate, heart rate variability, respiration rate, blood oxygenation saturation, pulse transit time, body temperature, etc.) from an image sequence of the skin or body, which leads to contactless, continuous and comfortable heath monitoring. The use of cameras also enables the measurement of human behaviors/activities and high-level visual semantic/contextual information, such as facial expression analysis for pain/discomfort/delirium detection, image based body mass index analysis, emotion recognition for depression measurement, body motion for sleep staging or bed exit/fall detection, activity recognition for patient actigraphy, etc. Understanding of the environment around the people is another unique advantage of cameras compared to the contact bio-sensors (e.g., wearables), which facilitates better understanding of human and scene for health monitoring (e.g. clinical workflow monitoring and optimization). The contactless monitoring of camera will bring a rich set of compelling CV and healthcare applications that directly improve upon the human’s life and care experience, called “AI healthcare”, such as in hospital care units (intensive care, neonatal care), automatic gating/triggering for medical imaging (MR/CT), sleep/senior centers, assisted-living homes, telemedicine and e-health, home-based baby and elderly care, fitness and sports, driver monitoring in automotive, AR/VR based therapy, etc. Especially in the background of COVID-19, contactless health monitoring of cameras can potentially serve in controlling the pandemic, such as vital signs screening, home-based monitoring, social distancing alarm, etc.
Understanding people and extracting health-related metrics is an emerging research topic in computer vision (CV) that has grown rapidly recently. Without the need of any physical contact of the human body, cameras have been used to measure vital signs remotely (e.g. heart rate, heart rate variability, respiration rate, blood oxygenation saturation, pulse transit time, body temperature, etc.) from an image sequence of the skin or body, which leads to contactless, continuous and comfortable heath monitoring. The use of cameras also enables the measurement of human behaviors/activities and high-level visual semantic/contextual information, such as facial expression analysis for pain/discomfort/delirium detection, image based body mass index analysis, emotion recognition for depression measurement, body motion for sleep staging or bed exit/fall detection, activity recognition for patient actigraphy, etc. Understanding of the environment around the people is another unique advantage of cameras compared to the contact bio-sensors (e.g., wearables), which facilitates better understanding of human and scene for health monitoring (e.g. clinical workflow monitoring and optimization). The contactless monitoring of camera will bring a rich set of compelling CV and healthcare applications that directly improve upon the human’s life and care experience, called “AI healthcare”, such as in hospital care units (intensive care, neonatal care), automatic gating/triggering for medical imaging (MR/CT), sleep/senior centers, assisted-living homes, telemedicine and e-health, home-based baby and elderly care, fitness and sports, driver monitoring in automotive, AR/VR based therapy, etc. Especially in the background of COVID-19, contactless health monitoring of cameras can potentially serve in controlling the pandemic, such as vital signs screening, home-based monitoring, social distancing alarm, etc.
Schedule
Schedule
Part 1: Camera based vital signs monitoring and applications
Part 1: Camera based vital signs monitoring and applications
Fundamentals and techniques for vital signs measurement
Pulse rate: different measurement principles and models (blood absorption, BCG motion); core algorithms for pulse extraction (physiological model based); solutions to improve robustness (multi-site measurement, distortion based optimization, etc.), RGB and IR setups (multi-wavelength cameras, time-sequential cameras, RGB2NIR systems, designed light source, auto-camera control, etc.);
Respiration rate: different measurement principles (motion based, temperature based); core algorithms for respiratory signal extraction (optical flow, profile correlation, pixel flow); limitations and sensitivities (body motion, etc.);
Blood oxygen saturation: different multi-wavelength settings (red-IR, full-IR); core algorithms for SpO2 signal extraction; solutions to improve robustness (parallax reduction between cameras, wavelength selection, etc.);
Other health informatics from camera: baby comfort/discomfort monitoring; respiration based posture detection, breathing area (mouth or nose) detection for sleep apnea diagnosis; body mass index measurement, patient actigraphy, etc.
Applications for health care
Ubiquitous cardio-respiratory screening on mobile devices (with a large benchmark);
Camera-based gating/triggering for Magnetic Resonance Imaging;
Patient monitoring in Hospital Care Units;
Drive health monitoring in automotive;
Fitness cardio-tracking in sport exercise;
Q & A
Part 2: Contact-free, pain-free, infection-free measurement of cardiac arrhythmia in a clinical environment
Part 2: Contact-free, pain-free, infection-free measurement of cardiac arrhythmia in a clinical environment
Cardiac arrythmias: Introduce to different arrythmias that are measured in a clinical environment with ECG. Among these which ones can be detected automatically for cardiac function assessment using cameras & AI algorithms
Arrythmias measurable by camera system: Few examples of arrythmias measurable by contact-free system are; atrial fibrillation, ventricular premature contraction and ventricular tachycardia. How representation learning and deep learning algorithms have the potential to create accurate detection suitable for long observation and monitoring periods with/without subject’s cooperation. Can the multi-class decision algorithms provide the key to separating different arrythmias? How good is the decision when compared to ECG-based approach? We will present results of our study conducted at URMC Heart Research Center
Q&A
Part 3: Machine learning approaches for remote physiological measurement from facial videos
Part 3: Machine learning approaches for remote physiological measurement from facial videos
Spatial-temporal networks for remote PPG signal recovery
3DCNN based network vs. RNN based network,
Skin segmentation-based attention module,
ROI partition constraints,
video enhancement network for rPPG measurement from highly compressed videos
Applications
Atrial fibrillation detection: the OBF dataset and approaches
Physiological monitoring in remote group psychotherapy: preliminary findings
Others: rPPG for biometric security, face anti-spoofing detection
Q & A
Part 4: Video-based remote physiological measurement via signal disentangling and cross-verification
Part 4: Video-based remote physiological measurement via signal disentangling and cross-verification
CNN based remote physiological measurement by converting a video clip to one still “image”
From temporal to spatial: converting a video clip to one still “image”
VIPL-HR database construction and release
Some tricks for CNN training with insufficient data
Signal disentangling and cross-verification for heart rate measurement
Disentangling the physiological signal and the non-physiological “noise” from a video
Cross-verification in a self-supervised mode. To validate the “correctness” of the disentangling, given two input videos, their disentangled signal and noise are exchanged to generate pseudo-videos, which are then disentangled again, leading to reconstruction losses and “self-supervised” signal-losses and noise-losses.
Overall loss function and the experimental analysis
Q & A
Speakers
Speakers
Wenjin Wang
Philips Research
Gerard de Haan
TU Eindhoven
Lalit K. Mestha
KinetiCor, UT Arlington
Xiaobai Li
University of Oulu
Shiguang Shan
Chinese Academy of Sciences
Page updated
Google Sites
Report abuse