CVPR 2022 Tutorial

Contactless Health Monitoring using Cameras and Wireless Sensors

Date and time: June 20, 2022, 9 AM - 12 PM Central Time (US and Canada, GMT-5), fully virtual/online

Zoom link: https://us06web.zoom.us/j/82296105004?pwd=VnZjaUFPUEhxUGNsd29OczhnclNZZz09

Link to CVPR 2022: http://cvpr2022.thecvf.com/

Abstract

Understanding people and extracting health-related metrics is an emerging research topic in computer vision that has grown rapidly recently. Without the need of any physical contact of the human body, cameras have been used to measure vital signs remotely (e.g. heart rate, heart rate variability, respiration rate, blood oxygenation saturation, pulse transit time, body temperature, etc.) from an image sequence of the skin or body, which leads to contactless, continuous and comfortable heath monitoring. The use of cameras also enables the measurement of human behaviors/activities and high-level visual semantic/contextual information leveraging computer vision and machine learning techniques. Understanding of the environment around the people is also a unique advantage of cameras compared to the contact bio-sensors (e.g., wearables), which facilitates better understanding of human and scene for health monitoring. In addition to camera based approach, Radio Frequency (RF) based methods for health monitoring have also been proposed, using Radar, WiFi, RFID, and acoustic signals. Radar based methods mainly use Doppler/UWB/FMCW radar for health monitoring. They can obtain high monitoring accuracy for different applications such as sleeping staging and posture estimation. By using off-the-shelf WiFi device, for example WiFi RSS and CSI data from commodity WiFi NIC, we can monitor breathing and heart rates for single and multiple persons. For acoustic based vital sign monitoring, the speaker and microphone of smartphones are used to build sonar based sensing system to monitor breathing and heart rates. The rapid developments of computer vision and RF sensing also give rise to new multi-modal learning techniques that expand the sensing capability by combining two modalities, while minimizing the need of human labels. The hybrid approach may further improve the performance of monitoring, such as using the camera images as beacon to gear human activity learning for the RF signals. The contactless monitoring of camera and RF will bring a rich set of compelling healthcare applications that directly improve upon contact-based monitoring solutions and improve people’s care experience and quality of life, called “AI health monitoring”, such as in care units of the hospital (patient monitoring), sleep/senior centers (sleep disorders like apnea), assisted-living homes (baby and elderly care), telemedicine and e-health, fitness and sports, driver monitoring in automotive, etc.

Speakers

Wenjin Wang

Southern University of Science and Technology

Xuyu Wang

California State University, Sacramento

Shiwen Mao

Auburn University

Chau-Wai Wong

North Carolina State University

Schedule

Part 1: Camera based vital signs monitoring and its applications

Fundamentals and techniques for vital signs measurement

Pulse rate: different measurement principles and models (blood absorption, BCG motion); core algorithms for pulse extraction (physiological model based); solutions to improve robustness (multi-site measurement, distortion based optimization, etc.), RGB and IR setups (multi-wavelength cameras, time-sequential cameras, RGB2NIR systems, designed light source, auto-camera control, etc.);
Respiration rate: different measurement principles (motion based, temperature based); core algorithms for respiratory signal extraction (optical flow, profile correlation, pixel flow); limitations and sensitivities (body motion, etc.);
Blood oxygen saturation: different multi-wavelength settings (red-IR, full-IR); core algorithms for SpO2 signal extraction; solutions to improve robustness (parallax reduction between cameras, wavelength selection, etc.);

Applications for camera-based health monitoring

Ubiquitous cardio-respiratory screening on mobile devices (with a large benchmark);
Camera-based gating/triggering for Magnetic Resonance Imaging;
Patient monitoring in Hospital Care Units;

Q & A

Part 2: Robust and privacy-aware physiological sensing

1. Physiological signal extraction under low SNR conditions Micro-signal extraction strategies; fitness motion handling; robust tracking of multiple weak frequency traces.

2. Privacy-aware physiological sensing Privacy protection: PulseEdit, identify-preserving transform; adversarial manipulation.

3. PPG2ECG (brief) Biophysical linkage between ECG and PPG; principled vs. data-driven approaches.

Part 3: Wireless based vital sign monitoring and activity monitoring

Radar based vital sign monitoring

Radar Techniques: Continuous-wave (CW) radar, Frequency-Modulated Continuous-Wave (FMCW) Radar, Ultra Wide-band (UWB) radar; Commodity Radar Vital Sign Devices including TI mmWave FMCW Radar and XeThru X4 UWB radar.
Radar based vital sign monitoring and activity recognition applications: breathing and heart rate monitoring, apnea detection, activity recognition using deep Learning.

Acoustic based vital sign monitoring with smartphones

FMCW based vital sign monitoring with acoustic signal: acoustic signal based breathing monitoring on smartphones with FMCW.
CW based vital sign monitoring with acoustic signal: acoustic signal based breathing monitoring on smartphones with CW and the detail Sonar beat system.

WiFi based vital sign monitoring

WiFi system: WiFi techniques; WiFi received signal strength (RSS) and channel state information (CSI).
WiFi based vital sign monitoring: WiFi CSI for breathing and heart rate monitoring, WiFi CSI values for monitoring multi-person breathing beats, robust breathing monitoring with CSI amplitude and phase difference.

Part 4: Multimodal learning of vision and RFID for health monitoring and pose estimation

RFID based vital sign monitoring

RFID system: RFID wireless channel, RFID data, RFID device.
RFID based vital sign sensing and its application to driving fatigue detection: breathing monitoring in indoor and driver environments; apnea detection using unsupervised deep learning; driver fatigue detection method based on head tracking.

Vision-aided 3D human pose estimation using RFID

RFID-Pose, a vision-aided real-time 3D human pose estimation system based on deep learning.
RFID based 3D human pose tracking: A subject generalization approach.
Meta-Pose: Environment-adaptive human skeleton tracking with RFID.

Q & A

Contact

Page updated

Google Sites

Report abuse