CVPR 2019 Tutorial

Camera based Physiological Measurement

Date and time: June 16th (AM), 2019

Location: Long Beach Convention Center (202), Long Beach, CA, U.S.

Tutorial schedule

Part I: Fundamentals

Part II: Methodologies

Part III: Applications

Part I: Fundamentals

      1. Human physiology
          • Cardiovascular and respiratory systems (heart bumps, pulmonary circulation, oxygen/co2 transport and exchange, arterial and venous blood, blood volume changes, etc.)
          • Different types of vital signs that can be measured from skin (pulse-rate, respiration-rate, blood oxygen saturation, etc.)
          • Relations between vital signs and human body conditions/diseases (arrhythmia, cardiac arrest, apnea, sleep disorders, fatigue, stress, cognitive state, well-being, etc.)
      2. Skin optics
          • Skin composition and structure (skin layer, skin chromophores, tissue compositions, etc.)
          • Light interactions with skin-tissues (light penetration depth, scattering, absorption, reflection, wavelength dependency, etc.)
          • Camera-based photoplethysmography (PPG) (basic PPG concepts like DC and AC)

Part II: Methodologies

      1. Camera photoplethysmography
          • Core algorithms (BSS, knowledge based)
          • Fundamental impact of skin melanin and makeup
          • Robust measurement (spatial redundancy, sub-band, discriminative extraction, living skin, CNN, etc.)
      2. Vital signs monitoring
          • Pulse rate (variability)
          • Blood oxygen saturation (Raito-of-Ratio, Adaptive signature)
          • Pulse transit time
          • Ballistocardiogram motion (source or artifact?)
          • Respiration rate (motion-based, PPG-based, thermal-based)
          • Cardiovascular age
          • Context monitoring (comfort/discomfort, pain)
      3. Monitoring setups
          • Multi-wavelength cameras (RGB, multi-sensor NIR, time-sequential NIR, IR-band of RGB, multi-spectral NIR)
          • Light source (sun/day light, fluorescent, incandescent, NIR-LED, polarization)
          • Examples of settings (lab, ICU/NICU, sleep, home, portable, baby, mirror, etc.)

Part III: Applications

      1. Strategy
      2. Clinical trials
          • Emergency department triage, general ward, sleep monitoring at 3 hospitals (as the showcase)
      3. Products
          • Vital signs APP for mobile phone (2011 and 2019 new)
          • VitalEye for MRI respiratory gating
      4. Propositions
          • Touchless baby monitoring
          • Fitness and sports (vital signs camera for gym exercise, swimming)
          • Vital mirror for daily care
          • Automotive monitoring
      5. Media publicity and impact
      6. Academic events

Tutorial organizers

Dr. Wenjin Wang is a scientist at Philips Research Eindhoven, The Netherlands, and assistant professor at Eindhoven University of Technology (TU/e), The Netherlands. He received his PhD from TU/e on the topic of camera based physiological measurement. His current research focuses on camera based contactless vital signs monitoring. His earlier work (2013-2018) in this field improved fundamental understanding and functionalities of the methods (e.g. core algorithms), leading to various peer-reviewed journal/conference publications, patent applications, and systems/applications/prototypes for video health monitoring. He serves as the reviewer for several well-known journals and conferences. He co-chaired the first international workshop of Computer Vision for Physiological Measurement (CVPM) at CVPR 2018. He will also co-organize a tutorial on camera based physiological measurement at CVPR 2019 and co-chair an invited session on optical health monitoring combining vital signs and brain activities at EMBC 2019. He will organize the second CVPM workshop at ICCV 2019 (Seoul, Korea) (Welcome to submit a paper:

Dr. Svitlana Zinger received her MSc in computer science in 2000 from the Radiophysics faculty of the Dnepropetrovsk State University, Ukraine. In 2004, she received a PhD from the Ecole Nationale Superieure des Telecommunications, France, for her thesis on interpolation and resampling of 3D data. In 2005 she was a postdoctoral fellow in the Multimedia and Multilingual Knowledge Engineering Laboratory of the French Atomic Agency, France, where she worked on creation of a large-scale image ontology for content based image retrieval. In 2006-2008 Sveta was a postdoctoral researcher at the Center for Language and Cognition Groningen and an associated researcher at the Artificial Intelligence department in the University of Groningen, the Netherlands, working on information retrieval from handwritten documents. Currently, she is an assistant professor in digital image analysis technologies for healthcare. Her areas of expertise include medical image and video analysis for diagnosis and prognosis. This comprises temporal data analysis, machine learning, context analysis of patient data. Multi-modal data processing and exploration of novel sensing technologies play an important role in this research.

Prof. dr. Gerard de Haan received BSc, MSc, and PhD. degrees from Delft University of Technology in 1977, 1979 and 1992, respectively. He joined Philips Research Laboratories in 1979 to lead research projects in the area of video processing/analysis. From 1988 till 2007, he has additionally taught post-academic courses for the Philips Center for Technical Training. In 2000, he was appointed “Fellow” in the Video Processing & Analysis group of Philips Research Eindhoven, and “Professor” at the Eindhoven University of Technology, as chair of the “Video Processing for Multimedia Systems” group and teaching a course on “Video Processing”.

He has a particular interest in the human visual system, image sequence analysis, computer vision, and physiological measurements. His research has been documented in 4 books and 3 book-chapters, about 200 papers, more than 200 patent applications, and resulted in various consumer electronics ICs for video format conversion and image enhancement.

Since 2008, Gerard de Haan focuses on video health monitoring, which has lead to his appointment in 2016 as chair of the “Video Health Monitoring” group at Eindhoven University. The new focus area has resulted in about 35 papers and 40 patent applications, so far, and a new course on Video Health Monitoring given since 2016.

He received 5 Best Paper Awards, the Gilles Holst Award, the Chester Sall Award, bronze, silver and gold patent medal awards, while his work on motion received the “1995 European Video Innovation Award of the Year” from the European Imaging and Sound Association (EISA) and a "Business Innovation Award" from the Wall Street Journal Europe. Gerard de Haan has served in the program committees of various international conferences on image/video processing and analysis, and has been a Guest-Editor for special issues of Elsevier, IEEE, and Springer.

Supported by