CVPR 2019 Tutorial

Camera based Physiological Measurement

Date and time: June 16th (AM), 2019

Location: Long Beach Convention Center (202), Long Beach, CA, U.S.

Tutorial schedule

Part I: Fundamentals

Part II: Methodologies

Part III: Applications

Part I: Fundamentals

      1. Human physiology
          • Cardiovascular and respiratory systems (heart bumps, pulmonary circulation, oxygen/co2 transport and exchange, arterial and venous blood, blood volume changes, etc.)
          • Different types of vital signs that can be measured from skin (pulse-rate, respiration-rate, blood oxygen saturation, etc.)
          • Relations between vital signs and human body conditions/diseases (arrhythmia, cardiac arrest, apnea, sleep disorders, fatigue, stress, cognitive state, well-being, etc.)
      2. Skin optics
          • Skin composition and structure (skin layer, skin chromophores, tissue compositions, etc.)
          • Light interactions with skin-tissues (light penetration depth, scattering, absorption, reflection, wavelength dependency, etc.)
          • Camera-based photoplethysmography (PPG) (basic PPG concepts like DC and AC)


Part II: Methodologies

      1. Hardware setups
          • Multi-spectral cameras in RGB and NIR (different settings and functionalities)
          • Illumination source (characteristics of different light sources)
          • Skin properties (challenges and opportunities)
      2. Core algorithms for vital signs extraction
          • Pulse-rate (variability) (BSS-based, model-based, signature-based, CNN-based)
          • Blood oxygen saturation (Raito-of-Ratio, Adaptive-PBV)
          • Pulse transit time (phase-shift of PPG-signals from different skin-sites)
          • Ballistocardiogram motion (BCG as an artifact or source for JVP measurement)
          • Respiration rate (motion-based, PPG-based, thermal-based)
          • Core body temperature (thermal-based method and PPG imaging)
          • Feasibility of other vital signs (glucose, Mayer wave, etc.)
      3. Strategies to automate and optimize the measurement
          • Living-skin detection (remote-PPG based)
          • Multi-site optimization (spatial redundancy, local vital signs selection)
          • Distortion-based optimization (using distortion artifacts to improve measurement)


Part III: Applications

      1. Clinical health monitoring
          • Sleep monitoring, neonatal care unit, MRI respiration gate, emergency department triage, general ward, patient fall detection, bed exit detection, patient actigraphy, etc.
      2. Home-based daily care
          • Vital signs APP for mobile device, Google Glass APP, etc.
      3. Fitness cardio training
      4. Driver monitoring in automotive
      5. Other applications

Tutorial organizers

Dr. Wenjin Wang is a scientist at Philips Research Eindhoven, The Netherlands, and visiting researcher at Eindhoven University of Technology (TU/e), The Netherlands. He received his PhD from TU/e on the topic of camera based physiological measurement. His current research focuses on camera based contactless vital signs monitoring. His earlier work (2013-2018) in this field improved fundamental understanding and functionalities of the methods (e.g. core algorithms), leading to various peer-reviewed journal/conference publications, patent applications, and systems/applications/prototypes for video health monitoring. He serves as the reviewer for several well-known journals and conferences. He co-chaired the first international workshop of Computer Vision for Physiological Measurement at CVPR 2018. He will also co-organize a tutorial on camera based physiological measurement at CVPR 2019 and co-chair an invited session on optical health monitoring combining vital signs and brain activities at EMBC 2019.

Dr. Svitlana Zinger received her MSc in computer science in 2000 from the Radiophysics faculty of the Dnepropetrovsk State University, Ukraine. In 2004, she received a PhD from the Ecole Nationale Superieure des Telecommunications, France, for her thesis on interpolation and resampling of 3D data. In 2005 she was a postdoctoral fellow in the Multimedia and Multilingual Knowledge Engineering Laboratory of the French Atomic Agency, France, where she worked on creation of a large-scale image ontology for content based image retrieval. In 2006-2008 Sveta was a postdoctoral researcher at the Center for Language and Cognition Groningen and an associated researcher at the Artificial Intelligence department in the University of Groningen, the Netherlands, working on information retrieval from handwritten documents. Currently, she is an assistant professor in digital image analysis technologies for healthcare. Her areas of expertise include medical image and video analysis for diagnosis and prognosis. This comprises temporal data analysis, machine learning, context analysis of patient data. Multi-modal data processing and exploration of novel sensing technologies play an important role in this research.

Prof. dr. Gerard de Haan received BSc, MSc, and PhD. degrees from Delft University of Technology in 1977, 1979 and 1992, respectively. He joined Philips Research Laboratories in 1979 to lead research projects in the area of video processing/analysis. From 1988 till 2007, he has additionally taught post-academic courses for the Philips Center for Technical Training. In 2000, he was appointed “Fellow” in the Video Processing & Analysis group of Philips Research Eindhoven, and “Professor” at the Eindhoven University of Technology, as chair of the “Video Processing for Multimedia Systems” group and teaching a course on “Video Processing”.

He has a particular interest in the human visual system, image sequence analysis, computer vision, and physiological measurements. His research has been documented in 4 books and 3 book-chapters, about 200 papers, more than 200 patent applications, and resulted in various consumer electronics ICs for video format conversion and image enhancement.

Since 2008, Gerard de Haan focuses on video health monitoring, which has lead to his appointment in 2016 as chair of the “Video Health Monitoring” group at Eindhoven University. The new focus area has resulted in about 35 papers and 40 patent applications, so far, and a new course on Video Health Monitoring given since 2016.

He received 5 Best Paper Awards, the Gilles Holst Award, the Chester Sall Award, bronze, silver and gold patent medal awards, while his work on motion received the “1995 European Video Innovation Award of the Year” from the European Imaging and Sound Association (EISA) and a "Business Innovation Award" from the Wall Street Journal Europe. Gerard de Haan has served in the program committees of various international conferences on image/video processing and analysis, and has been a Guest-Editor for special issues of Elsevier, IEEE, and Springer.

Supported by