WO2017025775A1

WO2017025775A1 - Device for adaptive photoplethysmography imaging

Info

Publication number: WO2017025775A1
Application number: PCT/IB2015/056097
Authority: WO
Inventors: Uldis Rubins; Janis ZAHARANS; Janis Spigulis
Original assignee: Latvijas Universitate
Priority date: 2015-08-11
Filing date: 2015-08-11
Publication date: 2017-02-16

Abstract

The present invention proposes a device for detection of hemodynamic parameter maps in a human skin by means of remote photoplethysmography imaging with adaptive illumination. The device comprises a video registration unit adapted for registering set of video frames of region of interest of subject's skin, said region having a number of sub-regions; a data processing unit adapted for processing obtained video signals and extracting photoplethysmography signals and set of hemodynamic parameters from each of the sub- regions; a display unit adapted to display hemodynamic parameter maps; an illuminator adapted to provide uniform illumination of the region of interest; a driver for regulating a power of illuminator; wherein the device further comprises a feedback unit adapted to receive from the data processing unit, the PPG signals from each of the sub-regions and to provide control of the illuminator to maintain uniform illumination of the region of interest; triggering unit for synchronous triggering of illuminator and video registration unit.

Description

DEVICE FOR ADAPTIVE PHOTOPLETHYSMOGRAPHY IMAGING

Field of the Invention

The present invention relates to a device for photoplethysmography imaging, which can be used for adaptive remote photoplethysmography and mapping of hemodinamic parameters in human skin.

Background Art

Recently vital sign monitoring using remote photoplethysmography (PPG) has been demonstrated and found relevant for patient monitoring. Remote PPG utilizes radiation sources (visible or near infrared) and video camera (detector), disposed remotely from the subject skin. The technique of PPG imaging is described, for instance, in W. Verkruysse et al., "Remote plethysmography imaging using ambient light", Optics Express 16(26), 2008. It is based on the principle that pulsed temporal variations of blood volume lead to variations in light absorptions by the skin, and, consequently, in intensity of the reflected radiation. Small blood volume pulsations can be registered by video camera that takes video images of a fixed skin area. Video processing allows calculating the averaged pixel values over selected region of interest (Rol) and extracting the PPG signal. There is a variety of hemodynamic parameters (e.g. pulse rate, breathing rate, PPG amplitude and phase, waveform parameters) which could be derived from the PPG signal. In the case when the skin surface is divided in multiple small Rols, the set of signals are derived and the set of hemodynamic parameters are calculated from signals. This technique is called PPG imaging where detailed spatial information is derived simultaneously from multiple sites of skin, which allows mapping of physiological parameters such as blood perfusion or blood oxygen saturation. The PPG imaging could be usable in clinical monitoring in the cases where contact measurements are not possible (wounds, sterile surfaces, fracture sites, hyperalgesia areas, etc.).

WO/2009/030934 discloses the method of blood oxygen saturation mapping over the skin by using remote PPG technique. By providing illumination over a range of excitation frequencies and using a multi time gating technique with regard to returned light, images are created which provide a penetration depth due to the varying scatter and absorption effects at the different excitation wavelengths. The sequence of images can be consolidated in order to provide a substantially real time view of PPG signals dependent upon blood oxygen saturation within the blood circulation system.

In WO/2013/156908 the device and method for obtaining vital sign information of a living being are described. The invention relates to a processing method for obtaining vital sign information of a living being from an image signal generated from spatio-temporal variations of received light. The device comprises a control unit analyzing derived vital sign information and generating control information for controlling the setting of one or more parameters of registration unit to optimize the quality of the vital sign information derived from obtained image signals.

In WO/2015/055405 the device and method for obtaining vital signs of a subject are described. The invention proposes to determine an optimal binning configuration of sensor based on a determined feature of the extracted PPG signal and a specific vital sign to be extracted from it. The binning configuration is chosen automatically and allows reliably and accurately obtaining a vital sign of a subject under varying lighting conditions ranging, potentially from full sun light to bedroom light levels at night.

WO/2014/128273 discloses the device of automotive vehicle occupant monitoring using near-infrared light. The device inside a car looks at occupants (driver, front passenger, rear passengers) and covers multiple security, comfort, driver assistance and occupant state related functions, as well as measures occupants' vital sign parameters (heart rate, respiration rate, blood oxygen saturation) using contactless imaging photoplethysmography.

There is a number of patent documents describing technology of adaptive lighting which is applicable in photo/video registration and machine vision (e.g. US 6207946, US 2015/0172524, US 2013/0258688). Summary of the Invention

Aim of the present invention is to provide a device and method for reliable and accurate obtaining of hemodynamic parameter maps characterized by higher quality in terms of increased dynamic range and better signal-to-noise ratio compared to the known devices and methods.

A device for obtaining hemodynamic parameter maps is proposed. The device comprising: a video registration unit adapted for registering set of video frames of region of interest of subject's skin, said region having a number of sub-regions; a data processing unit adapted for processing obtained video signals and extracting photoplethysmography signals and set of hemodynamic parameters from each of the sub-regions; a display unit adapted to display hemodynamic parameter maps; an illuminator adapted to provide uniform illumination of the region of interest, which is controlled by a driver for regulating a power of illuminator, wherein the device further comprises a triggering unit adapted to provide synchronous triggering of the illuminator and the video registration unit; a feedback unit adapted to receive from the data processing unit the PPG signals from each of the sub-regions and to provide control of the illuminator to maintain uniform illumination of the region of interest.

The feedback unit is (electrically) connected with the data processing unit and the driver of the illuminator. The data processing unit is connected with the video registration unit, the feedback unit and the display unit. The triggering unit is connected with the driver and the registration unit.

According to one embodiment the feedback unit comprises a low -pass filter adapted to filter PPG signals to remove the fast varying component and a proportional-integral-derivative controller adapted to minimize the amplitude of slow PPG component by adjusting the intensity of light source.

To provide good PPG signal, the skin should be illuminated spatially uniformly. The image pixel values should be in range where the image sensor has linear sensitivity. To reach high signal to noise ratio of PPG signal the pixels should generate possibly high values while preserving linearity of sensor. One possibility to reach this goal is to provide spatially uniform illumination of skin area. Several light sources placed in a proper distance from each other and skin surface provide spatially distributed illumination. Based on image analysis, intensity of every light source could be adjusted to reach light saturation almost in all pixels of image. This adjustment is needed when the skin surface has uneven structure or in cases of heterogeneous blood perfusion in the skin tissue.

To detect small PPG signal variations extracted from every pixel of the image, the analog to digital convertor (ADC) of video system should have high bit range or high enough quantization resolution. The disadvantage of this approach is high complexity and costs. On the other hand, the quality of PPG signal could be improved if the image pixel values remain saturated during large variations of reflected light, in cases of rapid hemodynamic changes in skin tissue. This is important when continuous monitoring of vital signs in clinical environment is needed. The invention proposes solution: to provide feedback between the measured PPG signal and intensity emitted from each of the light sources which is adjusted to reach optimal spatial and temporal distribution of illumination over the specified region of interest of the skin.

In the cases when the subject skin surface is not in steady position, the movements negatively affect the PPG signal. Translation movement artifacts could be partially avoided by using anti-shaking algorithms (e.g. algorithm described in patent application WO/2015/044518). Nevertheless, these algorithms are not able to fully remove artifacts in PPG signal in cases when the skin moves but an illuminator and a camera are in a steady position, because the movements change the uniformity of illumination of skin. The problem can be solved if the feedback system adjusts illuminator to provide spatially-uniform illumination of skin before the video stabilizing algorithm is applied. Brief Description of the Drawings

Fig. 1 is a schematic illustration of a device for obtaining hemodynamic parameter maps according to the present invention.

Fig. 2 shows illuminator comprising a set of light sources illuminating region of interest of skin.

Fig. 3 shows the image of region of interest which is illuminated by several light sources and a PPG signal obtained from binned pixels in the illuminated sub-region.

Fig. 4 shows detailed block-diagram of working principle of the device.

Detailed Description Of The Invention

Fig. 1 shows an embodiment of a device 1 for obtaining hemodynamic parameter maps according to the present invention. The living being e.g. human (palm) skin 2 can be illuminated by a light source - illuminator 10. A video registration unit (video camera) 5 is adapted to capture image frames from the region of interest 3 of skin, which is illuminated by the illuminator 10. The video registration unit 5 is adapted to forward recorded images to a data processing unit 7 which is adapted to convert video data to PPG signals 14. The data processing unit performs calculation of hemodynamic parameters (e.g. heart rate, breathing rate, the amplitude and phase of PPG pulse, waveform parameters etc.) from the PPG signals and forwards the parameter set to output device 11 , such as touch sensitive monitor screen. The hemodynamic parameters can be visualized on the monitor as a 2-D parametric map.

The device 1 further comprises a driver 16, which is adapted to regulate power of the illuminator 10. The driver 16 is adapted to be controlled by a feedback unit 9 (Fig. 1) to maintain uniform illumination of the region of interest 3.

With reference to Fig. 2 the illuminator 10 preferably comprises an array of light emitting diodes (LED) or similar light emitters 12. This illuminator 10 is adapted to be placed at the preferred distance from the skin surface. The light emitters 12 can be arranged in regular array or may be located irregularly. The illuminator system 10 is adapted to perform spatially uniform illumination of skin. It can consist of several light emitters 12 with wide angle of illumination so that the light beams from them cover each other, but sum of all beams produce light field of uniform illumination. In another embodiment of the illuminator 10 large number of light elements 12 (e.g. LEDs or laser diodes) are arranged in planar matrix, each of them provides narrow angle beam illumination so that the skin area is illuminated with non- overlapping light spots 13. According to another embodiment the illuminator 10 comprises a curved screen adapted to provide spatially uniform illumination of the region of interest 3. According to yet another embodiment the illuminator 10 comprises a set of light emitters 12 with multiple excitation frequencies, that are switched on and off in a sequential order synchronously with the video registration unit 5 which is configured for trigger mode. The illuminator 10 can be adapted to emit light pulses, as well as to emit light at several wavelength intervals in the range between 450 and 1300 nm. The light sources 12 emit the radiation in at least one wavelength interval, preferably in the green or near infrared region. According to the preferred embodiment the driver 16 is adapted to regulate power of each light source 12.

The illuminator 10 and the registration unit 5 are controlled by triggering unit 15 which is configured to provide synchronous switching of the illuminator 10 with capturing of current video frame.

The image frames of a specified skin area - the region of interest 3 are captured by the video registration unit 5 comprising suitable image sensor or several arrays of image sensors. The video registration unit 5 comprises a charge-coupled device (CCD), complementary metal-oxide-semiconductor (CMOS) sensor, hybrid or similar sensor-matrix which is able to transform the recorded light field into a set of pixel values. The device is configured to transfer video frames to the data processing unit 7. Fig. 3 shows the image frame from video in the case when the skin area is illuminated by N semi-parallel beams from N light emitters. First the proper binning of image pixels in each of the illuminated sub-areas 13 is performed. Then N PPG signals are derived from each sub-area from the video frames. Finally PPG signals are filtered by high-pass filter and N hemodynamic parameter sets are derived and sent to display unit 1 1 for displaying parameter maps. To maintain uniformly distributed saturation of image pixels, the feedback unit 9 is placed between the data processing unit 7 and the driver 16; it comprises a low-pass filter and a proportional-integral-derivative (PID) controller (Fig.4). The feedback unit 9 is adapted to receive PPG signals from each of the sub-regions 13 that contain both fast varying component (mostly produced by blood volume changes in blood vessels by heart activity and breathing) and slow varying component (produced by slowly varying hemodynamic changes). Each of the PPG signals is filtered using a low pass filter to remove the fast varying component. Remaining slow varying PPG signal is used to adjust the intensity of corresponding light source 12 aiming to keep the average pixel values from corresponding sub-region 13 close to video sensor saturation level and to achieve uniform illumination distribution in the image. Fast and precise feedback to light intensity control can be achieved using PID controller, which is configured to minimize the amplitude of the slow PPG component obtained from the corresponding sub-region 13. The frame rate of video is considered to be fast enough for extraction of fast periodic component of PPG and the hemodynamic parameters (e.g. heart rate, breathing rate, the amplitude and phase of PPG pulse, waveform parameters etc.).

The presented device and method allows to track the video signal and to stabilize the pixel values of individual video frames by adjusting the illuminating light intensities spatially and temporally.

Claims

1. A device (1) for obtaining hemodynamic parameter maps, comprising a video registration unit (5) adapted for registering set of video frames of region of interest (3) of subject's skin, said region having a number of sub-regions (13); a data processing unit (7) adapted for processing obtained video signals and extracting PPG signals and set of hemodynamic parameters from each of the sub-regions (13); a display unit (11) adapted to display hemodynamic parameter maps; an illuminator (10) adapted to provide illumination of the region of interest (3); wherein the device (1) further comprises a driver (16) adapted for regulating power of the illuminator (10); a triggering unit (15) adapted to provide synchronous triggering of the illuminator (10) and the video registration unit (5); a feedback unit (9) adapted to receive from the data processing unit (7) the PPG signals from each of the sub-regions (13) and to provide control of the illuminator (10) to maintain uniform illumination of the region of interest (3).

2. The device (1) as claimed in claim 1, wherein the illuminator (10) comprises a set of light emitters (12) adapted to provide spatially uniform illumination of the region of interest (3), wherein the driver (16) is adapted to regulate power of each light emitter (12).

3. The device (1) as claimed in claim 2, wherein the light emitters (12) are arranged in planar matrix, each of them having narrow angle of light emission so that the skin illumination spots do not overlap.

4. The device (1) as claimed in claim 1, wherein the illuminator (10) is adapted to emit light at wavelength interval in the range between 450 and 1300 nm.

5. The device (1) as claimed in claim 1, wherein the illuminator (10) comprises a set of light emitters (12) with multiple excitation frequencies, that are switched on and off in a sequential order synchronously with the video registration unit (5) which is configured for trigger mode.

6. The device (1) as claimed in claim 1, wherein the illuminator (10) comprises light emitting diodes (12) adapted to emit light at least in one wavelength interval.

7. The device (1) as claimed in claim 1, wherein the illuminator (10) comprises a set of lasers (12) arranged in planar matrix so to emit parallel light beams.

8. The device (1) as claimed in claim 1, wherein the illuminator (10) comprises a curved screen adapted to provide spatially uniform illumination of the region of interest (3).

9. The device (1) as claimed in claim 1, wherein the illuminator (10) is adapted to emit light pulses.

10. The device (1) as claimed in claim 1, wherein the video registration unit (5) comprises several arrays of image sensors.

11. The device (1) as claimed in claim 1, wherein the feedback unit (9) comprises a low-pass filter adapted to filter PPG signals to remove the fast varying component and a proportional- integral-derivative controller adapted to minimize the amplitude of slow PPG component by adjusting the intensity of light source.