JP2017202241A - Endoscope system, processor device, and signal processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an endoscope system and signal processing method capable of easily acquiring signals required to generate an image for observing a structure of a subject, and also to provide a processor device capable of easily generating an image for observing a structure of the subject, based on the signals acquired by the endoscope system.SOLUTION: An endoscope system according to an aspect of the present invention, acquires first and second luminance signals by first and second illumination light beams for image acquisition during first and second light emission controls, and also acquires first and second reference signals by emitting a third illumination light beam as reference light. The first and second reference signals can be used for correction of the first and second luminance signals during generating of an image for observation of a structure of a subject, using the first and second luminance signals. This results in easy acquisition of signals required to generate an image for observation of the structure of the subject.SELECTED DRAWING: Figure 12

Description

  The present invention relates to an endoscope system, a processor device of the endoscope system, and a signal processing method in the endoscope system, and in particular, an endoscope system that processes a signal for generating an image in which a structure of a subject is emphasized, The present invention relates to a processor device and a signal processing method.

  In examinations or diagnoses using an endoscope system, the importance of information on blood vessels has been recognized, and in recent years, endoscope systems that extract blood vessel information by various methods have been proposed (for example, the following patents). Reference 1 and Patent Reference 2). Patent Document 1 discloses that blood vessels present on the mucous membrane surface layer of a living tissue by weighting each of an image obtained using blue narrow-band light and an image obtained using green narrow-band light. And an endoscope system for extracting blood vessels existing in a deep layer of a living tissue.

  Further, in Patent Document 2, a subject tissue is irradiated with a plurality of narrowband lights having different wavelength regions, and based on a luminance ratio between narrowband image data acquired via an imaging device for each irradiation of each narrowband light. An endoscope system for calculating blood vessel depth and oxygen saturation concentration is disclosed.

Japanese Patent No. 5393525 JP 2011-218135 A

  As shown in Patent Literature 1 and Patent Literature 2, blood vessels existing at different depths from images for each of a plurality of wavelength bands acquired by multiframe imaging using a plurality of types of illumination light having different wavelength bands. Can be extracted. Then, by combining the images for each of the plurality of wavelength bands using the difference information of the images, it is possible to obtain an image in which a blood vessel having a specific depth can be easily observed.

  When synthesizing such images, it is conceivable to adjust the brightness (luminance) of each image to make it easier to observe a blood vessel having a specific depth in the synthesized image. However, in the case of multi-frame shooting, since the shooting timing for each frame is different, the shooting distance and angle vary from frame to frame, and the difference in image brightness differs from place to place. In such a situation, it is difficult to match the brightness of each image with the conventional techniques as described in Patent Document 1 and Patent Document 2 described above, and as a result, the structure of the subject is sufficiently observed. Was difficult.

  The present invention has been made in view of such circumstances, and an object thereof is to provide an endoscope system and a signal processing method capable of easily acquiring signals necessary for image generation for observing the structure of a subject. To do. It is another object of the present invention to provide a processor device that can easily generate an image for observing the structure of a subject using a signal acquired by an endoscope system.

  In order to achieve the above-described object, an endoscope system according to a first aspect of the present invention includes a first pixel, a second pixel, and a second pixel in which an objective lens and color filters corresponding to the three primary colors are arranged. An image pickup unit including an image pickup device having three pixels, and a first illumination light, a second illumination light, and a third light having a first wavelength, a second wavelength, and a third wavelength having different wavelengths, respectively. A first light emission control for causing the light source to emit the first illumination light and the third illumination light simultaneously, and the second illumination light and the second illumination light. A light source control unit that performs second light emission control for simultaneously emitting three illumination lights, and when the first light emission control by the light source control unit is performed, the first pixel, the second pixel, and Among the third pixels, the pixel having the highest spectral sensitivity at the first wavelength is used for the first brightness. A pixel that obtains a signal and has sensitivity to the third wavelength among the first pixel, the second pixel, and the third pixel of the image sensor, and has the highest spectral sensitivity at least to the first wavelength The first reference signal is acquired from a pixel different from the pixel, and when the second light emission control is performed by the light source control unit, the first pixel, the second pixel, and the third pixel of the image sensor The second luminance signal is acquired from the pixel having the highest spectral sensitivity at the second wavelength, and the sensitivity is applied to the third wavelength among the first pixel, the second pixel, and the third pixel of the image sensor. An image signal acquisition unit that acquires a second reference signal from a pixel that is different from the pixel having the highest spectral sensitivity at the second wavelength.

  According to the first aspect, in the first and second light emission control, in addition to acquiring the first and second luminance signals by the first and second illumination lights for image acquisition, the reference light As a result, the third illumination light is emitted to obtain the first and second reference signals. These first and second reference signals can be used to correct the first and second luminance signals in image generation for observing the structure of the subject using the first and second luminance signals. As described above, according to the endoscope system according to the first aspect, it is possible to easily acquire a signal necessary for image generation for observing the structure of the subject.

  In the first aspect, the “pixel having sensitivity to the third wavelength” may be a plurality of pixels among the first pixel, the second pixel, and the third pixel. Even if the sensitivity to the third wavelength in any one of the first pixel, the second pixel, and the third pixel is not zero, the sensitivity is the first pixel of the reference signal, A pixel is considered not a “pixel sensitive to the third wavelength” when it is sufficiently low in sensitivity to the third wavelength in the other pixels of the second pixel and the third pixel. be able to.

  In the first embodiment, the first and second light emission controls may be performed not only once but also two times or more (3 times or more as the total number of the first and second light emission controls). In this case, the wavelength of illumination light (first illumination light, second illumination light) in each light emission control may be changed. For example, the first light emission control can be performed twice to change the wavelength of the first illumination light in the first light emission control and the wavelength of the first illumination light in the second light emission control. By performing such light emission control, it is possible to acquire three types of luminance signals together with the second light emission control, thereby generating different images depending on the observation purpose.

  In an endoscope system according to a second aspect, in the first aspect, at least one of the first luminance signal and the second luminance signal based on the first reference signal and the second reference signal. A luminance correction unit that corrects the image for each pixel or each imaging region, and an image signal that generates an image signal that emphasizes the structure of the subject based on the first luminance signal and the second image signal that have been corrected by the luminance correction unit And a generation unit. According to the second aspect, at least one of the first and second luminance signals is corrected for each pixel or each imaging region based on the first and second reference signals, and based on the corrected luminance signal Since the image signal in which the structure of the subject is emphasized is generated, an image from which local luminance unevenness (difference in brightness) is removed can be generated, and the structure of the subject can be easily observed.

  In the endoscope system according to the third aspect, in the second aspect, the light emission amount of the third illumination light is smaller than the light emission amounts of the first illumination light and the second illumination light. In the third aspect, since the third illumination light is used as the reference light in the correction of the luminance signal, it is defined that the amount of emitted light is smaller than that of the first illumination light and the second illumination light. In the third aspect, the light emission amount of the third illumination light is preferably 40% or less of the light emission amount of the first illumination light and the second illumination light. More preferably, the amount of emitted light of the third illumination light is 10% or more and 20% or less of the amount of emitted light of the first illumination light and the second illumination light.

  The endoscope system according to a fourth aspect is the second or third aspect, wherein the first reference signal and the second reference signal are the first pixel, the second pixel, and the third pixel, respectively. This is an image signal obtained by adding the image signals of the respective colors of the reference signal acquisition pixels, which are pixels different from the pixels from which the first image signal and the second image signal are acquired. In the fourth aspect, the image signal of each color of the reference signal acquisition pixel is added in consideration of the case where any of the reference signal acquisition pixels has sensitivity (subsensitivity) to the third illumination light as the reference light. The image signal is a first reference signal and a second reference signal. As described above, the first and second luminance signals are acquired from the pixels having the highest spectral sensitivity at the first and second wavelengths, respectively, so that the reference signal is an image signal as in the fourth mode. Thus, it is possible to effectively correct the luminance signal based on the reference signal. In the fourth aspect, the reference signal acquisition pixel may include a plurality of pixels among the first pixel, the second pixel, and the third pixel.

  The “sub-sensitivity” means the sensitivity of the light receiving sensor of each pixel with respect to light of a color other than that of the color filter provided in each pixel of the image sensor. For example, if the pixel is a red pixel (the color filter is red) For example, the sensitivity to blue light and the sensitivity to green light. The sensitivity of the color filter to light is “main sensitivity”.

  In an endoscope system according to a fifth aspect, in the fourth aspect, the third illumination light has a wavelength that maximizes the sum of the spectral sensitivities of the reference signal acquisition pixels. By setting the wavelength of the third illumination light (reference light) as in the fifth aspect, the luminance signal can be corrected more effectively even when the amount of the third illumination light is reduced. In the fifth aspect, “the wavelength at which the sum of the spectral sensitivities is maximum” means not only the wavelength at which the sum of the spectral sensitivities is numerically the maximum value, but also the sum of the spectral sensitivities is substantially effective in correcting the luminance unevenness. It also includes wavelengths that deviate from the maximum numerical value within a range where there is no significant influence.

  The endoscope system which concerns on a 6th aspect is set with respect to each setting light quantity of 1st illumination light and 2nd illumination light, and 1st illumination light and 2nd illumination light in the 4th or 5th aspect. A reference signal correction unit that corrects the first reference signal and the second reference signal based on the spectral sensitivity of the reference signal acquisition pixel is further provided, and the brightness correction unit is corrected by the reference signal correction unit. Based on the reference signal and the second reference signal, at least one of the first luminance signal and the second luminance signal is corrected for each pixel or each imaging region. Even if the reference light (third illumination light) is emitted with the same light amount in the first and second light emission control, the reference signals (first and second image signals) due to the subsensitivity of the light receiving sensor of each pixel ( Since the values of the first and second reference signals are different, in the sixth aspect, the first and second references are based on the set light amounts of the first and second illumination lights and the spectral sensitivities of the reference signal acquisition pixels. The signal is corrected, and the luminance signal is corrected based on the corrected reference signal.

  An endoscope system according to a seventh aspect is the endoscope system according to any one of the second to sixth aspects, wherein an image indicated by the first luminance signal acquired by the image signal acquisition unit and an image indicated by the second luminance signal An alignment unit that performs the alignment is further provided. According to the seventh aspect, luminance unevenness can be corrected with high accuracy by aligning images.

  The endoscope system according to an eighth aspect is the correction of at least one of the first luminance signal and the second luminance signal by the luminance correction unit according to any one of the second to seventh aspects. Before, it further includes an average luminance correction unit that performs correction to match the average luminance of the first luminance signal and the second luminance signal. In the eighth aspect, local correction can be performed more effectively by adjusting the average luminance before correcting the luminance signal locally (for each pixel or each imaging region).

  In the endoscope system according to the ninth aspect, in any one of the second to eighth aspects, the luminance correction unit outputs one luminance signal of the first luminance signal and the second luminance signal, Multiply the ratio between the first reference signal and the second reference signal. In the ninth aspect, the correction contents in the luminance correction unit are specifically defined.

  An endoscope system according to a tenth aspect according to any one of the second to ninth aspects further includes a frequency processing unit that reduces high-frequency components of the first reference signal and the second reference signal, and brightness The correction unit converts at least one of the first luminance signal and the second luminance signal based on the first reference signal and the second reference signal processed by the frequency processing unit for each pixel or imaging region. Correct every time. In the tenth aspect, since the reference signal includes a signal due to illumination light due to the subsensitivity of the light receiving sensor of each pixel, it is specified that processing for reducing a high-frequency component (signal component due to illumination light) of the reference signal is performed. Is.

  The endoscope system according to an eleventh aspect is the tenth aspect, in which the frequency processing unit reduces high-frequency components of the first reference signal and the second reference signal based on the observation distance from the imaging unit to the subject. Let The eleventh aspect prescribes a specific aspect of reducing the high-frequency component of the reference signal. Considering that the influence of the sub-sensitivity of the light receiving sensor is determined according to the observation distance (distance from the imaging unit to the subject). ing.

  An endoscope system according to a twelfth aspect includes, in the eleventh aspect, a storage unit that holds target frequency characteristics targeted for each observation distance from the imaging unit to the subject, and the frequency processing unit is based on the imaging unit. Based on the observation distance at the time of imaging, the corresponding target frequency characteristic is acquired from the storage unit, and the frequency characteristics of the first reference signal and the second reference signal are adjusted to the acquired target frequency characteristic. The twelfth aspect defines a more specific aspect of reducing the high-frequency component of the reference signal, and adjusts the frequency characteristic of the reference signal to a target frequency characteristic corresponding to the observation distance.

  The endoscope system according to a thirteenth aspect is the eleventh or twelfth aspect, in which the frequency processing unit has a higher frequency from the first reference signal and the second reference signal as the observation distance during imaging by the imaging unit is closer. Strengthen processing to reduce ingredients. The thirteenth mode prescribes a more specific mode of reducing the high-frequency component of the reference signal. Since the influence of the sub-sensitivity of the light receiving sensor is larger as the observation distance is closer, the reduction processing is strengthened as the distance is shorter. ing.

  In the endoscope system according to a fourteenth aspect, in any one of the second to thirteenth aspects, the image signal generation unit includes a first luminance signal and a second luminance signal corrected by the luminance correction unit. A difference calculation unit that calculates a difference signal of the color difference signal, a color difference signal generation unit that generates a color difference signal from the calculated difference signal, and one of the first luminance signal and the second luminance signal, and the generated color difference signal Or a color image generation unit for generating a color image signal obtained by converting one of the first luminance signal and the second luminance signal and the generated color difference signal into each of the three primary color image signals. And having. The fourteenth aspect specifically defines the generation of a color image in the image signal generation unit.

  An endoscope system according to a fifteenth aspect is the endoscope system according to any one of the second to fourteenth aspects, wherein the structure of the subject is a blood vessel, and the image signal generation unit generates an image signal in which the blood vessel is emphasized by color To do. According to the fifteenth aspect, blood vessels as the structure of the subject can be easily observed visually.

  A processor device according to a sixteenth aspect of the present invention is a processor device for an endoscope system according to any one of the second to fifteenth aspects, and includes a light source, a light source control unit, and an image signal acquisition unit. And a luminance correction unit and an image signal generation unit. The sixteenth aspect defines a processor device that processes an image signal acquired by an image pickup apparatus (an image pickup unit of an endoscope system, an endoscope apparatus, etc.). As in the second to fifteenth aspects, By generating an image signal that emphasizes the structure of the specimen, the structure of the specimen can be easily observed.

  In order to achieve the above-described object, a signal processing method according to a seventeenth aspect of the present invention includes a first pixel, a second pixel, and a third pixel in which an objective lens and color filters corresponding to the three primary colors are arranged. When the subject is imaged by the imaging unit including the imaging element having the first and second pixels, the first illumination light and the second illumination have a first wavelength, a second wavelength, and a third wavelength having different wavelengths, respectively. First light emission control for simultaneously emitting first illumination light and third illumination light from a light source that emits light and third illumination light, and second illumination light and third illumination light simultaneously. The step of sequentially performing the second light emission control to emit light, and the first wavelength of the first pixel, the second pixel, and the third pixel of the image sensor when the first light emission control is performed. The first luminance signal is acquired from the pixel having the highest spectral sensitivity, and the image sensor The first reference from a pixel that is sensitive to the third wavelength among the one pixel, the second pixel, and the third pixel and that is different from the pixel having the highest spectral sensitivity at least at the first wavelength When the signal is acquired and the second light emission control is performed, the first pixel, the second pixel, and the third pixel of the image sensor are changed from the pixel having the highest spectral sensitivity to the second wavelength. 2 having a luminance signal of 2 and having sensitivity to the third wavelength among the first pixel, the second pixel, and the third pixel of the image sensor, and most spectrally at least at the second wavelength. Obtaining a second reference signal from a pixel different from the highly sensitive pixel. According to the seventeenth aspect, a signal necessary for image generation for observing the structure of the subject can be easily obtained as in the first aspect.

  A signal processing method according to an eighteenth aspect is the signal processing method according to the seventeenth aspect, wherein at least one of the first luminance signal and the second luminance signal is obtained based on the first reference signal and the second reference signal. The method further includes a step of correcting for each pixel or each imaging region, and a step of generating an image signal in which the structure of the subject is emphasized based on the corrected first luminance signal and second luminance signal. According to the eighteenth aspect, similarly to the second aspect, an image from which local luminance unevenness (difference in brightness) is removed can be generated, and the structure of the subject can be easily observed.

  In the signal processing methods according to the seventeenth and eighteenth aspects, configurations similar to the third to fifteenth aspects may be further included. In addition, a program that causes an endoscope system or a processor device to implement the signal processing method of these aspects, and a non-transitory recording medium that records a computer-readable code of such a program can also be cited as aspects of the present invention. it can. Examples of such non-temporary recording media include semiconductor recording media such as ROM (Read Only Memory) and EEPROM (Electronically Erasable and Programmable Read Only Memory), and magneto-optical recording such as DVD (Digital Versatile Disk) and hard disks. Although a medium can be mentioned, it is not limited to these.

  As described above, according to the endoscope system and the signal processing method according to the present invention, signals necessary for image generation for observing the structure of the subject can be easily acquired. Further, according to the processor device of the present invention, it is possible to easily generate an image for observing the structure of the subject based on a signal acquired by the endoscope system.

FIG. 1 is an external view showing an endoscope system according to the first embodiment. FIG. 2 is a block diagram showing a schematic configuration of the endoscope system. FIG. 3 is a graph showing an example of the spectrum of the light source. FIG. 4 is a graph showing the spectral characteristics of the color filter used in the image sensor. FIG. 5 is a graph showing the scattering coefficient of the observation target. FIG. 6 is a graph showing the absorption coefficient of hemoglobin. FIG. 7 is a block diagram illustrating functions of the reference light correction unit. FIG. 8 is a conceptual diagram showing the relationship between the observation distance and the strength of the low-pass filter. FIG. 9 is a block diagram illustrating functions of the special observation image processing unit. FIG. 10 is a graph schematically showing the relationship between the blood vessel depth and the blood vessel contrast. FIG. 11 is an explanatory diagram schematically showing an example of signal channel assignment when generating a blood vessel emphasized image of a specific depth. FIG. 12 is a flowchart showing a procedure of image processing in the special observation mode. FIG. 13 is a diagram showing the relationship between wavelength and spectral sensitivity. FIG. 14 is a diagram illustrating an example of a captured image captured using the first narrowband light having a center wavelength of 405 nm. FIG. 15 is an example of a captured image captured using the second narrowband light having a center wavelength of 445 nm. FIG. 16 is a diagram showing an example of a blood vessel enhancement image generated from the image shown in FIG. 14 and the image shown in FIG. FIG. 17 is a diagram illustrating an example of a captured image captured using the first narrowband light having a center wavelength of 405 nm. FIG. 18 is an example of a captured image captured using the second narrowband light having a center wavelength of 445 nm. 19 is a diagram showing an example of a blood vessel enhancement image generated from the image shown in FIG. 17 and the image shown in FIG.

  Hereinafter, embodiments of an endoscope system, a processor device, and a signal processing method according to the present invention will be described with reference to the accompanying drawings.

[First Embodiment]
FIG. 1 is an external view showing an endoscope system 10 (endoscope system) according to the first embodiment. FIG. 2 is a block diagram showing functions of the endoscope system 10. As shown in FIG. 1, an endoscope system 10 includes an endoscope 12, a light source device 14 (light source, light source control unit, processor device), a processor device 16 (processor device), a monitor 18, and a console 19. And have. The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16.

  The endoscope 12 includes an insertion portion 12a to be inserted into a subject, an operation portion 12b provided at a proximal end portion of the insertion portion 12a, a bending portion 12c and a distal end portion provided at the distal end side of the insertion portion 12a. 12d. By operating the angle knob 12e of the operation unit 12b, the bending unit 12c performs a bending operation. By this bending operation, the distal end portion 12d is directed in a desired direction.

  In addition to the angle knob 12e, the operation unit 12b is provided with a mode switch 13a and a zoom operation unit 13b. The operation unit 12b is provided with a still image acquisition instruction unit 13c (see FIG. 2).

  The mode switch 13a is used for the operation of switching the observation mode. The endoscope system 10 has a normal observation mode and a special observation mode as observation modes. The normal observation mode is a mode in which an image obtained by imaging an observation target using white light as illumination light is displayed on the monitor 18. An image obtained by imaging an observation target in the normal observation mode is referred to as a “normal observation image”. The normal observation mode can be rephrased as “white light observation mode”. The normal observation image can be rephrased as a “white light observation image”. The illumination light can be rephrased as “observation light”.

  The special observation mode generates a visualized image that emphasizes blood vessels in a specific depth region of the observation target using an image signal obtained by imaging the observation target using narrowband light of a specific wavelength band as illumination light. In this mode, an image suitable for blood vessel observation is displayed on the monitor 18. An image obtained in the special observation mode is referred to as a “special observation image”. The special observation mode can be rephrased as a “narrow band observation mode”. The special observation image can be rephrased as a “blood vessel enhancement image” or a “narrow band observation image”. The endoscope 12 of this example has a plurality of types of special observation modes in which the types of wavelength bands of narrowband light to be used or combinations thereof are different.

  The processor device 16 is electrically connected to the monitor 18 and the console 19. The monitor 18 is a display device that outputs and displays an image to be observed, information attached to the image to be observed, and the like. The console 19 functions as a user interface that receives input operations such as function settings of the endoscope system 10 and various instructions. An external storage device (not shown) may be connected to the processor device 16. The external storage device can record an image to be observed and information attached to the image. The storage 70 shown in FIG. 2 is an example of an external storage device, and functions as an external recording unit.

  As shown in FIG. 2, the light source device 14 includes a light source 20 (light source) and a light source control unit 22 (light source control unit) that controls the light source 20. The light source 20 includes, for example, a semiconductor light source such as a multi-color LED (Light Emitting Diode), a combination of a laser diode and a phosphor, a halogen light source such as a xenon lamp, or an appropriate combination thereof. The light source 20 includes an optical filter (not shown) for adjusting the wavelength band of light emitted from a light source such as an LED.

  The light source 20 of this embodiment includes a V-LED (Violet Light Emitting Diode) 23a, a B-LED (Blue Light Emitting Diode) 23b, a G-LED (Green Light Emitting Diode) 23c, and an R-LED (Red Light Emitting Diode). 23d four-color LED.

  FIG. 3 is a graph showing an example of the spectrum of the light source 20. The V-LED 23a is a violet semiconductor light source that emits violet light V in a wavelength band of 380 nm to 420 nm having a center wavelength of about 400 nm ± 10 nm. The B-LED 23b is a blue semiconductor light source that emits blue light B having a center wavelength of about 450 nm ± 10 nm and a wavelength band of 420 nm to 500 nm. The G-LED 23c is a green semiconductor light source that emits green light G having a center wavelength of about 550 nm ± 20 nm and a wavelength band ranging from 480 nm to 620 nm. The R-LED 23d is a red semiconductor light source that emits red light R having a center wavelength of about 610 nm ± 10 nm and a wavelength band ranging from 580 nm to 650 nm. The term center wavelength may be read as the peak wavelength at which the spectral intensity is maximized. In FIG. 3, the sum of the light intensities of the G-LED 23c and the R-LED 23d is indicated by a dotted line (reference symbol G + R).

  The light source control unit 22 controls the amount of illumination light by turning on (turning on) and turning off (off) a light source such as an LED and adjusting a drive current and a drive voltage of the LED. The light source control unit 22 controls the wavelength band of the illumination light by changing the optical filter or the like. The light source control unit 22 can individually control the lighting and extinction of each LED and the amount of emitted light at the time of lighting by individually inputting a control signal to each LED of the light source 20. The light source 20 generates a plurality of types of illumination light that irradiates the observation target under the control of the light source control unit 22.

  The light source 20 of this example includes a purple narrowband light having a center wavelength in the purple wavelength band (wavelength band of about 350 nm to 400 nm) and a blue narrowband light having a center wavelength in the blue wavelength band (wavelength band of about 400 nm to 500 nm). , Green narrowband light having a center wavelength in the green wavelength band (wavelength band of about 480 nm to 620 nm), and red narrowband light having a center wavelength in the red wavelength band (wavelength band of about 580 nm to 650 nm) Narrow band light can be generated.

  As a more specific example, the light source 20 is a narrow band light such as a purple narrowband light having a center wavelength of 405 nm, a blue narrowband light having a center wavelength of 445 nm, a green narrowband light having a center wavelength of 530 nm, and a red narrowband light having a center wavelength of 610 nm. Band light can be generated. The light source 20 can generate blue narrowband light having a center wavelength of 470 nm, and can also generate two or more types of blue narrowband light having different center wavelengths. Two or more types of narrowband light having different center wavelengths can be generated for each of the purple narrowband light, the green narrowband light, and the red narrowband light. The center wavelength and wavelength band of each narrow band light can be designated by selecting the LED to emit light and changing an optical filter (not shown).

  In the present disclosure, purple narrow band light having a central wavelength of 405 nm generated by the light source 20 may be referred to as “purple light V”. Also, blue narrowband light with a center wavelength of 445 nm is denoted as “blue light B”, green narrowband light with a center wavelength of 530 nm is denoted as “green light G”, and red narrowband light with a center wavelength of 610 nm is denoted as “red light R”. There is.

  When the special observation mode is selected, the light source 20 generates at least two types of narrowband light having different center wavelengths from among a plurality of types of narrowband light, and the observation target irradiated with each narrowband light is selected. Imaging is performed by the imaging sensor 48. Therefore, in the special observation mode, a plurality of types of endoscopic images corresponding to the types of narrowband light can be obtained. In the present embodiment, in the special observation mode, the light source 20 can alternately generate two types of narrowband light of first narrowband light and second narrowband light having different center wavelengths. Of these two types of narrowband light, the first narrowband light is a relatively narrow wavelength side narrowband light, and the second narrowband light is a relatively long wavelength side narrowband light. That is, the center wavelength of the second narrowband light is longer than the center wavelength of the first narrowband light. For example, the first narrowband light is violet narrowband light having a center wavelength of 405 nm, and the second narrowband light is blue narrowband light having a center wavelength of about 445 nm.

  In the present embodiment, two types of narrowband light, the first narrowband light and the second narrowband light, are illustrated in the order of shorter center wavelength, and a special observation image is generated by selectively switching these narrowband lights. Will be explained. In the implementation of the invention, the type of narrowband light is not limited to this example, and a mode in which many types of narrowband light are used is also possible.

  The light source 20 can generate white light. In the normal observation mode, the light source control unit 22 turns on all the V-LEDs 23a, B-LEDs 23b, G-LEDs 23c, and R-LEDs 23d. For this reason, in the normal observation mode, white light in a wide wavelength band including violet light V, blue light B, green light G, and red light R is used as illumination light. The light source device 14 corresponds to one form of a “light source unit”.

  Illumination light emitted from the light source 20 is incident on the light guide 41 through an optical path coupling unit formed by a mirror, a lens, or the like (not shown). The light guide 41 is built in the endoscope 12 and the universal cord. The universal cord is a cord that connects the endoscope 12 to the light source device 14 and the processor device 16. The light guide 41 is inserted into the insertion portion 12 a and propagates the illumination light generated by the light source 20 to the distal end portion 12 d of the endoscope 12.

  The distal end portion 12d of the endoscope 12 is provided with an illumination optical system 30a and an imaging optical system 30b (imaging unit). The illumination optical system 30 a has an illumination lens 45. The illumination light propagated by the light guide 41 is irradiated to the observation object through the illumination lens 45. The imaging optical system 30b includes an objective lens 46, a zoom lens 47, and an imaging sensor 48 (imaging device). Various types of light such as reflected light, scattered light, and fluorescence from the observation target due to the irradiation of the illumination light enter the image sensor 48 via the objective lens 46 and the zoom lens 47. As a result, an image to be observed is formed on the image sensor 48. The zoom lens 47 is freely moved between the telephoto end and the wide-angle end of the zoom range according to the operation of the zoom operation unit 13b, and enlarges or reduces the image of the observation target formed on the image sensor 48.

  The imaging sensor 48 is a color imaging sensor in which one of R (red), G (green), and B (blue) color filters (each of the three primary colors) is arranged for each pixel. The imaging sensor 48 images an observation target and outputs image signals of RGB color channels. As the image sensor 48, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor can be used. Further, instead of the image sensor 48 provided with the primary color filter, a complementary color image sensor having complementary color filters of C (cyan), M (magenta), Y (yellow), and G (green) may be used. . When the complementary color imaging sensor is used, four color image signals of CMYG are output. Therefore, RGB image signals similar to those of the image sensor 48 can be obtained by converting the CMYG four-color image signals into the RGB three-color image signals by complementary color-primary color conversion. In place of the image sensor 48, a monochrome sensor without a color filter may be used.

  FIG. 4 is a graph showing the spectral characteristics of the color filter used in the image sensor 48. The horizontal axis represents wavelength and the vertical axis represents transmittance. In FIG. 4, B-CF represents the spectral characteristics of the B color filter, G-CF represents the G color filter, and R-CF represents the spectral characteristics of the R color filter. Light in the purple to blue wavelength band is received by the B pixel (first pixel) provided with the B color filter in the image sensor 48. Light in the green wavelength band is received by the G pixel (second pixel) provided with the G color filter in the image sensor 48. Light in the red wavelength band is received by the R pixel (third pixel) provided with the R color filter in the image sensor 48. A signal corresponding to the amount of received light is output from the RGB color pixels of the image sensor 48.

  For example, in the special observation mode, when the first narrowband light that is a purple wavelength band is used as illumination light (first illumination light), the imaging sensor 48 observes the observation target irradiated with the first narrowband light. And a first image signal corresponding to the first narrowband light is output from the B pixel. In the special observation mode, when the second narrowband light that is the blue wavelength band is used as illumination light (second illumination light), the imaging sensor 48 uses the second narrowband light corresponding to the second narrowband light. An image signal is output from the B pixel.

  The endoscope 12 includes an AFE (Analog Front End) circuit 51 and an AD (Analog to Digital) converter 52. An image signal output from the image sensor 48 is input to the AFE circuit 51. The AFE circuit 51 includes a correlated double sampling (CDS) circuit and an automatic gain control (AGC) circuit. The AFE circuit 51 performs correlated double sampling and automatic gain control on the analog image signal obtained from the image sensor 48. The image signal that has passed through the AFE circuit 51 is converted into a digital image signal by the AD converter 52. The digital image signal after AD (Analog to Digital) conversion is input to the processor device 16. A configuration in which the AD converter 52 is mounted on the AFE circuit 51 is also possible.

  The processor device 16 includes an image signal acquisition unit 53 (image signal acquisition unit), a DSP (Digital Signal Processor) 56, a noise reduction unit 58, a memory 59, a signal processing unit 60, a video signal generation unit 68, It has.

  The image signal acquisition unit 53 acquires a digital image signal from the endoscope 12. The DSP 56 performs various signal processing such as defect correction processing, offset processing, gain correction processing, linear matrix processing, gamma conversion processing, and demosaicing processing on the image signal acquired via the image signal acquisition unit 53. In the defect correction process, the signal of the defective pixel of the image sensor 48 is corrected. In the offset process, the dark current component is removed from the image signal subjected to the defect correction process, and an accurate zero level is set. In the gain correction process, the signal level is adjusted by multiplying the image signal after the offset process by a specific gain.

  The image signal after gain correction processing is subjected to linear matrix processing for improving color reproducibility. After that, brightness and saturation are adjusted by gamma conversion processing. The demosaic process is performed on the image signal after the gamma conversion process, and a signal of a color that is insufficient at each pixel is generated by interpolation. The demosaic process is also called an isotropic process or a synchronization process. Through the demosaic processing, all pixels have signals of RGB colors.

  The noise reduction unit 58 reduces noise by performing noise reduction processing on the image signal that has been demosaiced by the DSP 56. As the noise reduction process, for example, a process using a moving average method, a median filter method, or the like can be employed. The image signal whose noise has been reduced by the noise reduction unit 58 is stored in the memory 59.

  The signal processing unit 60 acquires the image signal after noise reduction from the memory 59. The signal processing unit 60 performs signal processing such as color conversion processing, color enhancement processing, and structure enhancement processing on the acquired image signal as necessary, and obtains a color endoscope image in which an observation target is captured. Generate. The color conversion process is a process of performing color conversion on an image signal by 3 × 3 matrix processing, gradation conversion processing, three-dimensional lookup table processing, and the like. The color enhancement process is performed on the image signal that has been subjected to the color conversion process. The structure enhancement process is a process for enhancing a specific tissue or structure included in an observation target such as a blood vessel or a pit pattern, and is performed on the image signal after the color enhancement process. Further, the signal processing unit 60 calculates a distance (observation distance) from the imaging optical system 30b to the subject based on the acquired image signal and imaging conditions such as a zoom magnification.

  The contents of processing in the signal processing unit 60 differ depending on the observation mode. When the observation mode is the normal observation mode, the signal processing unit 60 generates a normal observation image by performing signal processing in which the observation target has a natural color. When the observation mode is the special observation mode, the signal processing unit 60 generates a special observation image by performing signal processing that emphasizes at least the blood vessel to be observed.

  The signal processing unit 60 includes an image processing switching unit 61, a normal observation image processing unit 66, a special observation image processing unit 67 (image signal generation unit), a registration processing unit 62 (positioning unit), and brightness correction. Including a processing unit 63 (average luminance correction unit), and performs signal processing corresponding to each of the normal observation mode and the special observation mode.

  The image processing switching unit 61 switches execution of normal observation image generation processing or special observation image generation processing according to the setting of the observation mode by the mode switch 13a. When the normal observation mode is set by operating the mode switch 13 a, the image processing switching unit 61 transmits the image signal received from the memory 59 to the normal observation image processing unit 66. When the special observation mode is set by operating the mode switch 13a, the image processing switching unit 61 receives the image signal received from the memory 59, the alignment processing unit 62, the brightness correction processing unit 63, and the reference light correction unit 64. , And the special observation image processing unit 67.

  The normal observation image processing unit 66 operates when the normal observation mode is set. The normal observation image processing unit 66 performs color conversion processing, color enhancement processing, and structure enhancement processing on the image signal captured by irradiating the observation target with white light, and generates a normal observation image signal. A color image using the normal observation image signal is a normal observation image.

  The special observation image processing unit 67 is an image processing unit that operates when the special observation mode is set. The special observation image processing unit 67 includes a first narrowband light and a first narrowband light, which are relatively short wavelengths, of two types of narrowband light (first and second narrowband lights) having different wavelength bands. A first image signal (first luminance signal and first reference signal) obtained by irradiating an observation target with third narrowband light (reference light) which is narrowband light on the longer wavelength side than the light; Obtained by irradiating the observation target with the second narrowband light having a relatively long wavelength side and the third narrowband light (reference light) being a narrowband light having a longer wavelength side than the second narrowband light. A special observation image in which a blood vessel having a specific depth is extracted using the second image signal (second luminance signal and second reference signal) and the extracted blood vessel is represented by a color difference with respect to other blood vessels. Is generated.

  Here, as the two types of narrowband light having different wavelength bands, a first narrowband light that is a purple narrowband light having a center wavelength of 405 nm and a second narrowband light that is a blue narrowband light having a center wavelength of 445 nm are used. A case will be described as an example. As reference light, a case where narrowband light having a center wavelength of 600 nm is used will be described as will be described later.

  The first image signal and the second image signal are input to the special observation image processing unit 67 via the alignment processing unit 62 and the brightness correction processing unit 63. The alignment processing unit 62 aligns the observation target represented by the first image signal and the observation target represented by the second image signal obtained by sequentially performing the irradiation of the narrowband light described above. By the alignment process of the alignment processing unit 62, the relative positions of the first image signal and the second image signal are associated with each other, and each of the first image signal and the second image signal is associated with each other. The same image range can be taken out from. The alignment processing unit 62 may perform image position correction for only one of the first image signal and the second image signal, or may perform image position correction for both image signals. Good. In this example, the second image signal is matched with the position of the first image signal with reference to the first image signal.

  The brightness correction processing unit 63 includes the first image signal and the second image signal so that the average luminance of the first image signal and the second image signal aligned by the alignment processing unit 62 matches. The luminance signal of at least one image signal is corrected.

  The reference light correction unit 64 (luminance correction unit, reference signal correction unit, frequency processing unit, storage unit) corrects and corrects the influence of the reduction of high frequency components included in the reference signal and the variation of the reference signal between frames. Based on the later reference signal, the luminance signal in the first image signal and the second image signal is corrected locally (for each pixel or for each imaging region).

  The special observation image processing unit 67 performs signal processing for emphasizing a blood vessel to be observed from the first luminance signal and the second luminance signal corrected by the reference light correction unit 64, and generates a special observation image. For example, in the special observation image generated by the special observation image processing unit 67, a blood vessel located at a relatively shallow position in the observation target with respect to the surface of the mucous membrane becomes a magenta color such as a brown color, for example. A blood vessel located at a relatively deep position in the observation target with respect to the surface of the light has a cyan color such as green. For this reason, the blood vessel to be observed is emphasized by the color difference with respect to the mucous membrane expressed in pink. A blood vessel located at a relatively shallow position with respect to the surface of the mucous membrane is called a “surface blood vessel”. Among superficial blood vessels, a blood vessel located in a very shallow position near the mucosal surface is called a “superficial blood vessel”. A blood vessel located at a relatively deep position with respect to the surface of the mucous membrane is called a “mid-deep blood vessel”.

  The signal processing unit 60 inputs the generated endoscopic image to the video signal generation unit 68. The video signal generator 68 converts the endoscopic image into a video signal for output display on the monitor 18. The endoscopic image generated by the signal processor 60 can be displayed on the monitor 18 via the video signal generator 68.

  When the still image acquisition instruction unit 13 c is operated and a release instruction is input, the signal processing unit 60 performs processing for saving the generated endoscopic image in the storage 70. The signal processing unit 60 is processed by the image signal read from the memory 59, the image signal processed by the alignment processing unit 62, the image signal processed by the brightness correction processing unit 63, and the reference light correction unit 64. Any one of the received image signals or an appropriate combination thereof can be stored in the storage 70.

  The storage 70 is an external storage device connected to the processor device 16. The storage 70 may be connected to the processor device 16 via a communication line such as a LAN (Local Area Network). The storage 70 is, for example, a file server of a system for filing endoscopic images such as PACS (Picture Archiving and Communication System), NAS (Network Attached Storage), or the like. The endoscopic image stored in the storage 70 can be used by the image processing device 72.

  The image processing device 72 is a device having a function of performing image processing on an endoscopic image and estimating a blood vessel depth. The image processing device 72 functions as a diagnosis support device that performs image processing on an endoscopic image and calculates blood vessel parameters for diagnosis support.

[Special Observation Image Generation Method]
First, a method for generating a special observation image in the endoscope system 10 will be described. In the special observation mode, the submucosal depth where the observable blood vessel exists is generally determined by the depth of illumination light irradiated when imaging the observation target. In general, light with a short wavelength has a low depth of penetration, and is scattered and absorbed near the mucosal surface, and a part of the light is observed as reflected light. The light absorption and scattering characteristics of the biological tissue to be observed are wavelength-dependent, and the longer the wavelength, the deeper the depth.

  FIG. 5 is a graph showing the scattering coefficient of the observation target. The horizontal axis in FIG. 5 represents the wavelength, and the vertical axis represents the normalized scattering coefficient. As shown in FIG. 5, the scattering coefficient increases as the wavelength becomes shorter. The greater the scattering, the more light that is reflected near the surface of the mucosa of the living tissue and the less light that reaches the mid-deep layer. Therefore, the depth of penetration is shallower as the wavelength is shorter, and the depth of penetration is deeper as the wavelength is longer. Although there are individual differences in the scattering coefficient, the tendency of wavelength dependence is common.

  The scattering coefficient of the observation target in each wavelength band of multiple types of narrowband light used in the special observation mode is the depth of each narrowband light, that is, the submucosal depth of the blood vessel that can be observed in that wavelength band. Related. However, as described above, it is difficult to grasp an accurate scattering coefficient from an endoscopic image for an actual observation target.

  On the other hand, the absorption coefficient of hemoglobin in the wavelength band of each narrowband light is related to the contrast of blood vessels that can be observed with each narrowband light.

  FIG. 6 is a graph showing the absorption coefficient of hemoglobin. The horizontal axis in FIG. 6 represents the wavelength, and the vertical axis represents the normalized absorption coefficient (absorption coefficient). As can be seen from FIG. 6, short-wavelength light has large hemoglobin absorption and large light scattering (see FIG. 5). For this reason, an image captured by irradiating narrow-band light with a short wavelength has a high contrast of blood vessels at shallow positions, but a contrast of blood vessels at deep positions is rapidly lowered. On the other hand, as the narrow-band light used for the illumination light has a longer wavelength, the contrast of the blood vessel at the shallow position is lowered, but the contrast of the blood vessel at the deep position is relatively moderate. By utilizing such characteristics, blood vessel information at an arbitrary depth can be visualized from difference information between two images picked up by changing the wavelength of illumination light.

  For example, it is obtained by performing imaging by irradiating illumination light of two types of wavelength bands, violet narrow band light having a center wavelength of 405 nm and blue narrow band light having a center wavelength of 445 nm, as illumination light on the short wavelength side. By assigning one of the images to the luminance signal and assigning the difference image between the two images to the color difference signal, blood vessels in the surface layer of the mucous membrane are extracted, and an image in which the extracted blood vessels are emphasized can be created.

  In addition, for example, when illumination light of two types of wavelength bands, that is, a green narrowband light having a center wavelength of 530 nm and a red narrowband light having a center wavelength of 610 nm, is used as illumination light on the long wavelength side, A blood vessel in a deep layer can be extracted, and an image in which the extracted blood vessel is emphasized can be created.

  The two types of illumination light used in the special observation mode are desirably light in a wavelength band in which the scattering coefficients of the observation target are different from each other and the absorption coefficient of hemoglobin is substantially equal. By using two types of illumination light satisfying such conditions, a blood vessel of a specific depth below the mucous membrane can be extracted particularly clearly.

  The conditions that “the scattering coefficients of observation objects are different from each other and the absorption coefficient of hemoglobin is almost equal” are different in the submucosal depth (depth of penetration) of the observable blood vessels, and the submucosal depth. This means that light of two wavelength bands that can be observed with blood vessels of different levels can be observed with substantially the same contrast is selected and used.

  As shown in FIG. 6, the first narrowband light having the center wavelength of 405 nm and the second narrowband light having the center wavelength of 445 nm used in this embodiment are shown in FIG. The coefficient = 3: 7) is approximately the same. A combination of the first narrowband light having a center wavelength of 405 nm and the second narrowband light having a center wavelength of 445 nm is an example of a preferable combination for blood vessel extraction.

  FIG. 7 is a block diagram illustrating functions of the reference light correction unit 64. The reference light correction unit 64 includes a storage unit 64a that holds a target frequency characteristic corresponding to the observation distance, and a frequency processing unit 64b that reduces a high-frequency component included in the reference signal according to the target frequency characteristic held in the storage unit 64a. A reference signal correction unit 64c that corrects variations in the reference signal value, and a luminance signal of the first image signal and the second image signal based on the corrected reference signal locally (for each pixel or And a luminance correction unit 64d for correction (for each imaging region). As will be described later, the reference light correction unit 64 holds the relationship between the observation distance and the target frequency characteristic (see FIG. 8).

  FIG. 9 is a block diagram illustrating functions of the special observation image processing unit 67. The special observation image processing unit 67 includes an arithmetic image signal generation unit 76 (difference calculation unit), a low pass filter (LPF) processing unit 77, and an image generation unit 78 (color difference signal generation unit, color image generation unit). With.

  The calculation image signal generation unit 76 performs calculation using the first image signal and the second image signal that have been subjected to luminance correction by the reference signal, and generates a calculation image signal. Specifically, the difference or ratio between the first luminance signal in the first image signal and the second luminance signal in the second image signal is calculated. The arithmetic image signal generation unit 76 of this example logarithmically converts each of the first luminance signal and the second luminance signal, and more specifically, the difference between the first luminance signal and the second luminance signal after logarithmic conversion, more specifically. Specifically, an arithmetic image signal ΔB that is a difference image (difference signal) obtained by subtracting the first luminance signal from the second luminance signal is generated. Logarithmic transformation is also referred to as “Log transformation”.

  In the first luminance signal and the second luminance signal, each pixel has a pixel value proportional to the amount of received light. However, when logarithmic conversion is performed, each pixel signal has a pixel value proportional to the density. Stable calculation results can be obtained regardless of the illumination intensity of the illumination light. In the present embodiment, it is assumed that there is no substantial difference in illuminance between the first narrowband light and the second narrowband light used as illumination light in the special observation mode, and the first luminance signal and the second luminance as described above. An arithmetic image signal ΔB is generated based on the difference from the signal.

  When the first luminance signal and the second luminance signal are used as they are without logarithmically converting the first luminance signal and the second luminance signal, the first luminance signal and the second luminance signal are used. The calculated image signal may be generated by calculating the ratio of the luminance signals for each pixel.

FIG. 10 is a graph schematically showing the relationship between the blood vessel depth and the blood vessel contrast. As shown in FIG. 10, when two types of violet light V and blue light B are used as illumination light, blood vessels in the entire range of the depth range _AS and the depth range _AD , that is, blood vessels that are generally in the surface layer ( The superficial blood vessels) can be observed. However, violet light V is shorter wavelength as compared to the blue light B, small penetration depth into the observation target, standing in the depth range A _S of shallow position of under relatively mucosa to blue light B instead of having Utsushidase only that vascular contrast of blood vessels located in the depth range a _S of shallow position is larger than the case of using the blue light B. “Contrast of blood vessel” means the ratio of the amount of light reflected from the surrounding mucous membrane to the amount of light reflected from the blood vessel. The contrast of the blood vessel can be calculated by using, for example, “YV / YM” or “(YV−YM) / (YV + YM)” using the blood vessel luminance YV and the mucosal luminance YM.

On the other hand, since the wavelength of the blue light B is longer than that of the purple light V, the blue light B has a deep depth to the object to be observed, and is present in the depth range _AD at a deep position below the mucosa relative to the purple light V. that instead of Utsushidaseru up vessel, the contrast of the blood vessels located in the depth range a _S of shallow position is smaller than the case of using a violet light V.

Therefore, the first is subtracted the luminance signal, electrode surface blood vessels of particular depth range A _S of shallow position of submucosal corresponding to violet light V from the second luminance signal corresponding to the blue light B The pixel value of the pixel to be expressed is emphasized and becomes a large value (white). Conversely, the pixel value of the pixel representing the blood vessel in the depth range _AD at a position deeper than the extreme surface blood vessel is a small value (black). Calculation of the calculated image signal ΔB corresponds to extracting a blood vessel at a specific depth below the mucous membrane.

  The low-pass filter processing unit 77 performs a low-resolution process by applying a low-pass filter to the calculation image signal ΔB generated by the calculation image signal generation unit 76. The strength of the filter processing that the low-pass filter processing unit 77 applies to the calculated image signal ΔB is determined by the cutoff frequency of the low-pass filter. The cut-off frequency of the low-pass filter is set in advance and lowers the sharpness at least than the sharpness of the original arithmetic image signal ΔB. The calculated image signal obtained by the low-pass filter processing of the low-pass filter processing unit 77 is an image that is more blurred than the original calculated image signal.

  The image generation unit 78 has a plurality of output channels using either the first luminance signal or the second luminance signal received by the special observation image processing unit 67 and the low-pass filtered arithmetic image signal ΔB. Generate an image. More specifically, the image generation unit 78 generates an image having a luminance channel Y and two color difference channels Cb and Cr related to color differences. The luminance channel Y corresponds to the first channel, and the two color difference channels Cb and Cr correspond to the second channel and the third channel, respectively. The image generation unit 78 assigns either the first luminance signal or the second luminance signal to the luminance channel Y, and assigns the low-pass filtered arithmetic image signal ΔB to the two color difference channels Cb and Cr. An image in which a blood vessel pattern having a specific depth is emphasized by color is generated. The YCC image (color image signal) generated in this way or the RGB image (color image signal) obtained by performing color conversion processing on this YCC image is referred to as “blood vessel emphasized image”. “YCC image” means a color image represented by a Y signal that is a luminance signal and a Cr signal and a Cb signal that are color difference signals.

  FIG. 11 is an explanatory diagram schematically showing an example of signal channel assignment when generating a blood vessel emphasized image of a specific depth. B1 in FIG. 11 represents the first luminance signal. In the case of the present embodiment, the first luminance signal corresponding to the narrow band light (purple light V) having a relatively short wavelength band is assigned to the luminance channel Y among the first luminance signal and the second luminance signal. . That is, the first luminance signal having a relatively high contrast of the extreme surface blood vessels is assigned to the luminance channel Y. The calculated image signal ΔB is assigned to the color difference channels Cb and Cr. When assigning the calculated image signal ΔB to the color difference channels Cb and Cr, they are multiplied by a coefficient α and a coefficient β, respectively. This is in order to align the color with the image displayed by the endoscope system that emphasizes and observes the surface blood vessels. The reason why the first luminance signal is assigned to the luminance channel Y is to select and emphasize the polar surface blood vessels from the surface blood vessels.

  In an endoscope system having an observation mode for emphasizing observation of a superficial blood vessel, one of the methods for generating a superficial blood vessel emphasized image is the following method using a B image signal and a G image signal of a captured image. That is, in the superficial blood vessel observation mode, the image of the observation object is obtained by irradiating the narrow band blue light to acquire the B image signal, and the observation object is imaged by irradiating the narrow band green light. Get the image signal. Then, the B image signal is assigned to the B channel and the G channel of the display image, and the G image signal is assigned to the R channel, so that the mid-deep blood vessels located deep in the submucosa are green (cyan) color. Then, the superficial blood vessels in the shallow position under the mucous membrane are highlighted in red (magenta) color.

  The ITU-R. In 601, the relationship between the RGB image signals, the luminance channel Y, and the color difference channels Cb and Cr is expressed by the following equations (1), (2), and (3). ITU is an abbreviation for “International Telecommunication Union”.

Y = 0.299R + 0.587G + 0.114B (1)
Cb = −0.169R−0.331G + 0.5B (2)
Cr = 0.5R−0.419G−0.081B (3)
In the equations (2) and (3) of the color difference channels Cb and Cr, when G is substituted for R and B is substituted for G, the color difference channels Cb, Cb, Cr can be represented by (GB).

Cb = −0.169G + 0.169B = −0.169 (GB) (4)
Cr = 0.5G−0.5B = 0.5 (GB) (5)
In contrast to the above-described method, in the special observation mode in the present embodiment, the first luminance signal is acquired using the first narrowband light having the purple wavelength band, and the second narrowband light having the blue wavelength band is obtained. In order to extract and display the polar surface blood vessel by acquiring the second luminance signal using the calculation signal, the calculation image signal ΔB is used instead of the (GB) signal in the equations (4) and (5). That is, the coefficient α = −0.169 is multiplied to assign the calculated image signal ΔB to the color difference channel Cb signal (color difference signal), and the coefficient β = 0.5 is multiplied to calculate the calculated image signal ΔB as the color difference channel Cr signal (color difference). Signal).

  Thereby, in the special observation mode of the endoscope system 10 of the present embodiment, it is possible to obtain a blood vessel emphasized image having substantially the same color as that of the surface blood vessel emphasized image obtained by the above-described surface blood vessel observation mode. However, in this embodiment, in order to emphasize the difference in color between the extreme surface blood vessel and the surface blood vessel located at a relatively deep position, the coefficient α and the coefficient β may be further multiplied by a coefficient depending on the setting or the like. is there.

  In order to generate an RGB blood vessel enhanced image from the luminance channel Y and the color difference channels Cb and Cr, ITU-R. According to the inverse transformation of 601, the following equation (6), equation (7), and equation (8) are used.

R = Y + 1.402Cr (6)
G = Y−0.344Cb−0.714Cr (7)
B = Y + 1.772 Cb (8)
The blood vessel emphasized image having a specific depth generated by the special observation image processing unit 67 is input to the video signal generating unit 68. The video signal generation unit 68 converts the blood vessel emphasized image having a specific depth into a video signal for display as an image that can be displayed on the monitor 18. Using this video signal, a blood vessel emphasized image having a specific depth is displayed on the monitor 18.

[Outline of image processing in special observation mode]
FIG. 12 is a flowchart showing the procedure of image processing (image generation method) in the special observation mode. This process proceeds according to a program recorded in a non-temporary recording medium such as a ROM (not shown) included in the processor device 16. When the special observation mode is selected, the image processing shown in FIG.

  In step S110, the light source 20 is illumination light (first illumination light) that is narrowband light in the first wavelength band (first wavelength) and narrowband light in the third wavelength band (third wavelength). Illumination light (third illumination light, reference light) is generated simultaneously (first light emission control). The first illumination light and the third illumination light emitted from the light source 20 are irradiated to the observation target.

  The first wavelength band is, for example, a purple wavelength band having a center wavelength of 405 nm. As shown in FIG. 13, the purple wavelength band is a wavelength band in a wavelength band (less than about 490 nm) in which the spectral sensitivity SB of the B pixel is higher than the spectral sensitivity SG of the G pixel and the spectral sensitivity SR of the R pixel.

  On the other hand, the third wavelength band is a wavelength band within a wavelength band (about 490 nm or more) in which the spectral sensitivity of at least one of the G pixel and the R pixel is higher than the spectral sensitivity of the B pixel. The wavelength. As shown in FIG. 13, the sum SS of the spectral sensitivity of the G pixel and the spectral sensitivity of the R pixel is maximized at a wavelength near 600 nm. The third illumination light having a center wavelength of 600 nm can be generated by light emission of the R-LED 23d and a change of an optical filter (not shown).

  In addition, since the 3rd illumination light is utilized as reference light, it is preferable to make light emission light quantity smaller than the light emission light quantity of 1st illumination light. Specifically, the light emission amount of the third illumination light is preferably 40% or less of the light emission amount of the first illumination light, and more preferably 10% or more and 20% or less. Therefore, the emitted light amount of the third illumination light can be set to, for example, 15% of the emitted light amount of the first illumination light, but is not limited to this value.

  Thus, the wavelength band in which the sum of the spectral sensitivity of the G pixel and the spectral sensitivity of the R pixel is maximized is the third wavelength band, and the emitted light quantity of the third illumination light (reference light) is set to the first illumination light and By reducing the amount of light emitted from the second illumination light (described later), it is possible to prevent deterioration in image quality due to reference light (decrease in contrast of difference images due to addition of long-wavelength light, reduction in resolution of surface blood vessels). The effect of correction using the reference light (removal of uneven local light quantity) can be obtained. The “wavelength at which the sum of spectral sensitivities is maximum” means not only the wavelength at which the sum of spectral sensitivities is numerically maximum, but the sum of spectral sensitivities has no substantial effect on the effect of correcting luminance unevenness. The wavelength may deviate from the maximum value in the range.

  The imaging sensor 48 images the observation target irradiated with the first illumination light and the third illumination light, and outputs an image signal corresponding to the first illumination light and the third illumination light (step S120). The image signal acquisition unit 53 acquires image signals corresponding to the first illumination light and the third illumination light from the imaging sensor 48 (step S130). The image signal acquired in step S130 corresponds to the first image signal already described. Among the first image signals, an image signal of a color corresponding to a B pixel (a pixel having the highest spectral sensitivity in the first wavelength band having a center wavelength of 405 nm) is set as a luminance signal (first luminance signal), and the third wavelength band is used. An image signal obtained by adding the image signals of the respective colors of the G pixel and the R pixel (reference signal acquisition pixel), which are pixels having sensitivity and different from the B pixel from which the first luminance signal is acquired, is referred to as the first reference signal (first signal). Reference signal).

  An example of the captured image obtained in step S130 is shown in FIG. FIG. 14 is an example of a captured image captured using the first narrowband light (first illumination light) having a center wavelength of 405 nm and the third illumination light (reference light) having a center wavelength of 600 nm. In FIG. 14, the blood vessels in the surface layer including the extreme surface blood vessels 112 are clearly projected. In FIG. 14, a region P1 indicates a region where the luminance is lower (darker) than the surroundings.

  In step S150 and step S250 (described later) in FIG. 12, the brightness correction processing unit 63 performs light amount correction on the acquired first image signal. The light amount correction in step S150 and step S250 is a process of matching the average luminance of the entire image of the first image signal and the average luminance of the second image signal before correcting the luminance signal. For example, the brightness correction processing unit 63 uses the irradiation light quantity ratio of the narrow band light used in the special observation mode, and performs gain correction for matching the average luminance of the first image signal with the average luminance of the second image signal. I do. In addition, for example, the brightness correction processing unit 63 calculates the average value of the pixel values of all the pixels of each image signal or the average value of the pixel values of a specific pixel area, thereby representing the observation represented by the first and second image signals. The brightness (average luminance) of the target image is calculated, the gain for matching the brightness of the observation target image represented by the first and second image signals is calculated, and the first image signal is calculated using the calculated gain. And at least one of the luminance signal of the second image signal is corrected.

  In step S150 and step S250, both the first luminance signal and the second luminance signal may be corrected, or only one of them may be corrected. When correcting only the second luminance signal, only the process of step S250 may be performed without performing the process of step S150. The light amount correction is synonymous with “brightness correction processing”.

  Further, after step S130, the light source 20 has illumination light (second illumination light) that is narrowband light in the second wavelength band (second wavelength) and illumination light (second illumination light) that is narrowband light in the third wavelength band (second wavelength light). (Third illumination light) are generated simultaneously (step S210: second light emission control). The second illumination light and the third illumination light emitted from the light source 20 are irradiated to the observation target. In the second light emission control, the light emission amount of the second illumination light may not be the same as the light emission amount of the first illumination light in the first light emission control. For example, the light emission amount of the second illumination light can be made smaller than the light emission amount of the first illumination light.

  The second wavelength band is, for example, a blue wavelength band having a central wavelength of 445 nm. Similarly to the first wavelength band, the wavelength band in which the spectral sensitivity of the B pixel is higher than the spectral sensitivity of the G pixel and the spectral sensitivity of the R pixel (about It is a wavelength band in less than 490 nm. The third wavelength band is the same as in the first light emission control, and narrow band light having a center wavelength of 600 nm is emitted by the light emission of the R-LED 23d and the change of an optical filter (not shown). The light emission amount of the third illumination light can be set to 15% of the light emission amount of the second illumination light as in the first light emission control, but the light emission amount of the third illumination light is set to this value. It is not limited.

  In the above example, the case where the wavelength band and the amount of emitted light of the third illumination light in the second emission control are the same as the wavelength band and the amount of emitted light of the third illumination light in the first emission control will be described. However, in the present invention, the wavelength band and the amount of emitted light of the third illumination light may not be exactly the same in the first and second light emission control. In an endoscope, automatic exposure correction is usually performed according to the observation distance, and the reference signal value is obtained by convolving the spectral sensitivity of the light receiving sensor and the wavelength distribution of the reference light. In the light emission control, it is preferable that the difference between the reference signal values obtained by convolving the spectral sensitivity of the light receiving sensor and the wavelength distribution of the reference light is within 10%.

  The imaging sensor 48 images the observation target irradiated with the second illumination light and the third illumination light, and outputs an image signal corresponding to the second illumination light and the third illumination light (step S220). The image signal acquisition unit 53 acquires image signals corresponding to the second illumination light and the third illumination light from the imaging sensor 48 (step S230). The image signal acquired in step S230 corresponds to the second image signal already described. Among the second image signals, an image signal of a color corresponding to a B pixel (a pixel having the highest spectral sensitivity in the second wavelength band having a central wavelength of 445 nm) is set as a luminance signal (second luminance signal), and the third wavelength band is used. An image signal obtained by adding the image signals of the respective colors of the G pixel and the R pixel (reference signal acquisition pixel), which are pixels having sensitivity and different from the B pixel from which the second luminance signal is acquired, is referred to as a second reference signal (second signal). Reference signal).

  In step S240, the alignment processing unit 62 performs alignment processing of the first image signal acquired in step S130 and the second image signal acquired in step S230. In this example, processing for correcting the image position of the second image signal is performed. An example of the captured image obtained in step S240 is shown in FIG. FIG. 15 is an example of a captured image captured using the second narrowband light (second illumination light) having a center wavelength of 445 nm and the third illumination light (reference light) having a center wavelength of 600 nm. Although not sufficiently expressed in FIG. 15 due to the illustrated constraints, the contrast of the extreme surface blood vessel 112 in the captured image illustrated in FIG. 15 is significantly lower than that of the captured image in FIG. Further, in the superficial blood vessel 114 located deeper than the extreme superficial blood vessel 112, the contrast lowers slowly than in FIG. In FIG. 15, a region P2 indicates a region having a higher brightness (brighter) than the surroundings.

  In step S250 of FIG. 12, the brightness correction processing unit 63 performs light amount correction on the acquired second image signal. The light amount correction in step S250 is a process for matching the average luminance of the first luminance signal and the average luminance of the second luminance signal, as in the case of the light amount correction in step S150, and the first luminance signal and the second luminance. Both of the signals may be corrected, or only one of them may be corrected. When correcting only the first luminance signal, only the process of step S150 described above may be performed without performing the process of step S250.

  In step S152, the frequency processing unit 64b performs LPF (Low Pass Filter) processing (low pass filter processing) for reducing high frequency components included in the reference signals (first and second reference signals). This is because, since the light receiving sensor has subsensitivity, the image signal of the surface blood vessel is reflected in the reference signal, and if the light amount is corrected using the reference signal as it is, the contrast of the surface blood vessel is lowered. Since the influence of the subsensitivity of the light receiving sensor (the degree to which the image signal of the surface blood vessel is reflected in the reference signal) varies depending on the observation distance to the subject, the target frequency characteristic is stored in the storage unit 64a for each observation distance. The frequency processing unit 64b acquires the target frequency characteristic from the storage unit 64a based on the observation distance at the time of imaging, and performs low-pass filter processing so that the frequency characteristic of the reference signal becomes the target frequency characteristic. For example, the characteristics of the low-pass filter are stored as the target frequency characteristics and used. In this way, the frequency processing unit 64b performs low-pass filter processing in which the strength of the low-pass filter increases as the observation distance is shorter as shown in FIG.

  Note that the high-frequency component of the reference signal including the effect of the sub-sensitivity is extracted without performing the above-described low-pass filter processing in step S152, and this high-frequency component is added to the difference image to maintain the contrast of the difference image. May be.

  The luminance correction unit 64d corrects variations in the reference signals (first reference signal and second reference signal) between the first and second image signals (step S154). This is because even if the reference light is emitted with the same light amount in each frame (first and second light emission control), the reference signal obtained from the reference light component is different in sub-sensitivity of the light receiving sensor depending on the wavelength band (see FIG. 13)), the reference signal (first and second reference signals) is determined from the set light quantity and spectral sensitivity of the short-wavelength illumination light (first and second illumination lights) of each frame. This is to correct the variation. In addition to the above-described correction or in addition to the above-described correction, the light emission amount of the reference light at the time of the first and second light emission control (steps S110 and S210) is set in consideration of variations in the reference signal in the image signal. May be.

  The luminance correction unit 64d corrects the first luminance signal and the second luminance signal locally (for each pixel or for each imaging region) based on the reference signal corrected in step S154 (step S156). . The process in step S156 is a process of correcting the first luminance signal and the second luminance signal based on the ratio of the reference signal in each pixel or each image region. For example, the ratio of the first reference signal and the second reference signal in the pixel to be processed is 1.0 to 0.9 (first reference signal: second reference signal = 1.0: 0.9). ), The second luminance signal is corrected by multiplying it by (1.0 / 0.9). In this example, only the second luminance signal is corrected. However, only the first luminance signal may be corrected, or both the first and second luminance signals may be corrected.

  In the case of so-called “multi-frame shooting” in which a plurality of narrow-band lights having different wavelength bands are switched sequentially as in this embodiment, since the shooting timing for each frame is different, the observation distance and angle change, and each frame The difference in image brightness (luminance) is not uniform, but changes from place to place. For this reason, the luminance may not be adjusted only by correcting the average luminance in steps S150 and S250, but luminance unevenness at each location is removed by correcting the luminance signal based on the ratio of the reference signals as described above. Images can be obtained (see FIGS. 14-19).

  In step S160, the arithmetic image signal generation unit 76 performs logarithmic conversion on the corrected first luminance signal. In step S260, the arithmetic image signal generation unit 76 performs logarithmic conversion on the corrected second luminance signal.

  In step S280, the calculation image signal generation unit 76 performs difference processing for generating a difference image between the second image signal logarithmically converted in step S260 and the first image signal logarithmically converted in step S160. . In the difference processing in step S280, an operation image signal representing the difference image is generated by an operation of subtracting the first luminance signal from the second luminance signal. The calculation image signal generated in step S280 corresponds to the calculation image signal ΔB already described.

  In step S290, the low-pass filter processing unit 77 performs low-pass filter processing on the arithmetic image signal generated in step S280.

  Thereafter, in step S300, the image generation unit 78 assigns the first luminance signal logarithmically converted in step S160 to the luminance channel Y, and the calculated image signal reduced in step S290 is the color difference channel. By assigning to Cr and Cb, a blood vessel emphasized image (YCC image) in which the blood vessel which is the structure of the subject is emphasized by color is generated. In addition, the image generation unit 78 performs color conversion processing for converting a YCC image into an RGB image, and generates an RGB image 100 representing a blood vessel emphasized image.

  The blood vessel emphasized image generated in step S300 is displayed on the monitor 18. Further, the blood vessel emphasized image generated in step S300 can be stored in the storage 70.

  An example of an output image generated through step S300 is shown in FIG. In FIG. 16, a color image in which the surface blood vessels are emphasized is obtained as an output image although it is not sufficiently expressed due to the constraints shown.

  In the blood vessel-enhanced image generated in this way, the extreme superficial blood vessels in the extreme superficial layer below the mucous membrane are displayed in magenta color, and the superficial blood vessels in the superficial layer deeper than the extreme superficial layer are cyan. It is expressed by coloring the system. Therefore, in the blood vessel-enhanced image, the extreme surface blood vessel and the superficial blood vessel can be distinguished by color, and are displayed on the monitor 18 as an image that allows easy observation of the extreme surface blood vessel. In the image displayed on the monitor 18, since the luminance signal is corrected based on the reference signal as described above, local luminance unevenness such as the regions P1 and P2 shown in FIGS. The structure of the specimen can be easily observed.

[Specific image example]
FIG. 17 is an example of a captured image captured using first narrowband light with a center wavelength of 405 nm as first illumination light and narrowband light with a center wavelength of 600 nm as third illumination light (reference light). . FIG. 17 corresponds to an image example based on the first image signal acquired in step S130 of FIG. The captured image illustrated in FIG. 17 is referred to as a first captured image 110. In the first captured image 110, a superficial blood vessel 112 and a superficial blood vessel 114 are shown. FIG. 14 described above corresponds to an enlarged image of a part of FIG. In FIG. 17, the illustration of the region P1 shown in FIG. 14 is omitted.

  FIG. 18 is an example of a captured image captured using second narrowband light with a center wavelength of 445 nm as second illumination light and narrowband light with a center wavelength of 600 nm as third illumination light (reference light). . FIG. 18 corresponds to an image example based on the second image signal acquired in step S230 of FIG. The captured image illustrated in FIG. 18 is referred to as a second captured image 120. In the second captured image 120, the contrast between the polar surface blood vessel 112 and the surface blood vessel 114 is lower than that of the first captured image 110. FIG. 15 described above corresponds to an enlarged image of a part of FIG. In FIG. 18, illustration of the region P2 shown in FIG. 15 is omitted.

  FIG. 19 is an example of a blood vessel emphasized image generated using a calculated image signal based on the difference between the first captured image 110 and the second captured image 120. FIG. 16 described above corresponds to an enlarged image of a part of FIG. In the blood vessel emphasized image shown in FIG. 19, the blood vessel as the structure of the subject can be easily observed visually.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to embodiment mentioned above, A various deformation | transformation is possible in the range which does not deviate from the meaning of this invention. For example, in the above-described embodiment, purple light V and red light R (reference light) are emitted by the first light emission control, the first image signal is acquired, and blue light B and red light R ( Reference light) is emitted to obtain a second image signal, and an image emphasizing blood vessels in the extreme surface layer to the surface layer is generated by these two image signals. Blue light B is generated by the first light emission control. And red light R (reference light) to obtain a first image signal, and second light emission control to emit green light G and red light R (reference light) to obtain a second image signal. Alternatively, an image in which a blood vessel in the middle layer is emphasized may be generated using these two image signals. The processing for generating an image in which such middle-layer blood vessels are emphasized can be performed in the same manner as in the above-described embodiment.

  Further, in the above-described embodiment, two narrowband images are acquired using two narrowband lights having different wavelengths, but three narrowband images are acquired using three narrowband lights having different wavelengths, A blood vessel in a desired layer may be highlighted based on these three narrow-band images. For example, purple light V and red light R (reference light) are emitted by the first light emission control to acquire a first image signal, and blue light B and red light R (reference light) are obtained by the second light emission control. The second image signal is acquired by emitting light, and the third image signal is acquired by emitting green light G and red light R (reference light) by the third light emission control. Then, an image in which blood vessels in the extreme surface layer to the surface layer are highlighted is generated from the first image signal and the second image signal, and an image in which the blood vessels in the middle layer are highlighted from the second image signal and the third image signal is generated. can do. Such a blood vessel enhanced image can be generated in the same manner as in the above-described embodiment.

  In the above-described embodiment, the reference light (center wavelength 600 nm) is generated by the emission of the R-LED 23d and the change of the optical filter (not shown). Light may be generated. Alternatively, both the R-LED 23d and the G-LED 23c may emit light while adjusting the light emission amount, and the reference light may be emitted by changing an optical filter (not shown) in accordance with this. Further, the reference light may be emitted by adjusting the light emission balance of the LEDs of a plurality of colors.

DESCRIPTION OF SYMBOLS 10 Endoscope system 12 Endoscope 12a Insertion part 12b Operation part 12c Bending part 12d Tip part 12e Angle knob 13a Mode changeover switch 13b Zoom operation part 13c Still image acquisition instruction | indication part 14 Light source device 16 Processor apparatus 18 Monitor 19 Console 20 Light source 22 Light source controller 23a V-LED
23b B-LED
23c G-LED
23d R-LED
30a Illumination optical system 30b Imaging optical system 41 Light guide 45 Illumination lens 46 Objective lens 47 Zoom lens 48 Imaging sensor 51 AFE circuit 52 AD converter 53 Image signal acquisition unit 58 Noise reduction unit 59 Memory 60 Signal processing unit 61 Image processing switching unit 62 Positioning processing unit 63 Brightness correction processing unit 64 Reference light correction unit 64a Storage unit 64b Frequency processing unit 64c Reference signal correction unit 64d Luminance correction unit 66 Normal observation image processing unit 67 Special observation image processing unit 68 Video signal generation unit 70 Storage 72 Image processing device 76 Operation image signal generation unit 77 Low-pass filter processing unit 78 Image generation unit 100 RGB image 110 First captured image 112 Polar surface blood vessel 114 Surface layer blood vessel 120 Second captured image Ad Range As Range B Blue light Cb Color difference channel Cr Color difference channel G Green light P1 region P2 region R red light S110 to S300 Steps SB of image generation method SB spectral sensitivity SG spectral sensitivity SR spectral sensitivity SS total V purple light Y luminance channel YM luminance YV luminance ΔB arithmetic image signal α coefficient β coefficient

Claims (18)

An imaging unit including an objective lens and an imaging element having a first pixel, a second pixel, and a third pixel in which color filters corresponding to the three primary colors are arranged;
A light source that emits a first illumination light, a second illumination light, and a third illumination light, each having a peak at a first wavelength, a second wavelength, and a third wavelength, each having a different wavelength;
First light emission control for simultaneously emitting the first illumination light and the third illumination light from the light source, and the second illumination light and the third illumination when the subject is imaged by the imaging unit. A light source controller that performs second light emission control for simultaneously emitting light;
When the first light emission control is performed by the light source control unit, the spectral sensitivity is highest at the first wavelength among the first pixel, the second pixel, and the third pixel of the image sensor. A pixel that obtains a first luminance signal from the pixel and is sensitive to the third wavelength among the first pixel, the second pixel, and the third pixel of the imaging device, and at least the first pixel When a first reference signal is acquired from a pixel different from the pixel having the highest spectral sensitivity for one wavelength and the second light emission control is performed by the light source control unit, the first pixel of the image sensor, Of the second pixel and the third pixel, the second luminance signal is obtained from the pixel having the highest spectral sensitivity at the second wavelength, and the first pixel, the second pixel, and the second pixel of the image sensor Of the third pixels, the pixels having sensitivity to the third wavelength, Both the image signal acquisition unit that acquires the second reference signal from a different pixel from the high pixel most spectral sensitivity in the second wavelength,
Endoscope system equipped with. A luminance correction unit that corrects at least one of the first luminance signal and the second luminance signal for each pixel or each imaging region based on the first reference signal and the second reference signal; ,
An image signal generation unit that generates an image signal in which the structure of the subject is emphasized based on the first luminance signal and the second luminance signal after correction by the luminance correction unit;
The endoscope system according to claim 1, further comprising:   The endoscope system according to claim 2, wherein a light emission amount of the third illumination light is smaller than a light emission amount of the first illumination light and the second illumination light.   The first reference signal and the second reference signal are pixels obtained from the first luminance signal and the second luminance signal, respectively, of the first pixel, the second pixel, and the third pixel. The endoscope system according to claim 2 or 3, wherein the endoscope system is an image signal obtained by adding image signals of respective colors of reference signal acquisition pixels which are different from the reference signal.   The endoscope system according to claim 4, wherein the third illumination light has a wavelength that maximizes a sum of spectral sensitivities of the reference signal acquisition pixels. Based on the set light amounts of the first illumination light and the second illumination light, and the spectral sensitivity of the reference signal acquisition pixel with respect to the first illumination light and the second illumination light, respectively, the first illumination light and the second illumination light. A reference signal correction unit for correcting the reference signal and the second reference signal,
The luminance correction unit is a luminance of at least one of the first luminance signal and the second luminance signal based on the first reference signal and the second reference signal corrected by the reference signal correction unit. The endoscope system according to claim 4 or 5, wherein the signal is corrected for each pixel or each imaging region.   7. The apparatus according to claim 2, further comprising an alignment unit configured to align an image indicated by the first luminance signal acquired by the image signal acquisition unit and an image indicated by the second luminance signal. The endoscope system described in 1.   Before correcting the luminance signal of at least one of the first luminance signal and the second luminance signal by the luminance correction unit, the average luminance of the first luminance signal and the second luminance signal is matched. The endoscope system according to any one of claims 2 to 7, further comprising an average luminance correction unit that performs correction.   The luminance correction unit multiplies one luminance signal of the first luminance signal and the second luminance signal by a ratio of the first reference signal and the second reference signal. The endoscope system according to any one of 8. A frequency processing unit for reducing high frequency components of the first reference signal and the second reference signal;
The luminance correction unit is a luminance of at least one of the first luminance signal and the second luminance signal based on the first reference signal and the second reference signal processed by the frequency processing unit. The endoscope system according to any one of claims 2 to 9, wherein the signal is corrected for each pixel or each imaging region.   The endoscope system according to claim 10, wherein the frequency processing unit reduces high frequency components of the first reference signal and the second reference signal based on an observation distance from the imaging unit to the subject. A storage unit that holds target frequency characteristics targeted for each observation distance from the imaging unit to the subject;
The frequency processing unit acquires a corresponding target frequency characteristic from the storage unit based on an observation distance at the time of imaging by the imaging unit, and acquires the frequency characteristics of the first reference signal and the second reference signal. The endoscope system according to claim 11, wherein the endoscope system is adjusted to a target frequency characteristic.   13. The frequency processing unit according to claim 11, wherein the frequency processing unit strengthens a process of reducing a high-frequency component from the first reference signal and the second reference signal as the observation distance at the time of imaging by the imaging unit is closer. Endoscope system. The image signal generator is
A difference calculation unit for calculating a difference signal between the first luminance signal and the second luminance signal corrected by the luminance correction unit;
A color difference signal generation unit that generates a color difference signal from the calculated difference signal;
A color image signal having one of the first luminance signal and the second luminance signal and the generated color difference signal, or the first luminance signal and the second luminance signal; A color image generation unit that generates a color image signal obtained by converting one luminance signal and the generated color difference signal into each image signal of the three primary colors;
The endoscope system according to any one of claims 2 to 13, comprising: The structure of the subject is a blood vessel,
The endoscope system according to claim 2, wherein the image signal generation unit generates the image signal in which the blood vessel is emphasized by color. The processor device of the endoscope system according to any one of claims 2 to 15,
The processor apparatus provided with the said light source, the said light source control part, the said image signal acquisition part, the said brightness correction part, and the said image signal generation part. When an object is imaged by an imaging unit including an objective lens and an imaging device having a first pixel, a second pixel, and a third pixel in which color filters corresponding to the three primary colors are arranged, the wavelengths are different from each other. The first illumination light and the first illumination light are emitted from a light source that emits the first illumination light, the second illumination light, and the third illumination light having peaks at the first wavelength, the second wavelength, and the third wavelength. Sequentially performing a first light emission control for simultaneously emitting the third illumination light and a second light emission control for simultaneously emitting the second illumination light and the third illumination light;
When the first light emission control is performed, the first pixel, the second pixel, and the third pixel of the image pickup device are changed from the pixel having the highest spectral sensitivity to the first wavelength. A pixel that obtains a luminance signal and is sensitive to the third wavelength among the first pixel, the second pixel, and the third pixel of the imaging device, and is at least the first wavelength When the first reference signal is acquired from a pixel different from the pixel having high spectral sensitivity and the second light emission control is performed, the first pixel, the second pixel, and the third pixel of the image sensor Of these, the second luminance signal is obtained from the pixel having the highest spectral sensitivity at the second wavelength, and the third of the first pixel, the second pixel, and the third pixel of the image sensor. A pixel having sensitivity to a wavelength of at least the highest spectral sensitivity to at least the second wavelength Obtaining a second reference signal from a different pixel from the element,
A signal processing method including: Correcting at least one luminance signal of the first luminance signal and the second luminance signal for each pixel or each imaging region based on the first reference signal and the second reference signal;
Generating an image signal in which the structure of the subject is emphasized based on the first luminance signal and the second luminance signal whose luminance has been corrected;
The signal processing method according to claim 17, further comprising:

Images