JP6312254B2 - Endoscope system and operating method thereof - Google Patents

Description

  The present invention relates to an endoscope system and an operating method thereof.

  In the medical field, diagnosis using an endoscope system including an endoscope light source device (hereinafter referred to as a light source device), an endoscope, and a processor device is widely performed. The light source device is a device that generates illumination light that irradiates an observation target such as a mucous membrane of a body cavity. The endoscope includes an image sensor that images an observation target irradiated with illumination light.

  As this endoscope system, a system using a light source device that emits white light as illumination light and using a simultaneous color sensor as an image sensor is known. This color sensor images incident light incident on an endoscope from an observation object by splitting it into blue (B) light, green (G) light, and red (R) light.

  White light emitted from the light source device can be adjusted for each color because the amount of B light, G light, and R light varies depending on the characteristics of the light source, and the spectral sensitivity characteristics of the image sensor differ for each color. A light source device is known (see Patent Document 1). This light source device makes it possible to adjust the amounts of B light, G light, and R light included in white light. In Patent Document 1, it is preferable to make the light amount of B light larger than the light amount of G light and make the light amount of G light larger than the light amount of R light (that is, B light> G light> R light).

JP 2012-125395 A

  Among the observation targets, the mucous membrane of the large intestine has a characteristic that the reflectance is smaller as the wavelength is shorter. On the other hand, the important information obtained from the vicinity of the surface of the mucosa is dominated by the return light of the shorter wavelength. Since the surface layer of the mucous membrane includes a fine structure such as a capillary vessel containing hemoglobin that strongly absorbs B light, which is a short wave light, in order to generate an image including this fine structure, it is described in Patent Document 1. As described above, it is preferable that the light amounts of the respective colors emitted from the light source device have a relationship of “B light> G light> R light”. However, at this time, the image pickup signal level of the image pickup element has a relationship of “B pixel signal> G pixel signal> R pixel signal”, and the level of the R pixel signal may decrease.

  Patent Document 1 describes that a special light observation image is generated using a B pixel signal and a G pixel signal. In such special light observation, since the R pixel signal is not used for image generation, there is little influence of the image quality deterioration due to the above light quantity relationship. However, in recent years, attention has been focused on color enhancement processing for performing image processing on an image captured using white light and clarifying a boundary region between a normal portion and a lesioned portion. As described above, when the color enhancement process is performed, it is preferable that a red image can be obtained with an image quality equivalent to that of a blue image or a green image.

  Some endoscopes use a complementary color sensor as an image sensor. The complementary color sensor captures and captures incident light incident on an endoscope from an observation target into cyan (Cy) light, magenta (Mg) light, yellow (Ye) light, and green (G) light. Such a complementary color type image sensor has higher sensitivity than the primary color type, but is inferior in color separation with respect to R, G, and B. For this reason, when a complementary color sensor is used as the image sensor, the image quality of the red image obtained by the color enhancement process is further deteriorated.

  It is an object of the present invention to provide an endoscope system and an operation method thereof that can improve the image quality of a red image and obtain a high-quality observation image by color enhancement processing.

  In order to achieve the above object, an endoscope system of the present invention includes a light source unit that emits first light, second light, and third light having different colors as illumination light, and a light emission timing and light emission intensity of the illumination light. And the second light, the first light, and the third light are individually emitted, and the light emission time of the first light is longer than the light emission time of the second light and the light emission time of the third light. A light source control unit that increases the emission intensity in the order of the first light, the second light, and the third light, a complementary color type imaging device that receives the return light from the observation target irradiated with the illumination light, and a light emission timing. In addition, an imaging control unit that controls the imaging timing of the imaging device to receive the second light, the first light, and the third light individually is provided.

  The first light is red light, the second light is green light, the third light is blue light, and the imaging device includes a cyan pixel that receives the second light and the third light, and the first light and the first light. It is preferable to include a magenta pixel that receives three lights, a yellow pixel that receives first and second lights, and a green pixel that receives second light.

  It is preferable that the imaging device has even pixel rows in which cyan pixels and yellow pixels are alternately arranged, and odd pixel rows in which magenta pixels and green pixels are alternately arranged.

  It is preferable that the imaging control unit causes the odd-numbered pixel rows to be read from the imaging element after receiving the second light, and causes the even-numbered pixel rows to be read from the imaging element after receiving the first light and the third light. In this case, it is preferable to include a signal processing unit that generates an observation image using the green pixel signal read from the odd pixel row and the cyan pixel signal and the yellow pixel signal read from the even pixel row. .

  It is also preferable that the imaging control unit read all the pixels of the imaging element after receiving the second light, and read all the pixels of the imaging element after receiving the first light and the third light. In this case, the green pixel signal, cyan pixel signal, and yellow pixel signal read from the image sensor after receiving the second light, and magenta read from the image sensor after receiving the first light and the third light. It is preferable to include a signal processing unit that generates an observation image using the pixel signal, the cyan pixel signal, and the yellow pixel signal.

  The signal processing unit generates a red pixel signal by subtracting the cyan pixel signal from the magenta pixel signal read from the image sensor after receiving the first light and the third light, and subtracts the yellow pixel signal from the magenta pixel signal. It is preferable to generate a blue pixel signal.

  The operation method of the endoscope system according to the present invention includes a light source unit that emits first light, second light, and third light having different colors as illumination light, and return light from an observation target irradiated with the illumination light. In the operation method of the endoscope system including the complementary color type image pickup device, the second light, the first light, and the third light are individually emitted by controlling the light emission timing and light emission intensity of the illumination light. In addition, the light emission time of the first light is set longer than the light emission time of the second light and the light emission time of the third light, and the light emission intensity is increased in the order of the first light, the second light, and the third light. And the second light, the first light, and the third light are individually received.

  According to the present invention, the second light, the first light, and the third light are individually emitted, and the light emission time of the first light is determined from the light emission time of the second light and the light emission time of the third light. The light intensity is increased in the order of the first light, the second light, and the third light, and the image sensor is controlled to receive the second light, the first light, and the third light individually. Therefore, it is possible to improve the image quality of the red image and obtain a high-quality observation image by the color enhancement process.

It is an external view of an endoscope system. It is a block diagram which shows the function of an endoscope system. It is a figure which shows the structure of an image pick-up element. It is a figure which shows the color arrangement | sequence of a color filter. It is a figure which shows the spectral characteristic of a color filter. It is a figure which shows the structure of the pixel of an image pick-up element. It is a figure which shows the timing of light emission and an imaging. It is a graph which shows the spectral reflectance of a mucous membrane. It is a figure which shows the timing of light emission and imaging of a global shutter system. It is a figure which shows the timing of light emission and imaging of 2nd Embodiment. It is a figure which shows the 1st modification of the color arrangement | sequence of a color filter. It is a figure which shows the 2nd modification of the color arrangement | sequence of a color filter. It is a figure which shows the 3rd modification of the color arrangement | sequence of a color filter.

[First Embodiment]
In FIG. 1, an endoscope system 10 includes an endoscope 12, an endoscope light source device (hereinafter referred to as a light source device) 14, a processor device 16, a monitor 18, and a console 19. The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16. The endoscope 12 includes an insertion portion 12a to be inserted into a body cavity, an operation portion 12b provided at the proximal end portion of the insertion portion 12a, a bending portion 12c and a distal end portion 12d provided at the distal end side of the insertion portion 12a. have.

  By operating the angle knob 12e of the operation unit 12b, the bending unit 12c performs a bending operation. By this bending operation, the distal end portion 12d is directed in a desired direction. In addition to the angle knob 12e, the operation unit 12b is provided with a zoom operation unit 13 and the like.

  The light source device 14 generates illumination light and supplies it to the endoscope 12. The illumination light supplied to the endoscope 12 is guided to the distal end portion 12d and irradiated from the distal end portion 12d to the observation target in the body cavity.

  The processor device 16 is electrically connected to the monitor 18 and the console 19. The monitor 18 outputs and displays an observation image and image information attached to the observation image. The console 19 functions as a user interface that receives input operations such as function settings. The processor device 16 can be connected to an external recording unit (not shown) that records observation images, image information, and the like.

  As shown in FIG. 2, the light source device 14 includes a light source unit 20 that generates illumination light for illuminating an observation target, and a light source control unit 21 that controls the light emission timing and light emission intensity of the light source unit 20. . The light source unit 20 includes a drive unit 22, first to third light sources 23 a to 23 c, and an optical path integration unit 24. The drive unit 22 drives the first to third light sources 23 a to 23 c based on the control from the light source control unit 21.

  The first light source 23a is a red LED (Light-emitting diode) that emits red light (first light) LR. For example, the red light LR has a wavelength band of 615 nm to 635 nm and a center wavelength of 620 ± 10 nm. The second light source 23b is a green LED that emits green light (second light) LG. For example, the green light LG has a wavelength band of 500 nm to 600 nm and a center wavelength of 520 ± 10 nm. The third light source 23c is a blue LED that emits blue light (third light) LB. For example, the blue light LB has a wavelength band of 440 nm to 470 nm and a center wavelength of 455 ± 10 nm. The red light LR, the green light LG, and the blue light LB are individually emitted under the control of the light source control unit 21.

  The optical path integration unit 24 is configured by a dichroic mirror or the like, and integrates the optical paths of the respective lights emitted from the first to third light sources 23a to 23c. The light emitted from the optical path integration unit 24 is supplied as illumination light to the light guide 25 inserted into the insertion unit 12a.

  The light guide 25 is built in the endoscope 12 and propagates illumination light to the distal end portion 12 d of the endoscope 12. A multimode fiber can be used as the light guide 25. For example, a thin fiber cable having a core diameter of 105 μm, a clad diameter of 125 μm, and a diameter of φ0.3 to 0.5 mm including a protective layer serving as an outer cover can be used as the light guide 25.

  The distal end portion 12d of the endoscope 12 is provided with an illumination optical system 30a and an imaging optical system 30b. The illumination optical system 30 a has an illumination lens 31. The illumination light that has propagated through the light guide 25 is applied to the observation target via the illumination lens 31. The imaging optical system 30 b includes an objective lens 32, a zoom lens 33, and an imaging element 34. Return light from the observation target enters the image sensor 34 via the objective lens 32 and the zoom lens 33. As a result, an optical image to be observed is formed on the imaging surface (not shown) of the imaging element 34. The zoom lens 33 is freely moved between the tele end and the wide end by operating the zoom operation unit 13, and enlarges or reduces the optical image of the observation target formed on the imaging surface of the image sensor 34. .

  The image sensor 34 is a simultaneous complementary color sensor, and receives the return light from the observation target irradiated with the illumination light and outputs an image signal. The imaging device 34 can receive the return light separately for each of cyan (Cy), magenta (Mg), yellow (Ye), and green (G) colors. As the imaging device 34, a complementary metal-oxide semiconductor (CMOS) imaging device is used. The image sensor 34 outputs an image signal including a Cy pixel signal, an Mg pixel signal, a Ye pixel signal, and a G pixel signal as an image signal.

  The processor device 16 includes an imaging control unit 40, a reception unit 41, a DSP (Digital Signal Processor) 42, a noise removal unit 43, an observation image generation unit 44, and a video signal generation unit 45. The DSP 42, the noise removal unit 43, and the observation image generation unit 44 correspond to the signal processing unit described in the claims.

  The imaging control unit 40 controls the imaging timing of the observation target by the imaging element 34. The receiving unit 41 receives a digital image signal output from the imaging device 34 of the endoscope 12. The DSP 42 performs various signal processing such as defect correction processing, offset processing, gain correction processing, linear matrix processing, gamma conversion processing, and demosaicing processing on the received image signal.

  In the defect correction process, the signal of the defective pixel of the image sensor 34 is corrected. In the offset process, the dark current component is removed from the image signal subjected to the defect correction process, and an accurate zero level is set. In the gain correction process, the signal level is adjusted by multiplying the image signal after the offset process by a specific gain value. The image signal after gain correction processing is subjected to linear matrix processing for improving color reproducibility. After that, brightness and saturation are adjusted by gamma conversion processing. The image signal after the linear matrix processing is subjected to demosaic processing (also called isotropic processing or synchronization processing).

  The noise removal unit 43 removes noise by performing noise removal processing (processing by a moving average method, a median filter method, or the like) on the image signal subjected to demosaic processing or the like by the DSP 42. The image signal from which the noise has been removed is input to the observation image generation unit 44.

  The observation image generation unit 44 generates an observation image by performing color conversion processing, color enhancement processing, and structure enhancement processing on the image signal input from the noise removal unit 43. In the color conversion processing, color conversion processing is performed on the image signal by 3 × 3 matrix processing, gradation conversion processing, three-dimensional LUT (look-up table) processing, and the like. The color enhancement process is performed on the image signal that has been subjected to the color conversion process. The structure enhancement process is a process for enhancing the structure of the observation target such as a surface blood vessel or a pit pattern, and is performed on the image signal after the color enhancement process.

  The observation image generated by the observation image generation unit 44 is input to the video signal generation unit 45. The video signal generation unit 45 converts each image into a video signal for display on the monitor 18. The monitor 18 displays an image based on the video signal input from the video signal generation unit 45.

  In FIG. 3, the imaging device 34 includes a pixel array unit 50, a readout scanning circuit 51, a reset scanning circuit 52, a column ADC (Analog-to-digital converter) circuit 53, a line memory 54, and a column scanning circuit 55. And a timing generator (TG) 56. The TG 56 generates a timing signal based on the imaging control signal input from the imaging control unit 40 of the processor device 16 and controls each unit.

  The pixel array unit 50 includes a plurality of pixels 50 a two-dimensionally arranged in a matrix in the row direction (X direction) and the column direction (Y direction), and is provided on the imaging surface of the imaging element 34. In the pixel array unit 50, a row selection line LS and a row reset line LR are arranged along the row direction, and a column signal line LV is arranged along the column direction.

  The row selection line LS and the row reset line LR are provided for each pixel row. The column signal line LV is provided for each pixel column. Here, the pixel row refers to one row of pixels 50a arranged in the row direction. The pixel column refers to one column of pixels 50a arranged in the column direction. Each pixel 50a in one pixel row is commonly connected to the row selection line LS and the row reset line LR.

  As shown in FIG. 4, a color filter array 60 is provided on the light incident side of the pixel array unit 50. The color filter array 60 includes a cyan (Cy) filter 60a, a magenta (Mg) filter 60b, a yellow (Ye) filter 60c, and a green (G) filter 60d. Any one of these filters is arranged on each pixel 50a. The color arrangement of the color filter array 60 is a so-called complementary checkered color difference line sequential method.

  The color filter array 60 has the spectral characteristics shown in FIG. The Cy filter 60a has a transmission characteristic that transmits substantially the blue light LB and the green light LG. The Mg filter 60b has a transmission characteristic that transmits substantially the blue light LB and the red light LR. The Ye filter 60c has a transmission characteristic that transmits substantially the green light LG and the red light LR. The G filter 60d has a transmission characteristic that transmits green light LG.

  Hereinafter, the pixel 50a in which the Cy filter 60a is disposed is referred to as a Cy pixel, the pixel 50a in which the Mg filter 60b is disposed is referred to as an Mg pixel, the pixel 50a in which the Ye filter 60c is disposed is referred to as a Ye pixel, and the G filter 60d. A pixel 50a in which is arranged is referred to as a G pixel. Cy pixels and Ye pixels are alternately arranged in each pixel row of even numbers (0, 2, 4,..., N−1). In each odd-numbered (1, 3, 5,..., N) pixel row, Mg pixels and G pixels are alternately arranged.

  As shown in FIG. 6, each pixel 50a includes a photodiode D1, an amplifier transistor M1, a pixel selection transistor M2, and a reset transistor M3. The photodiode D1 photoelectrically converts incident light to generate signal charges corresponding to the amount of incident light, and accumulates the signal charges. The amplifier transistor M1 converts the signal charge accumulated in the photodiode D1 into a voltage value (pixel signal). The pixel selection transistor M2 is controlled by the row selection line LS, and outputs the pixel signal generated by the amplifier transistor M1 to the column signal line LV. The reset transistor M3 is controlled by the row reset line LR, and discards (resets) the signal charge accumulated in the photodiode D1 to the power supply line.

  The read scanning circuit 51 generates a row selection signal based on the timing signal input from the TG 56. The readout scanning circuit 51 applies a row selection signal to the row selection line LS during the signal readout operation, thereby causing the pixel signal of the pixel 50a connected to the row selection line LS to which the row selection signal is applied to be output to the column signal line LV. To output.

  The reset scanning circuit 52 generates a reset signal based on the timing signal input from the TG 56. During the reset operation, the reset scanning circuit 52 applies a reset signal to the row reset line LR, thereby resetting the pixels 50a connected to the row reset line LR to which the reset signal is applied.

  The column ADC circuit 53 receives the pixel signal output to the column signal line LV during the signal read operation. The column ADC circuit 53 includes an ADC connected to each column signal line LV, and compares the pixel signal input from each column signal line LV with a reference signal (ramp wave) that changes stepwise with time. , Converted into a digital signal and output to the line memory 54.

  The line memory 54 holds pixel signals for one row digitized by the column ADC circuit 53. The column scanning circuit 55 sequentially outputs pixel signals from the output terminal Vout by scanning the line memory 54 based on the timing signal input from the TG 56. The pixel signal for one frame output from the output terminal Vout is the aforementioned image signal.

  The image sensor 34 can execute an “all-pixel readout method” and a “thinning-out method” as signal readout methods. In the all-pixel readout method, a row selection signal is given to the selected row selection line LS while the row scanning line LS of each pixel row is sequentially selected by the readout scanning circuit 51. As a result, for all the pixels 50a in the pixel array unit 50, signal reading is sequentially performed for each pixel row from the first pixel row “0” to the last pixel row “N”.

  In the thinning-out method, even (0, 2, 4,..., N−1) pixel rows (hereinafter referred to as even pixel rows) or odd (1, 3, 5,. N) pixel rows (hereinafter referred to as odd pixel rows) can be selectively read out. For example, when only the even-numbered pixel rows are read from the pixel array unit 50, only the row selection lines LS of the even-numbered pixel rows are sequentially selected by the read scanning circuit 51, and a row selection signal is sent to the selected row selection line LS. Given. As a result, signal readout is sequentially performed for each pixel row only for even-numbered pixel rows among all the pixel rows.

  In this case, the line memory 54 stores pixel signals (Cy pixel signal and Ye pixel signal) read from the even pixel rows. The column scanning circuit 55 scans the line memory 54 every time pixel signals (Cy pixel signal and Ye pixel signal) for one pixel row are stored in the line memory 54.

  The same applies to the case where only odd-numbered pixel rows are read out. In this case, the line memory 54 stores pixel signals (Mg pixel signal and G pixel signal) read from the odd pixel rows. The column scanning circuit 55 scans the line memory 54 every time pixel signals (Mg pixel signal and G pixel signal) for one pixel row are stored in the line memory 54.

  The image sensor 34 can execute a “sequential reset method” and a “batch reset method” as a reset method. In the sequential reset method, the reset signal is given to the selected row reset line LR while the row reset line LR is sequentially selected by the reset scanning circuit 52. As a result, in the sequential reset method, the reset is performed sequentially for each pixel row from the first pixel row “0” to the last pixel row “N”.

  In the collective reset method, all the row reset lines LR are selected by the reset scanning circuit 52, and a reset signal is collectively applied to all the row reset lines LR. Thereby, all the pixel rows of the pixel array unit 50 are simultaneously reset at the same time.

  Although not shown in FIG. 3, the imaging device 34 is also provided with a correlated double sampling (CDS) circuit and an automatic gain control (AGC) circuit as appropriate. The CDS circuit performs correlated double sampling processing on the pixel signal output from the pixel 50a to each column signal line LV. The AGC circuit performs gain adjustment on the pixel signal on which the correlated double sampling processing has been performed.

  The light source control unit 21 and the imaging control unit 40 are electrically connected to each other. The imaging control unit 40 controls the imaging timing of the imaging device 34 in accordance with the emission timing of the illumination light of the light source device 14 controlled by the light source control unit 21.

  Next, the light emission timing and imaging timing controlled by the light source control unit 21 and the imaging control unit 40 will be described. In the image pickup device 34 which is a complementary color sensor, the Cy pixel, the Mg pixel, and the Ye pixel receive two lights of the blue light LB, the green light LG, and the red light LR. Although the sensitivity is higher than that of the sensor, the color separation properties regarding R, G, and B are inferior. Since the above-described color enhancement processing is preferably performed on an image signal obtained by an image sensor having high color separation with respect to R, G, and B, in this embodiment, color separation of the image sensor 34 that is a complementary color sensor is performed. A control method is used that makes it possible to equalize the color separation property of the primary color sensor.

  As illustrated in FIG. 7, the light source control unit 21 controls the drive unit 22 to cause the second light source 23b to start emitting green light LG from time t0. At this time, the imaging control unit 40 controls the imaging device 34 to sequentially perform a reset operation by the reset method from time t0. As a result, all the pixel rows are sequentially reset one pixel row at a time. Each pixel row is sequentially placed in a charge accumulation state (exposure state) when unnecessary charges of the pixel 50a are discarded by resetting.

Then, when the first exposure time _TE1 has elapsed from time t0, the imaging control unit 40 controls the imaging element 34 and reads out only odd pixel rows (Mg pixels and G pixels) by the thinning-out method. Let the action take place. Thereby, the digitized Mg pixel signal and G pixel signal are output from the image sensor 34. Of the Mg and G pixels, only the G pixel has sensitivity to the green light LG, and therefore only the G pixel signal is input to the DSP 42 described above.

  The light source control unit 21 controls the drive unit 22 to stop the light emission operation of the second light source 23b and end the emission of the green light LG at time t1 when the reading of the last pixel row N is performed. In addition, the light source control unit 21 ends the emission of the green light LG and causes the third light source 23c to start the emission of the blue light LB. Then, the light source control unit 21 stops the light emission operation of the third light source 23c at time t2 after a lapse of a certain time from time t1, ends the emission of blue light LB, and causes the first light source 23a to emit red light LR. To start.

Thereafter, the imaging control unit 40 controls the imaging element 34 at the time when the second exposure time _TE2 has elapsed from the time t1, and only the even-numbered pixel rows (Cy pixel and Ye pixel) by the thinning-out method. Read operation is performed. The light source controller 21 stops the light emission operation of the first light source 23a at the time t3 when the final pixel row N is read, and ends the emission of the red light LR. Thereby, the digitized Cy pixel signal and Ye pixel signal are output from the image sensor 34. Of the Cy pixel and the Ye pixel, the Cy pixel is sensitive to the blue light LB, and the Ye pixel is sensitive to the red light LR.

During the second exposure time _TE2 , the green light LG is not emitted, and the Cy pixel and the Ye pixel are not exposed with the green light LG. Therefore, the Cy pixel signal can be regarded as a blue (B) pixel signal in the primary color sensor, and the Ye pixel signal can be regarded as a red (R) pixel signal in the primary color sensor. Therefore, the Cy pixel signal and the Ye pixel signal are input to the DSP 42 as the B pixel signal and the R pixel signal. The DSP 42 performs various signal processing using the G pixel signal, B pixel signal, and R pixel signal input from the image sensor 34 as the above-described image signals.

  Thus, the light receiving period for the green light LG of the image sensor 34 is a period from time t0 to time t1, and coincides with the light emission period of the green light LG by the light source device 14. The light receiving period of the image sensor 34 for the blue light LB and the red light LR is a period from time t1 to time t3, and coincides with the light emission period of the blue light LB and red light LR by the light source device 14.

The light source control unit 21, the emission intensity _{I B} of the blue light LB larger than the emission intensity _{I G} of the green light LG, and the emission intensity _{I G} of the green light LG larger than the emission intensity _{I R} of the red light LR (Ie, I _B > I _G > I _R ). This is because, as shown in FIG. 8, the mucous membrane of the large intestine among the objects to be observed has a characteristic that the reflectance is smaller as the wavelength is shorter, while the important information obtained from the surface of the mucosa is shorter This is because the return light of light is dominant. The mucous membrane surface layer includes a fine structure such as a capillary vessel containing hemoglobin that strongly absorbs blue light LB, which is a short wave light. Therefore, in order to generate an image including this fine structure, I _B > I _G > It is the relationship of I _R.

  Further, the light source control unit 21 makes the emission time of the red light LR longer than the emission time of the blue light LB and the emission time of the green light LG. This is because the blue light LB and the green light LG among the return light from the observation target include a lot of image information (high frequency components) regarding the structure of the surface layer portion and the middle layer portion of the mucous membrane. This is because the red light LR contains a lot of image information (low frequency component) related to the deep layer of the mucous membrane and is less affected by the blur. In this way, the light emission time of the red light LR is made longer than the light emission times of the blue light LB and the green light LG, and the exposure amount of the image sensor 34 for the red light LR is increased, thereby increasing the sensitivity to the red light LR ( (Improvement of S / N). Thereby, the image quality of the R image is improved, and a high-quality observation image is obtained by the color enhancement processing in the observation image generation unit 44.

  In addition, the G pixel signal, the B pixel signal, and the R pixel signal are directly acquired from the image pickup device 34 that is a complementary color sensor without performing signal processing by the above-described light emission and image pickup method. Can be equivalent to the color separation of the primary color sensor. Thereby, the image quality of the image obtained by the color enhancement process is improved.

  The operation of the endoscope system 10 configured as described above will be described. First, the insertion portion 12a of the endoscope 12 is inserted into the body cavity by the operator. When the operation panel or the like of the processor device 16 is operated to give an instruction to start photographing, the green light LG starts to be emitted from the second light source 23b. The image sensor 34 starts to emit the green light LG, and all the pixel rows are sequentially reset one by one by a sequential reset method.

Then, after the first exposure time _TE1 has elapsed from the light emission start time t0, the image sensor 34 performs a read operation for only odd-numbered pixel rows by a thinning-out method, and digitized Mg pixel signal and G pixel signal. Is output. Of the Mg pixel signal and G pixel signal, the G pixel signal is input to the DSP 42 of the processor device 16. Further, the second light source 23b stops the emission of the green light LG when the reading operation of the image sensor 34 is completed.

  Then, the third light source 23c starts to emit blue light LB from the emission stop time t1 of the green light LG, and the image sensor 34 sequentially resets all the pixel rows one pixel row at a time by the reset method. Thereafter, at time t2 after a certain time has elapsed from time t1, the second light source 23b stops emitting blue light LB, and the first light source 23a starts emitting red light LR.

After the second exposure time _TE2 has elapsed from time t1, the imaging device 34 performs a read operation for only even-numbered pixel rows by a thinning-out method, and outputs digitized Cy pixel signals and Ye pixel signals. The Cy pixel signal and the Ye pixel signal are input to the DSP 42 of the processor device 16 as a B pixel signal and an R pixel signal. Further, the first light source 23a stops the emission of the red light LR when the reading operation of the image sensor 34 is completed.

  Thereafter, the DSP 42 performs various signal processing on the image signal including the B pixel signal, the G pixel signal, and the R pixel signal input from the image sensor 34. The image signal that has been subjected to signal processing by the DSP 42 is subjected to noise removal processing by the noise removal unit 43 and then input to the observation image generation unit 44. The observation image generation unit 44 generates an observation image based on the input image signal. This observation image is displayed on the monitor 18 via the video signal generation unit 45. The above operation is repeated periodically until an imaging start instruction is given by the operator, and the observation images displayed on the monitor 18 are sequentially updated.

  In the above embodiment, as shown in FIG. 7, the blue light LB is emitted before the red light LR. However, the light emission order of the blue light LB and the red light LR may be reversed. In the above embodiment, the green light LG is emitted before the blue light LB and the red light LR. However, even if the light emission order of the green light LG, the blue light LB, and the red light LR is reversed. good.

  In the above embodiment, as shown in FIG. 7, the emission time of the blue light LB and the emission time of the green light LG are the same, but the emission time of the green light LG is longer than the emission time of the blue light LB. You may do it.

  In the above embodiment, as shown in FIG. 7, the reset method is sequentially used as the reset method, but instead, a collective reset method may be used as shown in FIG. In this case, the imaging control unit 40 resets all the pixel rows of the imaging element 34 by the batch reset method at time t0 when the second light source 23b starts to emit the green light LG.

The light source controller 21 causes the second light source 23b to stop emitting the green light LG at time t1 when the first exposure time _TE1 has elapsed from time t0. At this time, the imaging control unit 40 causes only the odd-numbered pixel rows to be read out by the thinning-out method from time t1. Then, the imaging control unit 40 resets all the pixel rows of the imaging device 34 by a batch reset method at time t2 when the read operation of the odd-numbered pixel rows ends. Further, the light source control unit 21 causes the third light source 23c to start emitting the blue light LB from time t2.

Thereafter, the light source control unit 21 causes the third light source 23c to stop emitting the blue light LB and causes the first light source 23a to start emitting the red light LR at a time t3 after a certain time has elapsed from the time t2. Then, the light source control unit 21 causes the first light source 23a to stop emitting the red light LR at time t4 when the second exposure time _TE2 has elapsed. At this time, the imaging control unit 40 causes the even-numbered pixel row to be read out by the partial reading method from time t4. Other operations are the same as those in the above embodiment.

  In this way, in the method shown in FIG. 9, all pixel rows start to receive light (charge accumulation) by collective reset. Therefore, the light reception start timing is the same, and reading is performed after each light emission is stopped. The light reception end timings of all the pixel rows are the same. Therefore, the method shown in FIG. 9 is a so-called global shutter method, and the light receiving period is the same for all the pixel rows.

On the other hand, the method shown in FIG. 7 is a rolling shutter method, and the light receiving period is different for each pixel row. Since the system shown in FIG. 9 is a global shutter system, the light receiving period for each color of all pixel rows is the same (so-called synchronism is obtained). In addition, the global shutter system shown in FIG. 9 has the advantage that the first and second exposure times T _E1 and T _E2 are longer and the exposure amount is increased than the rolling shutter system, in addition to the synchronism. There are advantages.

[Second Embodiment]
In the first embodiment, the thinning-out method is used as the signal reading method of the image sensor 34. However, in the second embodiment, the all-pixel reading method is used instead of the thinning-out method.

  In the second embodiment, as illustrated in FIG. 10, the imaging control unit 40 causes all pixel rows (even-numbered pixels and odd-numbered pixels) to be input to the image sensor 34 by the all-pixel readout method in accordance with the end of light emission of the green light LG. Row) are sequentially read out one pixel row at a time. Thereby, the digitized G pixel signal, Mg pixel signal, Cy pixel signal, and Ye pixel signal are output from the image sensor 34. Of the G pixel, Mg pixel, Cy pixel, and Ye pixel, the G pixel, Cy pixel, and Ye pixel other than the Mg pixel are sensitive to the green light LG. For this reason, the Cy pixel signal and the Ye pixel signal have signal components equivalent to the G pixel signal, and can be regarded as the G pixel signal. In addition to the G pixel signal from the G pixel, the Cy pixel signal and the Ye pixel signal are input to the DSP 42 as the G pixel signal.

  Similarly, the imaging control unit 40 causes the imaging device 34 to sequentially read out all the pixel rows one by one in accordance with the all-pixel reading method in accordance with the end of the emission of the red light LR. Thereby, the digitized G pixel signal, Mg pixel signal, Cy pixel signal, and Ye pixel signal are output from the image sensor 34. Of the G pixel, Mg pixel, Cy pixel, and Ye pixel, the Cy pixel has sensitivity to the blue light LB, the Ye pixel has sensitivity to the red light LR, and the Mg pixel has blue light LB and red color. Sensitive to light LR. Therefore, as in the first embodiment, the Cy pixel signal can be regarded as a B pixel signal, and the Ye pixel signal can be regarded as an R pixel signal. The Mg pixel signal is a signal obtained by adding the B pixel signal and the R pixel signal. The DSP 42 receives a Cy pixel signal and a Ye pixel signal as a B pixel signal and an R pixel signal, and an Mg pixel signal.

  In the second embodiment, the DSP 42 generates an R pixel signal by subtracting the Cy pixel signal (B pixel signal) from the Mg pixel signal as shown in the following equation (1). Further, the DSP 42 generates a B pixel signal by subtracting the Ye pixel signal (R pixel signal) from the Mg pixel signal as shown in the following equation (2). At this time, as the Cy pixel signal and the Ye pixel signal, signals of the Cy pixel and the Ye pixel adjacent to the Mg pixel from which the Mg pixel signal is acquired are used.

R = Mg-Cy (1)
B = Mg-Ye (2)

  Other processes and operations of the second embodiment are the same as those of the first embodiment. In the second embodiment, the G pixel signal, the Mg pixel signal, the Cy pixel signal, and the Ye pixel are both read out after receiving the green light LG and read out after receiving the blue light LB and the red light LR. Since three pixel signals among the signals are used for image generation, the resolution can be improved while maintaining the same color separation as in the first embodiment. In the second embodiment, the same modifications as in the first embodiment are possible.

  In the second embodiment, as shown in FIG. 4, Cy filters 60a and Ye filters 60c are alternately arranged in even pixel rows, and Mg filters 60b and G filters 60d are alternately arranged in odd pixel rows. However, instead of the color filter array 60, other color filters in which the arrangement order of the filters is changed may be used.

  For example, as shown in FIG. 11, a color filter array 61 in which Mg filters 60b and G filters 60d are alternately arranged in even pixel rows, and Cy filters 60a and Ye filters 60c are alternately arranged in odd pixel rows. It may be used. Further, as shown in FIG. 12, a color filter array 62 in which Cy filters 60a and G filters 60d are alternately arranged in even-numbered pixel rows, and Mg filters 60b and Ye filters 60c are alternately arranged in odd-numbered pixel rows. It may be used. Further, as shown in FIG. 13, a color filter array 63 in which Mg filters 60b and Ye filters 60c are alternately arranged in even pixel rows and Cy filters 60a and G filters 60d are alternately arranged in odd pixel rows. It may be used.

  As described above, in the second embodiment, the even-numbered pixel row and the odd-numbered pixel row are read by the all-pixel reading method at the time of reading both after receiving the green light LG and after receiving the blue light LB and the red light LR. Therefore, the signal processing is the same as in the second embodiment regardless of which of the color filter arrays 61 to 63 is used instead of the color filter array 60.

  In contrast, as in the first embodiment, when only the odd-numbered pixel rows are read after receiving the green light LG and only the even-numbered pixel rows are read after receiving the blue light LB and the red light LR, as shown in FIG. It is preferable to use the color filter array 60. This is because the Cy pixels in the even pixel rows are sensitive to the blue light LB and the Ye pixels are sensitive to the red light LR with respect to the exposure of the blue light LB and the red light LR. This is because the light LR can be received with good color separation.

  If the color filter array 61 shown in FIG. 11 is used in the first embodiment, among the Mg pixels and G pixels arranged in the even-numbered pixel rows, the Mg pixels are the blue light LB and the red light LR. Since the G pixel has sensitivity to both, and the G pixel does not have sensitivity to either the blue light LB or the red light LR, the blue light LB and the red light LR cannot be received with good color separation.

  In the first embodiment, when the color filter array 62 shown in FIG. 12 is used, among the Cy pixels and G pixels arranged in the even pixel rows, the Cy pixels have sensitivity to the blue light LB. Since the G pixel has no sensitivity to both the blue light LB and the red light LR, the blue light LB and the red light LR cannot be received with good color separation.

  Further, in the first embodiment, when the color filter array 63 shown in FIG. 13 is used, among the Mg pixels and Ye pixels arranged in the even pixel rows, the Mg pixels are the blue light LB and the red light LR. Both have sensitivity, and the Ye pixel has sensitivity to red light LR. In this case, the B pixel signal can be generated by subtracting the Cy pixel signal from the Mg pixel signal as shown in the above equation (1). Further, in this case, both the Cy pixel and the G pixel arranged in the odd-numbered pixel row are sensitive to the green light LG, and thus have a high resolution with respect to the green light LG.

  In each of the above embodiments, the light source device 14 and the processor device 16 are configured separately, but the light source device and the processor device may be configured as a single device.

DESCRIPTION OF SYMBOLS 10 Endoscope system 14 Light source device 16 Processor apparatus 20 Light source part 21 Light source control part 25 Light guide 34 Image pick-up element 40 Imaging control part

Claims (8)

A light source unit for emitting first light, second light, and third light having different colors as illumination light, and
The second light, the first light, and the third light are individually emitted by controlling the light emission timing and light emission intensity of the illumination light, and the light emission time of the first light is changed to the first light. A light source control unit that is longer than the light emission time of two lights and the light emission time of the third light, and increases the light emission intensity in the order of the first light, the second light, and the third light;
A complementary color type image sensor that receives return light from the observation target irradiated with the illumination light; and
An imaging control unit that controls the imaging timing of the imaging device in accordance with the light emission timing, and individually receives the second light, the first light, and the third light;
Equipped with a,
The first light is red light, the second light is green light, the third light is blue light,
The imaging device includes a cyan pixel that receives the second light and the third light, a magenta pixel that receives the first light and the third light, and a yellow light that receives the first light and the second light. An endoscope system having a pixel and a green pixel that receives the second light . The imaging device, and an even pixel rows in which the cyan pixels and said yellow pixels are arranged alternately, of claim 1, wherein the magenta pixel and said green pixel has a odd-numbered pixel rows are alternately arranged Endoscopic system. The imaging control unit reads the odd pixel rows from the imaging element after receiving the second light, and reads the even pixel rows from the imaging element after receiving the first light and the third light. The endoscope system according to claim 2 , wherein the endoscope system is released. The signal processing part which produces | generates an observation image using the green pixel signal read from the said odd pixel row, and the cyan pixel signal and yellow pixel signal read from the said even pixel row of Claim 3 is provided. Endoscopic system. The imaging control unit reads all the pixels of the imaging element after receiving the second light, and reads all the pixels of the imaging element after receiving the first light and the third light. The endoscope system according to 1 or 2 . Green pixel signal, cyan pixel signal, and yellow pixel signal read from the image sensor after receiving the second light, and magenta read from the image sensor after receiving the first light and the third light The endoscope system according to claim 5, further comprising a signal processing unit that generates an observation image using the pixel signal, the cyan pixel signal, and the yellow pixel signal. The signal processing unit generates a red pixel signal by subtracting the cyan pixel signal from the magenta pixel signal read from the imaging element after receiving the first light and the third light, and generates the magenta pixel signal. The endoscope system according to claim 6, wherein a blue pixel signal is generated by subtracting the yellow pixel signal from an image. A light source unit that emits first light, second light, and third light of different colors as illumination light, and a complementary color type image sensor that receives return light from an observation target irradiated with the illumination light ,
The first light is red light, the second light is green light, the third light is blue light,
The imaging device includes a cyan pixel that receives the second light and the third light, a magenta pixel that receives the first light and the third light, and a yellow light that receives the first light and the second light. In an operation method of an endoscope system having a pixel and a green pixel that receives the second light ,
The second light, the first light, and the third light are individually emitted by controlling the light emission timing and light emission intensity of the illumination light, and the light emission time of the first light is changed to the first light. It is longer than the emission time of two lights and the emission time of the third light, and the emission intensity is increased in the order of the first light, the second light, and the third light,
An operation method of an endoscope system that controls the imaging device to individually receive the second light, the first light, and the third light.

Images