JP2006341075A - Signal processor for biological observation system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prepare structural information on living tissues at a desired depth, based on a spectral image acquired by signal processing, to image information of a color tone suitable for observations. <P>SOLUTION: A matrix operation unit 436 has its outputs which are individually connected with integration units 438a to 438c and are integrated. Individual spectral image signals ΣF1 to ΣF3 are subjected to color conversions at a color adjusting unit 440 so that spectral color channel image signals Rch, Gch, Bch are produced from the spectral image signals ΣF1 to ΣF3. The images of the spectral color channel image signals Rch, Gch, Bch are sent to a display monitor 106 through a switching unit 439. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a signal processing apparatus for a biological observation apparatus that displays a spectral image on a monitor by signal processing using a color image signal obtained by imaging a living body.

  2. Description of the Related Art Conventionally, endoscope apparatuses that irradiate illumination light and obtain an endoscopic image in a body cavity have been widely used. In this type of endoscope apparatus, an electronic endoscope having an image pickup unit that guides illumination light from a light source device into a body cavity using a light guide or the like and picks up an image of a subject using the return light is used. An image signal from the imaging means is signal-processed to display an endoscopic image on an observation monitor and observe an observation site such as an affected area.

  When performing normal biological tissue observation in an endoscopic device, one method is to emit white light in the visible light region with a light source device, and use, for example, a rotation filter such as RGB to subject the surface sequential light to the subject. A color image is obtained by irradiating and image-processing the return light by the frame sequential light with a video processor. When performing normal biological tissue observation in an endoscope apparatus, as another method, a color chip is arranged in front of the imaging surface of the imaging means of the endoscope, and white light in the visible light region is emitted by the light source device. Is emitted, and the return light by the white light is imaged by separating each color component by a color chip and image processing is performed by a video processor to obtain a color image.

  Since living tissue has different light absorption characteristics and scattering characteristics depending on the wavelength of the irradiated light, for example, in Japanese Patent Application Laid-Open No. 2002-95635, illumination light in the visible light region has a narrow band of discrete spectral characteristics. A narrow-band optical endoscope apparatus that irradiates a living tissue with RGB sequential light and obtains tissue information of a desired deep portion of the living tissue has been proposed.

In Japanese Patent Laid-Open No. 2003-93336, narrowband light is obtained in which signal processing is performed on an image signal by illumination light in the visible light region to generate a discrete spectral image, and tissue information of a desired deep part of a living tissue is obtained. Endoscopic devices have been proposed.
JP 2002-95635 A JP 2003-93336 A

  However, for example, the apparatus described in Japanese Patent Application Laid-Open No. 2003-93336 does not require a filter for generating narrow-band RGB light by obtaining a spectral image by signal processing. Since the image is simply output to the monitor, the image displayed on the monitor may not be an image having a color tone suitable for observing tissue information in a desired deep part of the living tissue.

  The present invention has been made in view of the above circumstances, and a living body capable of adjusting tissue information of a desired deep part of a living tissue based on a spectral image obtained by signal processing into image information of a color tone suitable for observation. It aims at providing the signal processing apparatus for observation apparatuses.

  The signal processing apparatus for a biological observation apparatus of the present invention photoelectrically converts light reflected from the living body based on the illumination unit that irradiates light to the living body that is the subject and / or illumination light from the illumination unit, A signal processing apparatus for a biological observation apparatus that includes a signal processing control unit that controls an operation of an imaging unit that generates an imaging signal and outputs the imaging signal to a display device, and the optical wavelength narrow band from the imaging signal A spectral signal generation unit that generates a spectral signal corresponding to the image of the image by signal processing; and a color adjustment unit that adjusts a color tone for each of a plurality of bands forming the spectral signal when the spectral signal is output to the display device; The imaging signal is generated through a color separation unit.

  ADVANTAGE OF THE INVENTION According to this invention, there exists an effect that the tissue information of the desired deep part of the biological tissue based on the spectral image obtained by signal processing can be adjusted to the image information of the color tone suitable for observation.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIGS. 1 to 26, FIG. 35, FIG. 36, and FIG. 37 relate to the first embodiment of the present invention. FIG. 1 is a conceptual diagram showing a signal flow when a spectral image signal is created from a color image signal. 2 is a conceptual diagram showing the integral operation of the spectral image signal, FIG. 3 is an external view showing the external appearance of the electronic endoscope apparatus, FIG. 4 is a block diagram showing the configuration of the electronic endoscope apparatus of FIG. Is an external view showing the external appearance of the chopper of FIG. 4, FIG. 6 is a view showing an arrangement of color filters arranged on the imaging surface of the CCD of FIG. 3, and FIG. 7 is a view showing spectral sensitivity characteristics of the color filter of FIG. FIG. 8 is a configuration diagram showing the configuration of the matrix calculation unit of FIG. 4, FIG. 9 is a spectrum diagram showing the spectrum of the light source, and FIG. 10 is a spectrum diagram showing the reflection spectrum of the living body.

  11 is a diagram showing a layered structure of the biological tissue observed by the electronic endoscope apparatus of FIG. 4, and FIG. 12 is a diagram showing the arrival state of the illumination light from the electronic endoscope apparatus of FIG. 4 in the layer direction of the biological tissue. FIG. 13 is a diagram illustrating spectral characteristics of each band of white light, FIG. 14 is a first diagram illustrating each band image by white light in FIG. 13, and FIG. 15 is each band image by white light in FIG. FIG. 16 is a third diagram showing each band image by white light in FIG. 13, FIG. 17 is a diagram showing the spectral characteristics of the spectral image generated by the matrix operation unit in FIG. Is a first diagram showing each spectral image of FIG. 17, FIG. 19 is a second diagram showing each spectral image of FIG. 17, and FIG. 20 is a third diagram showing each spectral image of FIG.

  21 is a block diagram showing the configuration of the color adjustment unit in FIG. 4, FIG. 22 is a diagram for explaining the operation of the color adjustment unit in FIG. 21, and FIG. 23 is a block diagram showing a modification of the color adjustment unit in FIG. 24 is a diagram showing the spectral characteristics of the first modification of the spectral image of FIG. 17, FIG. 25 is a diagram showing the spectral characteristics of the second modification of the spectral image of FIG. 17, and FIG. 26 is the spectral characteristics of FIG. It is a figure which shows the spectral characteristic of the 3rd modification of an image.

  FIG. 35 is a diagram showing the appearance of the rigid endoscope, FIG. 36 is a diagram showing the appearance of the oral camera, and FIG. 37 is a diagram showing the appearance of the camera used in contact with the surface of the living body.

  In an electronic endoscope apparatus as a living body observation apparatus according to an embodiment of the present invention, an imaging unit emits light from a light source for illumination to a living body that is a subject and is reflected from the living body based on the irradiated light. A solid-state image sensor receives light and performs photoelectric conversion to generate an image signal that is a color image signal, and a spectral image signal that is a spectral signal corresponding to an optical wavelength narrow band image is generated from the image signal by signal processing. It is supposed to be.

  Hereinafter, before describing Example 1 according to the present invention, a matrix calculation method as a basis of the present invention will be described. Here, the matrix is a predetermined coefficient used when generating a spectral image signal as a spectral signal from a color image signal acquired to generate a color image (hereinafter referred to as a normal image). Further, following the description of this matrix, a correction method for obtaining a more accurate spectral image signal and an S / N improvement method for improving the S / N ratio of the generated spectral image signal will be described. The correction method and the S / N improvement method may be used as necessary. In the following, vectors and matrices (matrix) are indicated by bold letters or “” (for example, the matrix A is indicated as “A bold letters of A” or “A” ”), and the others are indicated without character modification.

(Matrix calculation method)
FIG. 1 shows a color image signal (here, R, G, and B for simplicity of explanation, but in a complementary color solid-state imaging device as in the embodiments described later, G, Cy, Mg, FIG. 4 is a conceptual diagram showing a signal flow when generating a spectral image signal corresponding to an image corresponding to an image in a narrower optical wavelength band.

  First, the electronic endoscope apparatus converts color sensitivity characteristics as spectral sensitivity characteristics of the R, G, and B imaging units into numerical data. Here, the color sensitivity characteristics of R, G, and B are output characteristics with respect to wavelengths obtained when a white object is imaged using a white light source.

  The color sensitivity characteristics of R, G, and B are shown on the right side of each image data as a simplified graph. Further, the color sensitivity characteristics of R, G, and B at this time are assumed to be n-dimensional column vectors “R”, “G”, and “B”, respectively.

  Next, the electronic endoscope apparatus uses spectral signals to be extracted, for example, narrow band Pandpass filters for spectral images as basic spectral characteristics of three spectral signals F1, F2, and F3 (the electronic endoscope apparatus uses foresight information). I know the characteristics of a filter that can extract the structure efficiently, which has passbands of about 590 nm to about 610 nm, about 530 nm to about 550 nm, and about 400 m to about 430 nm, respectively. )) Is converted into numerical data.

となるマトリックスの要素を求めればよい。 Here, “substantially” is a concept including about 10 nm in wavelength. The filter characteristics at this time are n-dimensional column vectors “F1”, “F2”, and “F3”, respectively. Based on the obtained numerical data, an optimum coefficient set that approximates the following relationship is obtained. That is,
[Equation 1]

What is necessary is just to obtain the elements of the matrix.

となる。従って、（１）式に示した命題は、以下の関係を満足するマトリックス「Ａ」を求めるに等しい。 The solution of the above optimization proposition is given mathematically as follows: Performing principal component analysis or orthogonal expansion (or orthogonal transformation), “C” for the matrix representing the color sensitivity characteristics of R, G, and B, “F” for the spectral characteristics of the narrowband Pandpass filter to be extracted If the coefficient matrix to be calculated is “A”,
[Equation 2]

It becomes. Therefore, the proposition shown in the equation (1) is equivalent to obtaining a matrix “A” that satisfies the following relationship.

ここで、分光特性を表すスペクトルデータとしての点列数nとしては、n＞３であるので、（３）式は１次元連立方程式ではなく、線形最小二乗法の解として与えられる。即ち、（３）式から擬似逆行列を解けばよい。マトリックス「Ｃ」の転置行列を「tＣ」とすれば、（３）式は
［数４］

となる。「tＣＣ」はn×nの正方行列であるので、（４）式はマトリックス「Ａ」についての連立方程式と見ることができ、その解は、
［数５］

与えられる。 [Equation 3]

Here, since the number of point sequences n as spectral data representing spectral characteristics is n> 3, equation (3) is not a one-dimensional simultaneous equation but is given as a solution of the linear least square method. That is, the pseudo inverse matrix may be solved from the equation (3). Assuming that the transposed matrix of the matrix “C” is “tC”, the equation (3) is expressed by [Equation 4].

It becomes. Since “tCC” is an n × n square matrix, equation (4) can be viewed as a simultaneous equation for the matrix “A”, and its solution is
[Equation 5]

Given.

  For the matrix “A” obtained by the equation (5), the electronic endoscope apparatus performs the conversion of the left side of the equation (3) to obtain the characteristics of the narrow band Pandpass filters F1, F2, and F3 to be extracted. Can be approximated. The above is the description of the matrix calculation method that is the basis of the present invention.

  Using the matrix calculated in this manner, a matrix calculation unit 436 described later generates a spectral image signal from the normal color image signal.

(Correction method)
Next, a correction method for obtaining a more accurate spectral image signal will be described.

  In the above description of the matrix calculation method, the light beam received by a solid-state imaging device such as a CCD is exactly white light (all wavelength intensities are the same in the visible range), and is applied accurately. . That is, the approximation is optimal when the RGB outputs are the same.

  However, under actual endoscopic observation, the light beam to be illuminated (the light beam from the light source) is not completely white light, and the reflection spectrum of the living body is not uniform, so the light beam received by the solid-state imaging device is not white light (color Are not the same).

  Therefore, in actual processing, in order to solve the proposition shown in the expression (3) more accurately, in addition to the RGB color sensitivity characteristics as the spectral sensitivity characteristics of the imaging unit, the illumination light as the spectral sensitivity characteristics of the illumination unit It is desirable to consider the spectral characteristics of the living body and the reflection characteristics of the living body as a set of spectral characteristic data of the subject.

  Here, color sensitivity characteristics (spectral sensitivity characteristics of the imaging unit) are R (λ), G (λ), and B (λ), respectively, and an example of spectral characteristics of illumination light (spectral sensitivity characteristics of the illumination unit) S ( λ), an example of the reflection characteristics of the living body (a set of spectral characteristic data of the subject) is H (λ). Note that the spectral characteristics of the illumination light and the reflection characteristics of the living body do not necessarily have to be the characteristics of the inspection apparatus and the subject, and may be general characteristics acquired in advance, for example.

Using these coefficients, the correction coefficients kR, kG, and kB are
[Equation 6]
kR = (∫S (λ) × H (λ) × R (λ) dλ) −1
kG = (∫S (λ) × H (λ) × G (λ) dλ) −1
kB = (∫S (λ) × H (λ) × B (λ) dλ) −1 (6)
Given in. When the sensitivity correction matrix is “K”, it is given as follows.

従って、係数マトリックス「Ａ」については、（５）式に（７）式の補正を加えて、以下のようになる。 [Equation 7]

Accordingly, the coefficient matrix “A” is as follows after adding the correction of the equation (7) to the equation (5).

また、実際に最適化を行う場合は、目標とするフィルタの分光感度特性（図１中のＦ1・Ｆ2・Ｆ3：基本分光特性）が負のときは画像表示上では０となる（つまりフィルタの分光感度特性のうち正の感度を有する部分のみ使用される）ことを利用し、最適化された感度分布の一部が負になることも許容されることが付加される。電子内視鏡装置は、ブロードな分光感度特性から狭帯域な分光感度特性を生成するためには、図１に示すように目標とするＦ1・Ｆ2・Ｆ3の特性に、負の感度特性を付加することで、感度を有する帯域を近似した成分を生成することができる。 [Equation 8]

When the optimization is actually performed, when the spectral sensitivity characteristics (F1, F2, F3: basic spectral characteristics in FIG. 1) of the target filter are negative, it becomes 0 on the image display (that is, the filter It is added that it is allowed that a part of the optimized sensitivity distribution becomes negative by utilizing only the part having the positive sensitivity among the spectral sensitivity characteristics. In order to generate a narrow-band spectral sensitivity characteristic from a broad spectral sensitivity characteristic, the electronic endoscope device adds a negative sensitivity characteristic to the target characteristics of F1, F2, and F3 as shown in FIG. By doing so, it is possible to generate a component approximating a sensitive band.

(S / N improvement method)
Next, a method for improving the S / N and accuracy of the generated spectral image signal will be described. This method for improving the S / N ratio solves the following problems by adding to the processing method described above.

(I) If any of the original signals (R, G, B) in the matrix calculation method described above is saturated, the characteristics of the filters F1 to F3 in the processing method are characteristics of a filter that can extract the structure efficiently (ideal (If the filters F1 to F3 are generated with only two signals in R, G and B, neither of the two original signals is saturated.) It is necessary).

(Ii) When a color image signal is converted into a spectral image signal, a narrowband filter is generated from a wideband filter. Therefore, sensitivity deterioration occurs, and the component of the generated spectral image signal is also reduced. N ratio is not good.

  As shown in FIG. 2, the method of improving the S / N ratio is that illumination light is irradiated several times (for example, n times, n times) in one field (one frame) of a normal image (general color image). (The irradiation intensity may be changed at each time. It is indicated by I0 to In in Fig. 2. This is realized only by controlling the illumination light. It is possible.)

  Thereby, the electronic endoscope apparatus can reduce the irradiation intensity of one time, and can suppress that each of the RGB signals is saturated. Further, the image signal divided into several times is added for n sheets in the subsequent stage. Thereby, the electronic endoscope apparatus can increase the signal component and improve the S / N ratio. In FIG. 2, the integration units 438a to 438c function as image quality adjustment units that improve the S / N ratio.

  The above is the matrix calculation method that is the basis of the present invention, the correction method for obtaining an accurate spectral image signal that can be performed together with this, and the method for improving the S / N ratio of the generated spectral image signal. It is an explanation.

  Here, a modified example of the above-described matrix calculation method will be described.

(Modification of matrix calculation method)
Assume that the color image signals are R, G, and B, and the spectral image signals to be estimated are F1, F2, and F3. Strictly speaking, the color image signals R, G, and B are also functions of the positions x and y on the image, and should be expressed as R (x, y), for example, but are omitted here.

  The goal is to estimate a 3 × 3 matrix “A” that calculates F1, F2, F3 from R, G, B. If “A” is estimated, F1, F2, and F3 can be calculated from R, G, and B by the following equation (9).

ここで、以下のデータの表記を定義する。 [Equation 9]

Here, the following data notation is defined.

Spectral characteristics of subject: H (λ), “H” = (H (λ2), H (λ2),..., H (λn)) t
λ is a wavelength, and t represents transposition in matrix calculation. Similarly,
Spectral characteristics of illumination light: S (λ), “S” = (S (λ2), S (λ2),..., S (λn)) t
Spectral sensitivity characteristics of CCD: J (λ), “J” = (J (λ2), J (λ2),..., J (λn)) t
Spectral characteristics of filters for color separation: For primary colors
R (λ), “R” = (R (λ2), R (λ2),..., R (λn)) t
G (λ), “G” = (G (λ2), G (λ2),..., G (λn)) t
B (λ), “B” = (B (λ2), B (λ2),..., B (λn)) t
“R”, “G”, and “B” are combined into a matrix “C” as shown in the equation (10).

画像信号Ｒ，Ｇ，Ｂ、分光信号Ｆ1，Ｆ2，Ｆ3を行列で以下のように表記する。 [Equation 10]

The image signals R, G, B and the spectral signals F1, F2, F3 are expressed in matrix as follows.

画像信号「Ｐ」は次式で計算される。 [Equation 11]

The image signal “P” is calculated by the following equation.

いま、「Ｑ」を得るための色分解フィルタを「Ｆ」とすると、（１２）式同様に
［数１３］

ここで、重要な第１の仮定として、いま、被検体の分光反射率が基本的な複数の（ここでは３つの）分光特性の線形和で近似して表現できると仮定すると、「Ｈ」は以下のように表記できる。 [Equation 12]

Assuming that the color separation filter for obtaining “Q” is “F”, [Formula 13] as in the equation (12).

Here, as an important first assumption, assuming that the spectral reflectance of the subject can be approximated by a linear sum of a plurality of (three in this case) spectral characteristics, “H” is It can be expressed as follows.

ここで、「Ｄ」は３つの基本スペクトルＤ1（λ）、Ｄ2（λ）、Ｄ3（λ）を列ベクトルに持つ行列で、「Ｗ」は「Ｈ」に対するＤ1（λ）、Ｄ2（λ）、Ｄ3（λ）の寄与をあらわす重み係数である。被検体の色調がそれほど大きく変動しない場合には、この近似が成立することが知られている。 [Formula 14]

Here, “D” is a matrix having three basic spectra D1 (λ), D2 (λ), and D3 (λ) as column vectors, and “W” is D1 (λ) and D2 (λ) for “H”. , D3 (λ) is a weighting coefficient representing the contribution. It is known that this approximation is established when the color tone of the subject does not vary so much.

  Substituting equation (14) into equation (12) yields:

ここで、３×３の行列「Ｍ」は、行列「ＣＳＪＤ」の計算結果を１つにまとめた行列を示す。 [Equation 15]

Here, the 3 × 3 matrix “M” indicates a matrix in which the calculation results of the matrix “CSJD” are combined into one.

  Similarly, the following equation is obtained by substituting the equation (14) into the equation (13).

同じく、「Ｍ’」は、行列「ＦＳＪＤ」の計算結果を１つにまとめた行列を示す。 [Equation 16]

Similarly, “M ′” indicates a matrix in which the calculation results of the matrix “FSJD” are combined into one.

  Eventually, “W” is eliminated from the equations (15) and (16), and the following equation is obtained.

「Ｍ-1」は行列「Ｍ」の逆行列を示す。結局、「Ｍ’Ｍ-1」は３×３の行列となり、推定目標の行列「Ａ」となる。 [Equation 17]

“M−1” indicates an inverse matrix of the matrix “M”. Eventually, “M′M−1” is a 3 × 3 matrix, which is the estimation target matrix “A”.

色分解用のバンドパスが完全なバンドパスでなく、他の帯域にも感度を持つ場合も考慮して、この仮定が成立する場合、（１５）式、（１６）式における「Ｗ」を上記「Ｈ」と考えれば、結局（１７）式と同様な行列が推定できる。 Here, as an important second assumption, when color separation is performed using a bandpass filter, it is assumed that the spectral characteristic of the subject in the band can be approximated by one numerical value. That is,
[Equation 18]

In consideration of the case where the bandpass for color separation is not a complete bandpass and has sensitivity in other bands as well, if this assumption is satisfied, “W” in Expressions (15) and (16) is Considering “H”, a matrix similar to the equation (17) can be estimated after all.

  Next, a specific configuration of the electronic endoscope apparatus as the living body observation apparatus according to the first embodiment of the present invention will be described with reference to FIG. The other embodiments described below have the same configuration.

  As shown in FIG. 3, an electronic endoscope apparatus 100 as a living body observation apparatus includes an endoscope 101 as an observation unit, an endoscope apparatus body 105, and a display monitor 106 as a display apparatus or a display output apparatus. ing. In addition, an endoscope 101 as an observation unit is provided on an insertion portion 102 to be inserted into the body of a subject, a distal end portion 103 provided at the distal end of the insertion portion 102, and a side opposite to the distal end side of the insertion portion 102. It is mainly composed of an angle operation unit 104 for instructing a bending operation or the like of the distal end portion 103.

  The subject image acquired by the endoscope 101 configured as a flexible endoscope is subjected to predetermined signal processing in the endoscope apparatus main body 105, and the processed image is displayed on the display monitor 106.

  Next, the endoscope apparatus main body 105 will be described in detail with reference to FIG. FIG. 4 is a block diagram of the electronic endoscope apparatus 100.

  As shown in FIG. 4, the endoscope apparatus main body 105 mainly includes a light source section 41 as a lighting section, a control section 42, and a main body processing apparatus 43. The control unit 42 and the main body processing device 43 constitute a signal processing control unit that controls the operation of the CDD 21 as the light source unit 41 and / or the imaging unit and outputs an imaging signal to the display monitor 106 that is a display device. .

  In this embodiment, the endoscope apparatus main body 105, which is a single unit, is described as having a light source unit 41 and a main body processing device 43 that performs image processing. However, the light source unit 41 and the main body processing are described. The apparatus 43 may be configured to be removable as a unit different from the endoscope apparatus main body 105.

  The light source unit 41, which is an illumination unit, is connected to the control unit 42 and the endoscope 101, and emits white light (including a case where it is not complete white light) with a predetermined light amount based on a signal from the control unit 42. I do. The light source unit 41 includes a lamp 15 as a white light source, a chopper 16 for adjusting the amount of light, and a chopper driving unit 17 for driving the chopper 16.

  As shown in FIG. 5, the chopper 16 has a configuration in which a notch having a predetermined length in the circumferential direction is provided in a disk-like structure having a predetermined radius r0 with the point 17a as the center. The center point 17a is connected to a rotating shaft provided in the chopper driving unit 17. That is, the chopper 16 performs rotational movement around the center point 17a. In addition, a plurality of notches are provided for each predetermined radius. In this figure, the notch has a maximum length = 2πr0 × 2θ0 degrees / 360 degrees and a width = r0−ra between the radius r0 and the radius ra. Similarly, the maximum length = 2πra × 2θ1 degrees / 360 degrees between the radius ra and the radius rb, the width = ra−rb, and the maximum length = 2πrb × 2θ2 between the radius rb and the radius rc. Degree / 360 degrees and width = rb−rc (respective radii are r0> ra> rb> rc).

  In addition, the length and width of the notch in the chopper 16 are examples, and are not limited to the present embodiment.

  Further, the chopper 16 has a protrusion 160a extending in the radial direction substantially at the center of the notch. The control unit 42 switches the frame when the light is blocked by the projection 160a, thereby minimizing the interval between the light irradiated one frame before and after the frame, and blurring due to the movement of the subject. Minimize.

  Further, the chopper driving unit 17 is configured to be movable in the direction with respect to the lamp 15 as indicated by an arrow in FIG.

  That is, the control unit 42 can change the distance R between the rotation center 17a of the chopper 16 shown in FIG. 5 and the luminous flux from the lamp (shown by a dotted circle). For example, in the state shown in FIG. 5, since the distance R is quite small, the amount of illumination light is small. By increasing the distance R (the chopper driving unit 17 is moved away from the lamp 15), the cutout portion through which the light beam can pass becomes longer, so the irradiation time becomes longer, and the control unit 42 can increase the amount of illumination light. .

  As described above, in the electronic endoscope apparatus, there is a possibility that the newly generated spectral image may not be sufficient as the S / N, and one of the RGB signals necessary for generating the spectral image is saturated. In such a case, the correct calculation is not performed, and it is necessary to control the amount of illumination light. The light amount adjustment is performed by the chopper 16 and the chopper driving unit 17.

  The endoscope 101 connected to the light source unit 41 via the connector 11 includes an objective lens 19 and a solid-state imaging device 21 such as a CCD (hereinafter simply referred to as a CCD) at the distal end portion 103. The CCD in this embodiment is a single plate type (CCD used for a simultaneous electronic endoscope) and is a primary color type. FIG. 6 shows the arrangement of color filters arranged on the imaging surface of the CCD. The color filter arranged on the imaging surface of the CCD constitutes a color separation unit. FIG. 7 shows the spectral sensitivity characteristics of RGB in the color filter of FIG.

  As shown in FIG. 4, the insertion unit 102 transmits the light guide 14 that guides the light emitted from the light source unit 41 to the distal end portion 103 and the image of the subject obtained by the CCD to the main body processing device 43. A signal line for performing the treatment, a forceps channel 28 for performing the treatment, and the like. A forceps port 29 for inserting forceps into the forceps channel 28 is provided in the vicinity of the operation unit 104.

  In addition, the main body processing device 43 as a signal processing device for the living body observation device is connected to the endoscope 101 via the connector 11, similarly to the light source unit 41. The main body processing device 43 includes a CCD drive 431 for driving the CCD 21. The main body processing device 43 has a luminance signal processing system and a color signal processing system as signal circuit systems for obtaining a normal image.

  The luminance signal processing system includes a contour correction unit 432 that is connected to the CCD 21 and performs contour correction, and a luminance signal processing unit 434 that generates a luminance signal from data corrected by the contour correction unit 432. The color signal processing system is connected to the CCD 21, and samples and hold circuits (S / H circuits) 433a to 433c and S / H circuits 433a to 433c that sample the signals obtained by the CCD 21 and generate RGB signals. A color signal processing unit 435 is connected to the output and generates a color signal.

  Then, a normal image generating unit 437 that generates one normal image from the output of the luminance signal processing system and the output of the color signal processing system is provided, and the display monitor 106 via the switching unit 439 from the normal image generating unit 437. Y signal, RY signal, and BY signal are sent to.

  On the other hand, a matrix calculation unit 436 is provided as a signal circuit system for obtaining a spectral image, that is, the outputs (RGB signals) of the S / H circuits 433a to 433c are inputted and a predetermined matrix calculation is performed on the RGB signals. It has been. The matrix calculation unit 436 constitutes a spectral signal generation unit. Matrix calculation refers to a process of performing addition processing or the like between color image signals and multiplying the matrix obtained by the above-described matrix calculation method (or a modification thereof).

  In this embodiment, a method using electronic circuit processing (processing by hardware using an electronic circuit) will be described as the matrix calculation method. However, as in the embodiments described later, numerical data processing (program A method using software processing using Moreover, when implementing, it is also possible to combine these methods.

  FIG. 8 shows a circuit diagram of the matrix calculation unit 436. The RGB signals are input to the amplifiers 32a to 32c through the resistor groups 31a to 31c, respectively. Each resistor group has a plurality of resistors to which RGB signals are respectively connected, and the resistance value of each resistor is a value corresponding to the matrix coefficient. In other words, the gain of the RGB signal is changed by each resistor, and the amplifiers add (or subtract). The outputs of the amplifiers 32a to 32c are the outputs of the matrix operation unit 436. That is, the matrix calculation unit 436 performs so-called weighted addition processing. Note that the resistance value of each resistor used here may be variable.

  The outputs of the matrix operation unit 436 are connected to the integration units 438a to 438c, respectively, and after integration is performed, the color adjustment operation described later is performed on the spectral image signals ΣF1 to ΣF3 by the color adjustment unit 440. Thus, spectral color channel image signals Rch, Gch, and Bch are generated from the spectral image signals ΣF1 to ΣF3. The generated spectral color channel image signals Rch, Gch, and Bch are sent to the RGB color channels R (ch), G (ch), and B (ch) of the display monitor 106 via the switching unit 439. The configuration of the color adjustment unit 440 will be described later.

  Note that the switching unit 439 switches between a normal image and a spectral image, and can also switch between spectral images. That is, the operator selectively displays the monitor 106 from the normal image, the spectral color channel image by the color channel R (ch), the spectral color channel image by the color channel G (ch), and the spectral color channel image by the color channel B (ch). Can be displayed. Further, any two or more images may be displayed on the display monitor 106 at the same time. In particular, when a normal image and a spectral color channel image (hereinafter also referred to as a spectral channel image) can be displayed simultaneously, it is possible to easily compare the normal image and the spectral channel image that are generally observed. In addition, each feature (a feature of a normal image is easy to observe because the color degree is close to that of a normal naked eye. A feature of a spectral channel image can observe a predetermined blood vessel or the like that cannot be observed with a normal image). Above, it can be observed and is very useful diagnostically.

  Note that an endoscope 101 that is an endoscope configured as a flexible endoscope used in the present embodiment is an endoscope configured as a rigid endoscope such as an endoscope 101a illustrated in FIG. There may be.

  The endoscope 101a includes a rigid insertion portion 102a that is inserted into the body of a subject, and a television camera 103a that is configured to be detachable from the proximal end portion of the insertion portion 102a.

  The insertion portion 102 a has a cable 112 having a configuration that is detachable from the light source portion 41 of the endoscope apparatus main body 105. The insertion portion 102a and the cable 112 are provided with a light guide (not shown) that guides illumination light from the light source portion 41 to the distal end portion of the insertion portion 102a.

  Furthermore, the distal end portion of the insertion portion 102a has an objective optical system (not shown) for forming an image of the subject. The insertion portion 102a is provided on the proximal end side of the objective optical system, and has a relay lens (not shown) from the distal end portion to the proximal end portion.

  Since the insertion unit 102a has the above-described configuration, an image of the subject is formed on the front end surface of the relay lens by the objective optical system, and then transmitted through the relay lens group. The transmitted light of the subject image is focused on a CCD (not shown) of the television camera 103a provided on the rear end face side of the relay lens group. The CCD outputs a focused image of the subject as an imaging signal.

  The television camera 103 a includes a cable 111 having a configuration that is detachable from the main body processing device 43 of the endoscope apparatus main body 105. With such a configuration, the television camera 103 a outputs an imaging signal to the main body processing device 43 via the cable 111.

  Further, the endoscope 101 used in the present embodiment may be configured as an oral camera, such as a scope 201 shown in FIG.

  The scope 201 includes a light source such as an LED (not shown) that emits substantially the same illumination light as the light source unit 41, an objective optical system (not shown) that forms an image of a subject illuminated by the light source, and a connection between the objective optical system. A CCD (not shown) that is provided at the image position and outputs a captured image of the subject as an imaging signal, and a color filter (not shown) provided on the CCD are provided at the distal end portion. A cable 201a having a detachable configuration is provided at the base end.

  Further, the endoscope 101 used in the present embodiment may be configured as a camera that is used in contact with the surface of a living body, such as a scope 301 shown in FIG.

  The scope 301 includes a light source such as an LED (not shown) that emits substantially the same illumination light as the light source unit 41, an objective optical system (not shown) that forms an image of a subject illuminated by the light source, and a connection between the objective optical system. For the main body processing device 43, a contact unit 301 a provided with an unillustrated CCD provided at an image position and outputting a captured image of the subject as an imaging signal, and a color filter (not illustrated) provided in the CCD. It has a detachable configuration, and has a cable 301 b that transmits an imaging signal from the contact portion 301 a to the main body processing device 43.

  Next, the operation of the electronic endoscope apparatus 100 according to the present embodiment will be described in detail with reference to FIG.

  In the following, the operation when observing a normal image will be described first, and the operation when observing a spectral image will be described later.

  First, the operation of the light source unit 41 will be described. Based on the control signal from the control unit 42, the chopper driving unit 17 is set to a predetermined position and rotates the chopper 16. The light beam from the lamp 15 passes through the notch portion of the chopper 16 and is a light guide 14 that is an optical fiber bundle provided in the connector 11 at the connection portion of the endoscope 101 and the light source portion 41 by a condenser lens. Condensed at the incident end of.

  The condensed light flux passes through the light guide 14 and is irradiated into the body of the subject from the illumination optical system provided at the distal end portion 103. The irradiated light beam is reflected in the subject, and signals are collected for each color filter shown in FIG.

  The collected signals are input in parallel to the luminance signal processing system and the color signal processing system. The luminance signal-based contour correction unit 432 adds the signals collected for each color filter for each pixel and inputs them. After contour correction, the signals are input to the luminance signal processing unit 434. In the luminance signal processing unit 434, a luminance signal is generated and input to the normal image generating unit 437.

  On the other hand, signals collected by the CCD 21 are input to the S / H circuits 433a to 433c for each color filter, and R, G, and B signals are respectively generated. Further, R, G, and B signals are generated by a color signal processing unit 435, and a normal image generation unit 437 generates a Y signal, an RY signal, and a BY signal from the luminance signal and the color signal. Then, the normal image of the subject is displayed on the display monitor 106 via the switching unit 439.

  Next, the operation when observing a spectral image will be described. In addition, what performs the same operation | movement as observation of a normal image is abbreviate | omitted here.

  The operator gives an instruction to observe the spectral image from the normal image by operating a keyboard or the like provided on the endoscope apparatus main body 105 or a switch or the like provided on the operation unit 104 of the endoscope 101. At this time, the control unit 42 changes the control state of the light source unit 41 and the main body processing device 43.

  Specifically, the amount of light emitted from the light source unit 41 is changed as necessary. As described above, since it is not desirable that the output from the CCD 21 is saturated, the amount of illumination light is made smaller when observing a spectral image than when observing a normal image. Further, the control unit 42 can control the light amount so that the output signal from the CCD is not saturated, and can change the illumination light amount in a range where the output signal is not saturated.

  Further, as a control change to the main body processing device 43 by the control unit 42, the signal output from the switching unit 439 is switched from the output of the normal image generation unit 437 to the output of the color adjustment unit 440. Further, the outputs of the S / H circuits 433a to 433c are amplified and added by the matrix calculation unit 436, and are output to the integration units 438a to 438c according to the respective bands, and are output to the color adjustment unit 440 after the integration processing. Is done. Even when the amount of illumination light is reduced by the chopper 16, the signal intensity can be increased and the S / N can be improved as shown in FIG. 2 by storing and integrating in the integrating units 438 a to 438 c. Spectral images can be obtained.

  Hereinafter, specific matrix processing of the matrix calculation unit 436 in the present embodiment will be described. In this embodiment, from the spectral sensitivity characteristics of the RGB color filter indicated by the solid line in FIG. 7, the ideal narrow-band bandpass filters F1 to F3 (in this case, the respective transmission wavelength regions are indicated by F1). : 590 nm to 620 nm, F2: 520 nm to 560 nm, F3: 400 nm to 440 nm) When trying to create a bandpass filter (hereinafter referred to as a pseudo bandpass filter), the above formula (1) (5) Depending on what is shown in the equation, the following matrix is optimal.

更に、（６）式及び（７）式に示した内容により補正を行うと、以下の補正係数を得る。 [Equation 19]

Further, when correction is performed according to the contents shown in the equations (6) and (7), the following correction coefficients are obtained.

なお、（６）式に示す光源のスペクトルＳ（λ）は図９に示すものであり、（７）式に示す注目する生体の反射スペクトルＨ（λ）は図１０に示すものである、という先見情報を使用している。 [Equation 20]

Note that the spectrum S (λ) of the light source shown in the equation (6) is as shown in FIG. 9, and the reflection spectrum H (λ) of the target organism shown in the equation (7) is as shown in FIG. Use foresight information.

  Therefore, the processing performed by the matrix calculation unit 436 is mathematically equivalent to the following matrix calculation.

このマトリックス演算を行うことにより擬似フィルタ特性（図７にはフィルタ擬似Ｆ1乃至Ｆ3の特性として示されている）が得られる。即ち、上述のマトリックス処理は、カラー画像信号に、上述のようにして予め生成された擬似バンドパスフィルタ（マトリックス）を用いて、分光画像信号を作成するものである。 [Equation 21]

By performing this matrix operation, pseudo filter characteristics (shown as characteristics of filter pseudo F1 to F3 in FIG. 7) are obtained. That is, the matrix processing described above creates a spectral image signal using a pseudo bandpass filter (matrix) generated in advance as described above for a color image signal.

  An example of an endoscopic image generated using this pseudo filter characteristic is shown below.

  As shown in FIG. 11, the body cavity tissue 45 often has an absorber distribution structure such as blood vessels that differ in the depth direction. A large number of capillaries 46 are mainly distributed near the surface of the mucous membrane, and a blood vessel 47 that is thicker than the capillaries is distributed in addition to the capillaries in the middle layer deeper than this layer, and a thicker blood vessel 48 is further distributed in the deep layer. become.

  On the other hand, the depth of light in the body cavity tissue 45 in the depth direction depends on the wavelength of the light, and the illumination light including the visible region has a blue (B) color as shown in FIG. In the case of light with a short wavelength, the light only reaches the surface layer due to the absorption and scattering characteristics in living tissue, and is absorbed and scattered in the depth range up to that point, and light emitted from the surface is observed. . In the case of green (G) light, which has a wavelength longer than that of blue (B) light, it reaches deeper than the range where blue (B) light deepens, absorbs and scatters within that range, and exits from the surface. Light is observed. Still further, red (R) light having a wavelength longer than that of green (G) light reaches a deeper range.

The RGB light during normal observation of the body cavity tissue 45, as shown in FIG.
(1) Band images having shallow layer and middle layer tissue information including a lot of tissue information in the shallow layer as shown in FIG.
(2) In addition, the image signal picked up by the CCD 21 with the G-band light is picked up with a shallow layer image including a lot of tissue information in the middle layer and a band image having the middle layer tissue information as shown in FIG.
(3) Further, a band image having middle layer and deep layer tissue information including a lot of tissue information in the deep layer as shown in FIG.

  Then, the endoscope apparatus body 105 performs signal processing on these RGB imaging signals, so that it is possible to obtain an endoscopic image having a desired or natural color reproduction as an endoscopic image.

The matrix processing in the matrix calculation unit 436 is to create a spectral image signal by using the pseudo bandpass filter (matrix) generated in advance as described above for the color image signal. For example, spectral image signals F1 to F3 are obtained using pseudo bandpass filters F1 to F3 having discrete and narrow-band spectral characteristics capable of extracting desired deep tissue information as shown in FIG. As shown in FIG. 17, the pseudo bandpass filters F1 to F3 are not overlapped with each other.
(4) A band image having tissue information in a shallow layer as shown in FIG. 18 is captured in the spectral image signal F3 by the pseudo bandpass filter F3,
(5) A band image having tissue information in the middle layer as shown in FIG. 19 is captured in the spectral image signal F2 by the pseudo bandpass filter F2, and (6) the spectral image signal F1 by the pseudo bandpass filter F1 A band image having tissue information in the deep layer as shown in FIG. 20 is captured.

  Next, as an example of the simplest color conversion, the color adjustment unit 440 uses the spectral image signal F1 as a color channel R (ch) as an example of the simplest color conversion for the spectral image signals F1 to F3 thus obtained. F2 is assigned to the color channel G (ch), and the spectral image signal F3 is assigned to the color channel B (ch), and output to the display monitor 106 via the switching unit 439.

  As shown in FIG. 21, the color adjustment unit 440 includes a 3 × 3 matrix circuit 61, three sets of LUTs 62a, 62b, 62c, 63a, 63b, and 63c provided before and after the 3 × 3 matrix circuit 61, and an LUT 62a, The color conversion processing circuit 440a includes table data 62b, 62c, 63a, 63b, and 63c and a coefficient changing circuit 64 that changes coefficients of the 3 × 3 matrix circuit 61.

  The spectral image signals F1 to F3 input to the color conversion processing circuit 440a are subjected to inverse γ correction, nonlinear contrast conversion processing, and the like for each band data by the LUTs 62a, 62b, and 62c.

  Next, after color conversion is performed in the 3 × 3 matrix circuit 61, γ correction and appropriate gradation conversion processing are performed in the subsequent LUTs 63a, 63b, and 63c.

  The table data of these LUTs 62a, 62b, 62c, 63a, 63b, 63c and the coefficients of the 3 × 3 matrix circuit 61 can be changed by the coefficient changing circuit 64.

  The change by the coefficient changing circuit 64 is performed based on a control signal from a processing conversion switch (not shown) provided in the operation unit or the like of the endoscope 101.

  Upon receiving these control signals, the coefficient changing circuit 64 calls appropriate data from the coefficient data previously written in the color adjustment unit 440, and rewrites the current circuit coefficient with this data.

  Next, specific color conversion processing contents will be described. An example of the color conversion formula is shown in Formula (22).

この式（２２）による処理は、分光チャンネル画像信号Ｒch、Ｇch、Ｂchに分光画像信号Ｆ1乃至Ｆ3を波長の短い順に割り当てる色変換である。 [Equation 22]

The processing according to the equation (22) is color conversion that assigns the spectral image signals F1 to F3 to the spectral channel image signals Rch, Gch, and Bch in the order of shorter wavelengths.

  When observed with a color image of these color channels R (ch), G (ch), and B (ch), an image as shown in FIG. 22, for example, is obtained. A thick blood vessel is in a deep position, the spectral image signal F3 is reflected, and a color image as a predetermined target color is shown as a blue pattern. Since the spectral image signal F2 is strongly reflected in the vascular network near the middle layer, the color image as the predetermined target color is shown as a red pattern. Among the vascular network, those existing near the mucosal surface are expressed as a yellow pattern as a predetermined target color.

  In particular, this pattern change near the mucosal surface is important for early differential detection and diagnosis of lesions. However, the yellow pattern as the predetermined target color tends to have low contrast with the background mucosa and low visibility.

  Therefore, in order to reproduce the pattern near the mucosal surface more clearly, the conversion shown in Expression (23) is effective.

この式（２３）による処理は、分光画像信号Ｆ1をある一定の比率で分光画像信号Ｆ2に混合し生成されたデータを所定の目標色として新たに分光Ｇチャンネル画像信号Ｇchとする変換例であり、血管網などの吸収散乱体が深さ位置で異なることをより明確化することが可能となる。 [Equation 23]

The processing according to the equation (23) is a conversion example in which the spectral image signal F1 is mixed with the spectral image signal F2 at a certain ratio and the generated data is newly set as a spectral G channel image signal Gch as a predetermined target color. Further, it becomes possible to further clarify that the absorbing scatterers such as the blood vessel network are different in the depth position.

  Accordingly, by adjusting the matrix coefficient through the coefficient changing circuit 64, the user can adjust the display effect. As an operation, the matrix coefficient is set to a default value from the through operation in the image processing unit in conjunction with a mode change switch (not shown) provided in the operation unit of the endoscope 101.

The through operation here means a state in which the 3 × 3 matrix circuit 61 is mounted with a unit matrix, and the LUTs 62a, 62b, 62c, 63a, 63b, and 63c are mounted with non-conversion tables. The default value means that set values of, for example, ωG = 0.2 and ωB = 0.8 are given to the matrix coefficients ω _G and ω _B.

  Then, the user operates the operation unit or the like of the endoscope 101 and adjusts the coefficient so that ωG = 0.4, ωB = 0.6, and the like. A reverse γ correction table and a γ correction table are applied to the LUTs 62a, 62b, 62c, 63a, 63b, and 63c as necessary.

  The color conversion processing circuit 440a performs color conversion using a matrix computing unit including the 3 × 3 matrix circuit 61. However, the present invention is not limited to this, and the color conversion processing circuit may be configured by a numerical operation processor (CPU) or LUT.

  For example, in the above embodiment, the color conversion processing circuit 30a is shown with a configuration centered on the 3 × 3 matrix circuit 61. However, as shown in FIG. 23, the color conversion processing circuit 30a is a three-dimensional LUT 65 corresponding to each band. The same effect can be obtained by replacing with. In this case, the coefficient changing circuit 64 performs an operation of changing the contents of the table based on a control signal from a processing conversion switch (not shown) provided in the operation unit or the like of the endoscope 101.

  Note that the filter characteristics of the pseudo bandpass filters F1 to F3 are not limited to the visible light range. As a first modification of the pseudo bandpass filters F1 to F3, the filter characteristics are discrete spectrum as shown in FIG. It may be a narrow band of characteristics. The filter characteristic of the first modification is that normal observation is performed by setting F3 in the near ultraviolet region and F1 in the near infrared region in order to observe the irregularities on the surface of the living body and the absorber near the extreme deep layer. It is suitable for obtaining image information that cannot be obtained by.

  As a second modification of the pseudo bandpass filters F1 to F3, two pseudo bandpass filters F3a and F3b whose filter characteristics are close in the short wavelength region are used instead of the pseudo bandpass filter F2 as shown in FIG. It is also good. This is suitable for imaging a subtle difference in the scattering characteristic rather than the absorption characteristic by utilizing the fact that the wavelength band in the vicinity reaches only near the extreme surface layer of the living body. Medically, it is assumed to be used for identification diagnosis of diseases such as early cancer, which are accompanied by disorder of cell arrangement near the mucosal surface layer.

  Furthermore, as a third modification of the pseudo bandpass filters F1 to F3, two pseudo-bandwidth narrowband filter characteristics of two discrete spectral characteristics from which desired layer structure information can be extracted as shown in FIG. The band pass filters F2 and F3 may be generated by the matrix calculation unit 436.

  In the case of the pseudo bandpass filters F2 and F3 in FIG. 26, the color adjustment unit 440 performs spectral channel image signal Rch ← spectral image signal F2, spectral channel image signal Gch in colorization of an image when observing a narrowband spectral image. ← RGB 3 channel color image is generated as spectral image signal F3, spectral channel image signal Bch ← spectral image signal F3.

  That is, for the spectral image signal F2 and the spectral image signal F3, the color adjustment unit 440 generates RGB three-channel spectral color channel image signals (Rch, Gch, Bch) by the following equation (24).

例えば、ｈ11＝１、ｈ12＝０、ｈ21＝０、ｈ22＝１．２、ｈ31＝０、ｈ32＝０．８とする。 [Equation 24]

For example, h11 = 1, h12 = 0, h21 = 0, h22 = 1.2, h31 = 0, h32 = 0.8.

  For example, the spectral image F3 with basic spectral characteristics is an image whose central wavelength mainly corresponds to 415 nm, and the spectral image F2 with basic spectral characteristics is an image whose central wavelength mainly corresponds to 540 nm.

  Further, for example, the spectral image F3 having the basic spectral characteristic is an image whose central wavelength is mainly equivalent to 415 nm, the spectral image F2 having the basic spectral characteristic is an image mainly corresponding to the central wavelength is 540 nm, and the spectral image F1 having the basic spectral characteristic is Even if the center wavelength is calculated as an image corresponding mainly to 600 nm, the color adjusting unit 440 can use the F2 and F3 images without using the F1 image. In this case, a matrix operation of the following equation (24 ′) may be applied instead of the equation (24).

Rch = h11 × F1 + h12 × F2 + h13 × F3
Gch = h21 × F1 + h22 × F2 + h23 × F3
Bch = h31 × F1 + h32 × F2 + h33 × F3 (24 ′)
In the matrix calculation of the above equation (24 ′), the coefficients of h11, h13, h21, h22, h31, and h32 may be set to 0 and the other coefficients may be set to predetermined numerical values.

  As described above, according to the present embodiment, a pseudo narrowband filter is generated by using a color image signal for generating a normal electronic endoscopic image (normal image). A spectral image having tissue information of a desired deep portion such as a blood vessel pattern can be obtained without using an optical wavelength narrow-band bandpass filter, and the parameters of the color conversion processing circuit 440a of the color adjustment unit 440 are converted into spectral images. By setting it accordingly, it becomes possible to realize an expression method that makes use of the feature of depth-of-depth information when observing a narrow-band spectral image, and it is possible to effectively obtain tissue information of a desired deep part near the tissue surface of a living tissue. It can be visually recognized separately.

  Therefore, the color adjustment unit 440 performs signal conversion so that a channel including subject information to be output with the highest contrast among a plurality of spectral signals is reproduced as a luminance image on the display monitor 106.

In particular, in the color adjustment unit 440,
(1) In the case of a two-band spectral image, for example, when an image corresponding to 415 nm is assigned to the color channel G (ch) and B (ch), for example, an image corresponding to 540 nm is assigned to the color channel R (ch),
Or
(2) In the case of a three-band spectral image, for example, an image corresponding to 415 nm is used as the color channel B (ch), an image corresponding to 445 nm is used as the color channel G (ch), and an image corresponding to 500 nm is used as the color channel. When assigned to R (ch), the following image effect is obtained.

・ The epithelium or mucous membrane of the outermost layer of living tissue is reproduced with low saturation color, and the outermost capillaries are reproduced with low brightness, that is, dark lines, so that high visibility of the outermost capillaries is obtained. It is done.

At the same time, since the blood vessel at a deeper position than the capillary is rotated and reproduced in the blue direction in the hue direction, it is easier to distinguish from the outermost capillary.

  Further, according to the channel assignment method, in colonoscopy, residues and bile observed in a yellow tone under normal observation are observed in a red tone.

  FIG. 27 is a block diagram illustrating a configuration of the electronic endoscope apparatus according to the second embodiment of the present invention.

  Since the second embodiment is almost the same as the first embodiment, only different points will be described.

  This embodiment mainly differs from the first embodiment in the light source unit 41 that controls the amount of illumination light. In this embodiment, the amount of light emitted from the light source unit 41 is controlled not by the chopper but by the current control of the lamp 15. Specifically, a current control unit 18 is provided in the lamp 15 shown in FIG.

  As an operation of this embodiment, the control unit 42 controls the current control unit 18 to control the current flowing through the lamp 15 so that none of the RGB color image signals are saturated. Thereby, since the current used for light emission to the lamp 15 is controlled, the amount of light changes in accordance with the magnitude of the current.

  Since other operations are the same as those in the first embodiment, they are omitted here.

  According to the present embodiment, as in the first embodiment, a spectral image in which the blood vessel pattern is clearly displayed can be obtained. Further, the present embodiment has an advantage that the control method is simpler than the light amount control method using the chopper as in the first embodiment.

  FIG. 28 is a block diagram illustrating the configuration of the matrix calculation unit according to the third embodiment of the present invention.

  Since the third embodiment is almost the same as the first embodiment, only different points will be described, and the same components are denoted by the same reference numerals and description thereof will be omitted.

  This embodiment mainly differs from the first embodiment in the configuration of the matrix calculation unit 436. In the first embodiment, the matrix operation is performed by so-called hardware processing using an electronic circuit. However, in this embodiment shown in FIG. 28, this matrix operation is performed by numerical data processing (processing by software using a program).

  A specific configuration of the matrix calculation unit 436 in the present embodiment is shown in FIG. The matrix calculation unit 436 includes an image memory 50 that stores RGB color image signals. The coefficient register 51 stores each value of the matrix “A ′” shown in the equation (21) as numerical data.

  The coefficient register 51 and the image memory 50 are connected to the multipliers 53a to 53i, the multipliers 53a, 53d, and 53g are connected to the multiplier 54a, and the output of the multiplier 54a is connected to the integrating unit 438a in FIG. Is done. The multipliers 53b, 53e, and 53h are connected to the multiplier 54b, and the output thereof is connected to the integrating unit 438b. The multipliers 53c, 53f, and 53i are connected to the multiplier 54c, and the output thereof is connected to the integrating unit 438c.

  As an operation of this embodiment, the input RGB image data is once stored in the image memory 50. Next, each count of the matrix “A ′” from the coefficient register 51 is multiplied by the multiplier and the RGB image data stored in the image memory 50 by a calculation program stored in a predetermined storage device (not shown). The

  FIG. 28 shows an example in which the R signal and each matrix count are multiplied by multipliers 53a to 53c. Also, as shown in the figure, the G signal and each matrix count are multiplied by multipliers 53d to 53f, and the B signal and each matrix count are multiplied by multipliers 53g to 53i. The data multiplied by the matrix count is obtained by multiplying the outputs of the multipliers 53a, 53d and 53g by the multiplier 54a, the outputs of the multipliers 53b, 53e and 53h by the multiplier 54b, and the multipliers 53c, 53f, The outputs of 53i are respectively multiplied by the multiplier 54c. The output of the multiplier 54a is sent to the integrating unit 438a. The outputs of the multiplier 54b and the multiplier 54c are sent to the integrating units 438b and 438c, respectively.

  According to the present embodiment in FIG. 28, as in the first embodiment, a spectral image in which the blood vessel pattern is clearly displayed can be obtained.

  Further, in the present embodiment, the matrix processing is not performed by hardware as in the first embodiment, but is performed by using software, and therefore, for example, it is possible to quickly cope with a change in each matrix coefficient.

  In addition, the matrix count is stored not only as a result value, that is, as a matrix “A ′”, but separately as S (λ), H (λ), R (λ), G (λ), and B (λ). When the matrix “A ′” is obtained and used by performing calculations as necessary, only one of the elements can be changed, and convenience is improved. For example, it is possible to change only the spectral characteristic S (λ) of the illumination light.

  29 and 30 relate to the fourth embodiment of the present invention, FIG. 29 is a block diagram showing the configuration of the electronic endoscope apparatus, and FIG. 30 is a diagram showing the charge accumulation time of the CCD of FIG.

  Since the fourth embodiment is almost the same as the first embodiment, only differences from the first embodiment will be described, and the same components are denoted by the same reference numerals and description thereof will be omitted.

  In the present embodiment, the light source unit 41 and the CCD 21 are mainly different from the first embodiment. In the first embodiment, the color filter shown in FIG. 6 is provided in the CCD 21 and a color signal is generated by this color filter. In contrast, in this embodiment, the illumination light is emitted in a period of one frame. A so-called frame sequential method is used in which color signals are generated by illuminating in the order of RGB.

  As shown in FIG. 29, the light source unit 41 in the present embodiment is provided with a diaphragm 25 for dimming the front surface of the lamp 15, and R, G, B plane sequential light is emitted further on the front surface of the diaphragm 25. For this purpose, an RGB filter 23 that rotates, for example, once in one frame is provided. The RGB filter 23 constitutes a color separation unit. The diaphragm 25 is connected to a diaphragm control unit 24 serving as a light amount control unit, and restricts a light beam to be transmitted among light beams emitted from the lamp 15 in accordance with a control signal from the diaphragm control unit 24. Dimming is possible by changing. The RGB rotation filter 23 is connected to the RGB rotation filter control unit 26 and rotates at a predetermined rotation speed.

  As the operation of the light source unit in the present embodiment, the light beam output from the lamp 15 is limited to a predetermined light amount by the diaphragm 25, and the light beam transmitted through the diaphragm 25 passes through the RGB rotation filter 23 for a predetermined time. The light is output from the light source unit as R, G, and B illumination light each time. Each illumination light is reflected in the subject and received by the CCD 21. Signals obtained by the CCD 21 are distributed by a switching unit (not shown) provided in the endoscope apparatus main body 105 according to the irradiation time, and input to the S / H circuits 433a to 433c, respectively. That is, when the illumination light is irradiated from the light source unit 41 through the R filter, the signal obtained by the CCD 21 is input to the S / H circuit 433a. Other operations are the same as those in the first embodiment, and are omitted here.

  In the present embodiment, when the subject is illuminated with each of the illumination lights that have passed through the R, G, and B filters, the CCD 21 that outputs an imaging signal based on the image of the reflected light of each of the illumination lights, It is not limited to one configured as a single plate type, and may be configured as a multi-plate type such as a three-plate type.

  According to the present embodiment, as in the first embodiment, a spectral image in which the blood vessel pattern is clearly displayed can be obtained. Further, in this embodiment, unlike the first embodiment, it is possible to enjoy the merits of the so-called frame sequential method. This merit is, for example, the same as in Example 5 described later.

  In the above-described embodiment, the illumination light amount (light amount from the light source unit) is controlled and adjusted in order to avoid saturation of the RGB color signals. On the other hand, in this embodiment, a method of adjusting the electronic shutter of the CCD 21 is adopted. In the CCD, a charge proportional to the intensity of light incident within a predetermined time is accumulated, and the amount of the charge is used as a signal. The accumulation time corresponds to what is called an electronic shutter. By adjusting the electronic shutter with the CCD drive circuit 431, the amount of accumulated charge, that is, the signal amount can be adjusted. As shown in FIG. 30, a similar spectral image can be obtained by obtaining an RGB color image in a state where the charge accumulation time is sequentially changed for each frame. That is, in each of the embodiments described above, the illumination light amount control by the diaphragm 25 is used for obtaining a normal image, and when obtaining a spectral image, the electronic shutter is changed to avoid saturation of RGB color signals. Is possible.

  FIG. 31 is a diagram showing the charge accumulation time of the CCD according to Example 5 of the present invention.

  Since the fifth embodiment is almost the same as the fourth embodiment, only differences from the fourth embodiment will be described, and the same components are denoted by the same reference numerals and description thereof will be omitted.

  The present embodiment mainly uses the frame sequential method similarly to the fourth embodiment, and takes advantage of this advantage. By adding a weight for each of R, G, and B to the charge accumulation time by the electronic shutter control in the fourth embodiment, the generation of spectral image data can be simplified. That is, in this embodiment, the CCD drive 431 that can change the charge accumulation time of the CCD 21 for each of R, G, and B within one frame period is provided. Others are the same as in the fourth embodiment.

As an operation of the example of FIG. 31, when each illumination light is irradiated through the RGB rotation filter 23, the charge accumulation time by the electronic shutter in the CCD 21 is changed. Here, tdr, tdg, and tdb are the charge accumulation times of the CCD 21 when the illumination light is R, G, and B (in this figure, since the B color image signal has no accumulation time, tdb is omitted). ). For example, the F3 pseudo filter image in the case of performing the matrix processing represented by the equation (21) is obtained from an RGB image obtained by a normal endoscope,
[Equation 25]
F3 = −0.050R−1.777G + 0.829B (25)
Therefore, the charge accumulation time by RGB electronic shutter control in FIG.
tdr: tdg: tdb = 0.050: 1.777: 0.829 (26)
It should be set so that. Further, the matrix calculation unit simply adds the signal obtained by inverting only the R and G components and the B component. Thereby, the same spectral image as Example 1 thru | or Example 4 can be obtained.

  According to the present embodiment, a spectral image in which a blood vessel pattern is clearly displayed can be obtained as in the fourth embodiment. Further, in the present embodiment, as in the fourth embodiment, the frame sequential method is used for creating the color signal, and the charge accumulation time can be made different for each color signal using an electronic shutter. Therefore, in the matrix calculation unit, it is only necessary to perform addition and difference processing, and the processing can be simplified.

  32 and 33 relate to the living body observation apparatus according to the sixth embodiment of the present invention. FIG. 32 is a diagram illustrating an arrangement of color filters, and FIG. 33 is a diagram illustrating spectral sensitivity characteristics of the color filters.

  Since the sixth embodiment is almost the same as the first embodiment, only differences from the first embodiment will be described, and the same components are denoted by the same reference numerals and description thereof will be omitted.

  In the present embodiment, the color filter provided in the CCD 21 is mainly different from that in the first embodiment. In the first embodiment, an RGB primary color filter is used as shown in FIG. 6, whereas in the present embodiment, a complementary color filter is used.

  As shown in FIG. 32, the complementary color filter array is composed of G, Mg, Ye, and Cy elements. The relationship between each element of the primary color filter and each element of the complementary color filter is Mg = R + B, Cy = G + B, Ye = R + G.

  In this case, all the pixels of the CCD 21 are read out, and the image from each color filter is subjected to signal processing or image processing. Further, when the equations (1) to (8) and (19) to (21) for the primary color filter are modified in the case of a complementary color filter, the following equation (33) is obtained from the following equation (27): become that way. However, the characteristics of the target narrow-band bandpass filter are the same.

［数２８］

［数２９］
ｋG＝（∫Ｓ（λ）×Ｈ（λ）×Ｇ（λ）ｄλ）-1
ｋMg＝（∫Ｓ（λ）×Ｈ（λ）×Ｍｇ（λ）ｄλ）-1
ｋCy＝（∫Ｓ（λ）×Ｈ（λ）×Ｃｙ（λ）ｄλ）-1
ｋYe＝（∫Ｓ（λ）×Ｈ（λ）×Ｙｅ（λ）ｄλ）-1 …（２９）
［数３０］

［数３１］

［数３２］

［数３３］

また、図３３に、補色型カラーフィルタを用いた場合の分光感度特性、目標とするバンドパスフィルタ及び上記（２７）式乃至（３３）式により求められ擬似バンドパスフィルタの特性を示す。 [Equation 27]

[Equation 28]

[Equation 29]
kG = (∫S (λ) × H (λ) × G (λ) dλ) −1
kmg = (∫S (λ) × H (λ) × Mg (λ) dλ) −1
kCy = (∫S (λ) × H (λ) × Cy (λ) dλ) −1
kYe = (∫S (λ) × H (λ) × Ye (λ) dλ) −1 (29)
[Equation 30]

[Equation 31]

[Formula 32]

[Equation 33]

FIG. 33 shows the spectral sensitivity characteristics when the complementary color filter is used, the target band-pass filter, and the characteristics of the pseudo band-pass filter obtained from the equations (27) to (33).

  When the complementary color filter is used, it goes without saying that the S / H circuit shown in FIG. 4 is performed for G, Mg, Cy, and Ye instead of R, G, and B, respectively.

  Further, even when a complementary color filter is used, the matrix estimation method shown in the equations (9) to (18) can be applied. In this case, when the number of the complementary color filters is four, the biological spectral reflectance assumed in the equation (14) can be approximated by three basic spectral characteristics, or four or less. Become. Accordingly, in accordance with this, the dimension for calculating the estimation matrix is changed from 3 to 4.

  According to the present embodiment, as in the first embodiment, a spectral image in which the blood vessel pattern is clearly displayed can be obtained. Further, in this embodiment, it is possible to enjoy the advantages of using a complementary color filter.

  According to each embodiment described above, there is an effect that tissue information of a desired deep part of a living tissue based on a spectral image obtained by signal processing can be adjusted to image information of a color tone suitable for observation.

  The embodiments of the present invention have been described above. However, the present invention may be used by combining various embodiments described above, and modifications may be made without departing from the spirit of the present invention.

  For example, for all the embodiments described above, a new pseudo band-pass filter can be created by the operator himself / herself at other timings in the clinic and applied to the clinic. That is, as shown in the first embodiment, a design unit (not shown) that can calculate and calculate matrix coefficients may be provided in the control unit 42 in FIG.

  Thereby, by inputting conditions via the keyboard provided in the endoscope main body shown in FIG. 3, a pseudo bandpass filter suitable for obtaining a spectral image that the operator wants to know is newly designed, Correction coefficient (corresponding to each element of the matrix “K” in the expressions (20) and (32)) to the calculated matrix coefficient (corresponding to each element of the matrix “A” in the expressions (19) and (31)) By setting the final matrix coefficient (corresponding to each element of the matrix “A ′” in the equations (21) and (33)) in the matrix calculation unit 436 in FIG. it can.

  FIG. 34 shows a flow until application. This flow will be described in detail. First, the operator inputs information about a target bandpass filter (for example, a wavelength band) via a keyboard or the like. As a result, the matrix “A ′” is calculated along with the characteristics of the light source / color filter already stored in a predetermined storage device or the like, and as shown in FIG. 32, along with the characteristics of the target bandpass filter, The calculation result (pseudo bandpass filter) by the matrix “A ′” is displayed on the monitor as a spectrum diagram.

  After confirming the calculation result, the operator performs setting when using the newly created matrix “A ′”, and generates an actual endoscopic image using this matrix “A ′”. Is done. In addition, the newly created matrix “A ′” is stored in a predetermined storage device and can be used again according to a predetermined operation by the operator.

  As a result, the operator can generate a new band-pass filter based on his / her own experience etc. without being bound by the existing matrix “A ′”, and is particularly effective when used for research purposes. .

  As described above, according to each embodiment of the present invention, a more accurate or reliable spectral signal can be obtained.

  The present invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the scope of the present invention.

FIG. 3 is a conceptual diagram showing a signal flow when creating a spectral image signal from a color image signal according to Embodiment 1 of the present invention. FIG. 3 is a conceptual diagram showing integration calculation of spectral image signals according to Embodiment 1 of the present invention. 1 is an external view showing an external appearance of an electronic endoscope apparatus according to Embodiment 1 of the present invention. The block diagram which shows the structure of the electronic endoscope apparatus of FIG. External view showing the external appearance of the chopper of FIG. The figure which shows the arrangement | sequence of the color filter arrange | positioned at the imaging surface of CCD of FIG. The figure which shows the spectral sensitivity characteristic of the color filter of FIG. The block diagram which shows the structure of the matrix calculating part of FIG. The spectrum figure which shows the spectrum of the light source which concerns on Example 1 of this invention The spectrum figure which shows the reflection spectrum of the living body concerning Example 1 of the present invention, FIG. 11 is a diagram showing a layer direction structure of a biological tissue observed by the electronic endoscope apparatus of FIG. The figure explaining the arrival state to the layer direction of the biological tissue of the illumination light from the electronic endoscope apparatus of FIG. Diagram showing spectral characteristics of each band of white light The 1st figure which shows each band image by the white light of FIG. 2nd figure which shows each band image by the white light of FIG. 3rd figure which shows each band image by the white light of FIG. The figure which shows the spectral characteristic of the spectral image produced | generated by the matrix calculating part of FIG. The 1st figure which shows each spectral image of FIG. The 2nd figure which shows each spectral image of FIG. The 3rd figure which shows each spectral image of FIG. FIG. 21 is a block diagram showing the configuration of the color adjustment unit in FIG. The figure explaining the effect | action of the color adjustment part of FIG. FIG. 4 is a block diagram showing a modification of the color adjustment unit in FIG. The figure which shows the spectral characteristic of the 1st modification of the spectral image of FIG. The figure which shows the spectral characteristic of the 2nd modification of the spectral image of FIG. The figure which shows the spectral characteristics of the 3rd modification of the spectral image of FIG. FIG. 3 is a block diagram showing a configuration of an electronic endoscope apparatus according to Embodiment 2 of the present invention. FIG. 9 is a block diagram showing the configuration of a matrix calculation unit according to the third embodiment of the present invention. FIG. 29 according to the fourth embodiment of the present invention is a block diagram showing a configuration of an electronic endoscope apparatus. The figure which shows the charge accumulation time of CCD of FIG. The figure which shows the charge accumulation time of CCD which concerns on Example 5 of this invention. The figure which shows the arrangement | sequence of the color filter which concerns on Example 6 of this invention. FIG. 32 shows the spectral sensitivity characteristics of the color filter. Flowchart at the time of matrix calculation in a modification according to the present invention Diagram showing appearance of rigid endoscope Diagram showing appearance of oral camera The figure which shows the external appearance of the camera used in contact with the surface of a living body

Explanation of symbols

DESCRIPTION OF SYMBOLS 41 ... Light source part 42 ... Control part 43 ... Main body processing apparatus 100 ... Electronic endoscope apparatus 101 ... Scope 102 ... Insertion part 103 ... Tip part 104 ... Angle operation part 105 ... Endoscope apparatus main body 106 ... Display monitor 436 ... Matrix Arithmetic unit 440 ... color adjustment unit 440a ... color conversion processing circuit

Claims (10)

Controlling the operation of the imaging unit that generates an imaging signal by photoelectrically converting light reflected from the illumination unit that irradiates light to the living body that is the subject and / or illumination light from the illumination unit A signal processing device for a biological observation device comprising a signal processing control unit for outputting the imaging signal to a display device,
A spectral signal generation unit that generates a spectral signal corresponding to an optical wavelength narrowband image from the imaging signal by signal processing;
A color adjusting unit that adjusts a color tone for each of a plurality of bands forming the spectral signal when the spectral signal is output to the display device;
Have
The signal processing apparatus for a biological observation apparatus, wherein the imaging signal is generated through a color separation unit.   The biological imaging apparatus signal processing apparatus according to claim 1, wherein the imaging unit is included in an endoscope.   The signal processing apparatus for a biological observation apparatus according to claim 2, wherein the endoscope is a flexible mirror.   The biological observation apparatus signal processing apparatus according to claim 2, wherein the endoscope is a rigid endoscope.   The biological imaging apparatus signal processing apparatus according to claim 1, wherein the imaging unit is included in an oral camera.   The signal processing apparatus for a biological observation apparatus according to claim 1, wherein the imaging unit is included in a camera that images the living body in a state where the imaging unit is in contact with the surface of the living body.   The signal processing apparatus for a biological observation apparatus according to claim 1, wherein the color separation unit is a color filter provided in the imaging unit.   The biological color observation apparatus signal processing device according to claim 1, wherein the color separation unit is a color filter provided in the illumination unit.   The signal processing apparatus for a biological observation apparatus according to claim 1, wherein the primary color of the color separation unit is an RGB primary color system. The signal processing apparatus for a biological observation apparatus according to claim 1, wherein the primary color of the color separation unit includes a CMY complementary color system.

Images