JP2022133968A - Image calibration device - Google Patents

Abstract

To provide an image calibration device which reduces degradation of quality of a synthesized image and artifact when two images are synthesized in an endoscope system.SOLUTION: An image calibration device has: a first chart; a second chart; a third chart; a light source irradiating the first, second and third charts with illumination light; a stage mounting the first, second, and third charts; and a calibration processor. The calibration processor has: a first calculation part calculating space frequency characteristic value of two first images obtained by capturing an image of the first chart respectively; a second calculation part calculating difference in a relative position relation between two second images obtained by capturing an image of the second chart with an optical device; a third calculation part calculating difference in relative brightness ratio of two images of the third chart for every RGB signal; and a parameter calculation part calculating a first correction parameter and a second correction parameter.SELECTED DRAWING: Figure 2

Description

The present invention relates to an image calibration device. For example, the present invention relates to an image calibration device for image synthesis of endoscopes used in medical and industrial fields.

Endoscopes are devices that are widely used in the medical and industrial fields. In particular, in the medical field, an image obtained by an endoscope inserted into a body cavity is used for diagnosis and treatment of an observation site.

2. Description of the Related Art Generally, it is known that in endoscope systems and other devices equipped with an imaging device, the depth of field becomes narrower as the number of pixels of the imaging device increases. That is, if the pixel pitch (the vertical and horizontal dimensions of one pixel) is reduced in order to increase the number of pixels in an image pickup device, the permissible circle of confusion will also become smaller, and the depth of field of the image pickup device will be narrower.

The following Patent Documents 1, 2, 3, and 4 propose depth-enhancing endoscopes that acquire two images with different focal points at the same time and synthesize the images.

There is an endoscope that divides an image from an endoscope objective optical system into two by a polarizing beam splitter (PBS), synthesizes the two divided images, and expands the depth, expands the dynamic range, and improves the resolution. At this time, as pre-processing for image synthesis processing, calibration processing is performed for geometrically aligning the two images and correcting the brightness and color tone to substantially the same.

WO2013/027459 WO2013/061819 WO2014/002740 WO2016/043107

Japanese Patent Application Laid-Open No. 2002-200002 proposes a depth-enhancing endoscope system that corrects differences in position, magnification, rotation, color tone, and brightness of two images by image processing and synthesizes the images. In Patent Documents 2 and 3, the correction value is held as a unique correction parameter, and when the target endoscope is connected to the image processing unit, the endoscope reads out the correction parameter and corrects it in real time. A system is proposed. Patent Literature 4 proposes an endoscope system that switches correction parameters according to the observation mode of the endoscope so that the optimum synthetic depth of field can be obtained even in narrowband observation.

Patent Documents 1, 2, 3, and 4 do not disclose a specific acquisition method or configuration of the correction parameters themselves. Therefore, when two images are synthesized, if the optimum correction is not performed, there is a problem that the image quality of the synthesized image deteriorates and artifacts (noise) occur.

When aligning two images, a chart for alignment is placed on the objective optical system of the endoscope, the image is captured, and calibration is performed. There is a problem that it is not possible to In addition, when correcting brightness and color tone, if an inappropriate chart or light source is used, correct calibration cannot be performed due to the influence of polarization dependence and wavelength dependence, and artifacts are likely to occur in the composite image. Furthermore, the endoscope system has various observation modes, each of which has different characteristics of brightness and color tone. Therefore, there is a problem that switching the observation mode deviates from the optimum correction state.

The present invention has been made in view of such problems. An object of the present invention is to provide an image calibration device that reduces artifacts (noise).

In order to solve the above-described problems and achieve the object, an image calibration device according to at least some embodiments of the present invention comprises an objective optical system and an object image obtained by the objective optical system, in order from the object side. An image calibration device for an optical device having an optical path splitting unit that splits into two optical images with different optical path differences, and an imaging device that acquires two images from the two optical images,
a first chart;
a second chart;
a third chart;
a light source that irradiates the first chart, the second chart, and the third chart with illumination light;
a stage on which the first chart, the second chart and the third chart are placed;
a calibration processor, the calibration processor comprising:
a first calculator that calculates spatial frequency characteristic values of each of the first two images obtained by imaging the first chart;
A second image obtained by imaging the second chart with an optical device in a state where the second chart is placed at a position of the first chart where the spatial frequency characteristic values of the first two images are equivalent. a second calculator that calculates the difference in the relative positional relationship between the two images of
a third calculator for calculating, for each RGB signal, a difference in relative brightness ratio between two third images obtained by imaging a third chart illuminated by a light source with an optical device; A parameter calculation unit that calculates a first correction parameter for correcting the difference in the brightness ratio and a second correction parameter for correcting the difference in the brightness ratio;
have

INDUSTRIAL APPLICABILITY The present invention can provide an image calibration device that reduces deterioration in image quality and artifacts (noise) of a synthesized image by obtaining optimum correction parameters in advance when synthesizing two images in an endoscope system. It has the effect of

1 is a perspective configuration diagram of an image calibration device according to an embodiment; FIG. 1 is a configuration diagram of an image calibration device according to an embodiment; FIG. FIG. 4 is a diagram (normal observation state) showing a cross-sectional configuration of an objective optical system, an optical path splitter, and an imaging device of an optical device installed in an image calibration device; FIG. 2 is a schematic configuration diagram of a depolarizing plate, an optical path dividing section, and an imaging element included in an optical device; 2 is a schematic configuration diagram of an imaging element included in the optical device; FIG. 4 is a flow chart showing a calibration process of the image calibration device according to the embodiment; (a) is a diagram showing a first chart. (b) is a diagram showing two images of the first chart captured by the imaging device. It is a figure which shows the spatial frequency characteristic value of two images of a 1st chart. FIG. 3 is another perspective configuration diagram of the image calibration device according to the embodiment; FIG. 10 is a diagram showing a second chart; It is a figure which shows two images of the 2nd chart which the imaging device imaged. 3 is another perspective configuration diagram of the image calibration device according to the embodiment; FIG. It is a figure which shows two images of the 3rd chart which the imaging device imaged. It is a figure which shows the spectral distribution characteristic of a 3rd chart. It is a functional block diagram which shows the structure of an endoscope system.

An image calibration device according to an embodiment will be described in detail below with reference to the drawings. In addition, this invention is not limited by this embodiment.

FIG. 1 is a perspective configuration diagram of an image calibration device 100 according to an embodiment. FIG. 2 is another configuration diagram of the image calibration device 100. As shown in FIG.

The image calibration device 100 includes, in order from the object side, an objective optical system OBL (FIG. 3), an optical path splitter 120 that splits the subject image obtained by the objective optical system OBL into two optical images with different optical path differences, An image calibration device 100 for an optical device 200 having an imaging device 122 that acquires two images from two optical images. The optical device 200 is included in the endoscope system 1 (FIG. 15) capable of increasing the depth of field. The configuration of the endoscope system 1 and the configuration of the optical device 200 will be described later.

As shown in FIG. 2, the image calibration device 100 includes a first chart 101 (FIG. 7), a second chart 102 (FIG. 10), a third chart 103 (FIG. 2), and these charts. 101, 102, 103 through a light guide path 116, a stage 104 on which the first chart 101, the second chart 102 and the third chart 103 are placed, and a calibration processor 108. (FIG. 2).

The processor unit 4, which will be described later, has a light source 115. As shown in FIG. A light source 115 provides light for illuminating the first, second and third charts 101 , 102 , 103 . Light from the light source 115 is guided by the light guide path 116 in the direction in which the first chart 101, the second chart 102, and the third chart 103 are placed.

The light guide path 116 can be composed of an optical fiber or the like. The first chart 101 and the second chart 102 are transmissive charts. The light emitted from the light guide path 116a irradiates the first chart 101 or the second chart 102 from the left side of the paper surface. Light emitted from the light guide path 116b is guided into the third chart 103 (integrating sphere).

For example, light from the light source 115 is guided through the light guide path 116 to light the backlight. The guided light irradiates a transmissive first chart 101 or second chart 102 provided side by side with the backlight.

The optical device 200 captures images of the first chart 101 and the second chart 102 illuminated by the backlight.

As for the third chart 103 (integrating sphere), illumination light guided from the light source 115 through the light guide path 116b illuminates the interior of the integrating sphere.

The optical device 200 images the inside of the integrating sphere. As a modification, the light source 115 may be externally installed separately. Further, as the light source 115, illumination light from the light source device 3 (FIG. 15) for the endoscope illumination optical system of the endoscope system 1 can be used.

Returning to FIG. 2, the description continues. The calibration processor 108 has a first calculator 110 , a second calculator 111 , a third calculator 112 and a parameter calculator 113 .

The first calculator 110 calculates the spatial frequency characteristic values of the first two images 101a and 101b (FIG. 7B) obtained by imaging the first chart 101 .

Next, the second calculation unit 111 places the second chart 102 at the position L where the spatial frequency characteristic values of the first two images 101a and 101b are equal, and calculates the second chart 102 Calculate the difference in the relative positional relationship between the second two images 102a and 102b (FIG. 11) obtained by imaging with the optical device 200,

The third calculator 112 calculates the relative brightness ratio of the third two images 103a and 103b (FIG. 13) obtained by imaging the third chart 103 illuminated by the light source with the optical device 200. A difference is calculated for each RGB signal.

The parameter calculator 113 calculates a first correction parameter for correcting the difference in positional relationship between the two images 102a and 102b, and a second correction parameter for correcting the difference in brightness ratio.

First, an example of the optical device 200 to be calibrated by the image calibration device 100 will be described. FIG. 3 is a view (normal observation state) showing a cross-sectional configuration of the objective optical system OBL, the optical path splitter 120, and the imaging device 122 of the optical device 200 installed in the image calibration device 100. As shown in FIG.

The objective optical system OBL has a three-group configuration of a first group G1, a second group G2, and a third group G3 each having lenses L1 to L10.

The optical device 200 includes, in order from the object side, an objective optical system OBL, a depolarizing plate 121a, an optical path splitting section 120 that splits the light from the objective optical system OBL into two, and an imaging device that captures the two split images. element 122; The depolarizing plate 121 a is arranged in the optical path between the objective optical system OBL and the optical path splitter 120 . The depolarizing plate 121a has a function of depolarizing.

FIG. 4 is a schematic configuration diagram of the depolarizing plate 121a included in the optical device 200, the optical path splitter 120, and the imaging element.

The light emitted from the objective optical system OBL enters the optical path splitting section 120 through the depolarizing plate 121a. The depolarizing plate 121a has a function of depolarizing light with a simple structure, as will be described later with reference to FIG.

The optical path splitter 120 has a polarizing beam splitter 121 that splits the subject image into two optical images with different focus, and an imaging device 122 that captures the two optical images to obtain two images.

As shown in FIG. 5, the polarizing beam splitter 121 includes an object-side prism 121b, an image-side prism 121e, a mirror 121c, and a λ/4 plate 121d. Both the object-side prism 121b and the image-side prism 121e have beam splitting surfaces that are inclined at 45 degrees with respect to the optical axis AX.

A polarization separating film 121f is formed on the beam splitting surface of the prism 121b on the object side. The object-side prism 121b and the image-side prism 121e constitute a polarizing beam splitter 121 with their beam splitting surfaces in contact with each other via a polarizing splitting film 121f.

The mirror 121c is provided in the vicinity of the end surface of the prism 121b on the object side via a λ/4 plate 121d. An imaging element 122 is attached to the end face of the prism 121e on the image side via a cover glass CG. I is an imaging plane (imaging plane).

An object image from the objective optical system OBL is separated into a P-polarized component (transmitted light) and an S-polarized component (reflected light) by a polarization separation film 121f provided on the beam splitting surface of a prism 121b on the object side. is separated into two optical images, an optical image on the side and an optical image on the transmitted light side.

The optical image of the S-polarized component is reflected by the polarizing separation film 121f toward the imaging device 122, passes through the A optical path, passes through the λ/4 plate 121d, is reflected by the mirror 121c, and is turned back toward the imaging device 122. be The folded optical image is transmitted through the λ/4 plate 121 d again, the polarization direction of which is rotated by 90°, and is transmitted through the polarization separation film 121 f to be imaged on the imaging element 122 .

The optical image of the P-polarized component passes through the polarization separation film 121f, passes through the B optical path, and is reflected by a mirror surface provided on the opposite side of the beam splitting surface of the prism 121e on the image side that is vertically folded toward the imaging device 122. and is imaged on the imaging device 122 . At this time, the prism glass path is set so as to generate a predetermined optical path difference of, for example, about several tens of μm between the A optical path and the B optical path, and two optical images with different focal points are captured on the light receiving surface of the image sensor 122. to form an image.

That is, the optical path length ( It is arranged so that the optical path length on the reflected light side is shorter (smaller) than the glass path length).

Further, the configuration is not limited to the configuration of the present embodiment as long as the configuration can obtain an arbitrary optical path length difference in each of the optical path lengths (glass path lengths) of the two optical images. For example, the prism 121b on the object side may be arranged so that the optical path length on the reflected light side is longer (larger) than the optical path length (glass path length) on the transmitted light side reaching the image sensor 122 .

FIG. 5 is a schematic configuration diagram of the imaging element 122. As shown in FIG. As shown in FIG. 5, the imaging element 122 has two light receiving areas (effective pixels regions) 122a and 122b are provided.

FIG. 6 is a flow chart showing the calibration process of the image calibration device 100 according to the embodiment.

In step S<b>1 , the first chart 101 is placed on the holder 106 on the stage 104 . FIG. 7A is a front view of the first chart 101. FIG.

The first chart 101 is a chart for obtaining spatial frequency characteristic values, and has a chart pattern including black and white edge lines 101bk. The first chart 101 is arranged so that the visual field centers 122ac and 122bc of the two images 101a and 101b intersect with the black and white edge ridgelines 101abk and 101bbk, respectively.

Generally, the visual resolution defined by ISO12233, the limit resolution (SIN wave chart), the SFR (Spatial Frequency Response) chart, etc. are used for evaluation of the spatial frequency characteristic value (which is synonymous with resolution here). The first chart 101 in this embodiment is preferably an SFR chart. The reason is as follows.

In order to obtain the observation distance L of the second chart 102 to be described later, it is necessary to obtain the spatial frequency characteristic value for each observation distance with an arbitrary step width in each of the A and B optical paths. When using a chart for evaluating visual resolution or limit resolution, it is necessary to prepare a chart with a different pitch pattern for each observation distance, or to consider a correction coefficient according to the observation magnification. In that case, the configuration of the image calibration apparatus 100 becomes complicated and the cost increases, which is not preferable. Therefore, in this embodiment, an SFR chart that is not affected by observation magnification is used.

In step S2, the imaging element 122 captures an image 101a that is an image focused on the near point (hereinafter referred to as a "near point image"). The imaging device 122 captures an image 101b that is an image in which the far point is in focus (hereinafter referred to as a "far point image").

Further, the image calibration device 100 has a drive section 114 and a storage section 109 .

An endoscope that expands the depth of field can switch between a normal focus (far point object observation) mode and a close focus (near point object observation) mode by driving the focus lens (the second group G2 in FIG. 3). You can switch. In this embodiment, it is desirable to perform calibration on the normal focus mode side. When performing calibration in the close focus mode side, it is necessary to set the observation distance between the first chart and the optical device 200 with relatively high accuracy compared to the normal focus mode, and the actuator and control software are expensive. This is because it is easy to become

The drive unit 114 changes the observation distance of the first chart 101 by moving the stage 104 with respect to the optical device 200 in arbitrary steps (step S5). The drive unit 114 is, for example, a motor or an actuator.

Note that the observation distance of the first chart 101 may be changed relative to the optical device 200 . Therefore, the position of the optical device 200 held by the holder 107 may be changed by driving the stage 105 instead of the stage 104 .

The storage unit 109 stores still images of the first chart 101 for each observation distance. The first calculator 110 calculates spatial frequency characteristic values of each of the first two images calculated from the still image of the first chart 101 stored in the storage unit 109 . The first calculator 110 calculates the observation distance L at which the spatial frequency characteristic values of the two images 101a and 101b are the same (steps S3 and S4).

FIG. 8 shows the spatial frequency characteristic values of two images 101a (near point images, plotted with triangles) and the spatial frequency characteristic values of an image 101b (far point image, plotted with circles) of the first chart 101. . The first calculator 110 calculates the observation distance L at which the two images 101a and 101b have the same spatial frequency characteristic value. The horizontal axis is the defocus amount (unit: mm), and the vertical axis is the spatial frequency characteristic value (unit: %). The spatial frequency characteristic value is obtained by SFR (Spatial Frequency Response).

The second chart 102 is placed on the holder 106 of the stage 104 at the observation distance L calculated by the first calculator 110 (step S6). If the first chart 101 and the second chart 102 are arranged on the same plane of the chart substrate, it is not necessary to place the second chart 102 on the holder 106 again. The storage unit 109 stores the still images 102a and 102b of the second chart (step S7). The parameter calculator 113 calculates a first correction parameter based on the still images 102a and 102b of the second chart and the difference in the positional relationship calculated by the second calculator 111 (steps S8 and S9).

In the optical device 200, when image synthesis is performed according to the spatial frequency characteristic values of the images, it is necessary to perform alignment correction of the two images with high precision. Here, if one of the two images has a low spatial frequency characteristic value (that is, the image is out of focus), the alignment correction accuracy decreases. As a result, the resolution of the synthesized image is degraded and artifacts (noise) are generated, which is undesirable.

In contrast, in the present embodiment, the observation distance L at which the spatial frequency characteristic values of the two images 101a and 101b of the first chart 101 are the same is calculated. Then, the second chart 102 for alignment is arranged at the calculated observation distance L. FIG. Accordingly, there is an effect that a correction parameter (first correction parameter) for alignment can be obtained.

FIG. 9 shows a configuration in which a second chart is arranged in the image calibration device 100. As shown in FIG. FIG. 10 is a diagram showing the second chart 102. As shown in FIG. FIG. 11 shows two images 102a and 102b of the second chart captured by the optical device 200. FIG.

A second chart 102 is a grayscale checker pattern in which at least two or more different gradation areas are randomly arranged.

A normal black and white pattern chart is edge matching using only two gradations. On the other hand, in the present embodiment, a pattern in which multilevel gradations are randomly arranged is used. Specifically, an error function of luminance differences of a plurality of gradation patterns between the two images 102a and 102b of the second chart 102 is calculated. Then, the two images are aligned by minimizing the calculated value of the error function. This enables highly accurate alignment.

In a preferred aspect of the present embodiment, the parameter calculator 113 calculates the first correction parameter using one of the second two images 102a and 102b as a reference.

The purpose of the calibration in this embodiment is to avoid artifacts and superimposed images in the synthesized image by relatively eliminating the difference between the two images. Therefore, finding the absolute value of the difference between two images is not very meaningful. Deriving the absolute value of the difference (correct coordinate value) between the two images is very difficult considering the measurement method and manufacturing error. It is not preferable because it is very complicated and the scale becomes large. Therefore, in this embodiment, it is desirable to use one of the two images as the reference image.

Also, in a preferred aspect of this embodiment, the third chart 103 is an integrating sphere. The spectral characteristics of the illumination light guided to the integrating sphere are the same as the spectral characteristics when the optical device 200 is used.

FIG. 12 is another perspective view of the third chart 103 (integrating sphere) in the image calibration device 100 according to this embodiment. FIG. 13 shows two images 103a and 103b of the third chart 103 captured by the optical device 200. FIG.

When correcting the relative brightness difference between the two images, it is desirable to correct using illumination light that has the same characteristics as the light source characteristics (spectral characteristics) used in the endoscope system 1 product. For example, if a commercially available LED illumination device is used, the color tone and brightness are completely different from those of the endoscope light source, so the second correction parameters are deviated from the optimum. Even if the image processing unit corrects the color tone and brightness further, it is not preferable because the circuit size increases and the noise in the image increases.

The inner surface of the integrating sphere, which is the third chart 103, corresponds to the perfect diffusion surface. Therefore, the light emitted from the integrating sphere has almost no polarization dependence. In the optical device 200 using a polarizing beam splitter in the optical path splitter 120, when correcting color tone and brightness, if a subject dependent on polarization is used as a reference (opposition), uniform image splitting cannot be performed. Therefore, it is difficult to correct the two images correctly. Therefore, in the present embodiment, the subject is an integrating sphere with extremely little polarization dependence (step S10).

As described above, the third calculator 112 calculates the relative brightness of the two images 103a and 103b of the third chart 103 obtained by imaging the third chart 103 illuminated by the light source with the optical device 200. A difference in ratio is calculated for each RGB signal (step S11).

The parameter calculator 113 calculates a second correction parameter for correcting the difference in brightness ratio between the two images 103a and 103b (step S12).

14A, 14B, and 14C are image diagrams of spectral distribution characteristics emitted from the light source 115. FIG. The horizontal axis indicates a wavelength band (nm), and the vertical axis indicates an arbitrary intensity Is. As described above, with respect to the third chart 103 (integrating sphere), the illumination light guided from the light source 115 through the light guide path 116b illuminates the interior of the integrating sphere. The optical device 200 images the inside of the integrating sphere. Therefore, the light having spectral distribution characteristics as shown in FIGS. 14(a), (b), and (c) is guided to the integrating sphere by the light guide path 116b.

The optical device 200 is an endoscope system 1 (FIG. 15) capable of switching between different observation modes.
). The illumination light from the light source is switched for each observation mode of the endoscope system 1 . The parameter calculator 113 calculates a second correction parameter for each observation mode.

Observation modes of endoscopes include observation modes using narrow-band wavelengths in addition to observation modes using white light. In calibration of brightness and color tone, matching may not be appropriate with correction parameters that target only white light. Therefore, in this embodiment, it is possible to calculate and store correction parameters for each illumination light used in each observation mode.

The illumination light used for the calibration of brightness and color tone is illumination light corresponding to each observation mode of the endoscope, at least broadband light of 400 to 700 nm used in normal observation mode and narrowband observation mode. Brightness and color tone calibration can be performed for each viewing mode using a plurality of narrow-band illumination lights.

In general, polarizing elements such as polarizing beam splitters, retardation plates, and depolarizing plates have wavelength dependence. Therefore, better image quality can be obtained by setting a correction parameter for each observation mode. There are individual differences in endoscopes that expand the depth of field due to manufacturing errors in optical elements and assembly. Therefore, a unique correction parameter is required for each individual (optical device 200) as in the present embodiment.

The brightness and color tone correction parameters obtained by calibration are switched in conjunction with each observation mode of the endoscope to correct the image.

For calibration of brightness and color tone matching, an integrating sphere, which is less dependent on polarization and wavelength, is used as the subject, and the light source is a light source with spectral characteristics equivalent to the product. It is also desirable to use a light guide that guides light and that has a spectral transmission characteristic equivalent to that of the product.

As described above, in the present embodiment, highly accurate alignment and calibration of brightness and color tone independent of wavelength and polarization are possible, and an optimal composite image can be obtained in various observation modes. There is

FIG. 15 is a functional block diagram showing the configuration of an endoscope system that stores the correction parameters described above. An endoscope system 1 of this embodiment includes an endoscope 2 inserted into a subject, a light source device 3 that supplies illumination light to the endoscope 2, a processor device 4, and an image display unit 5. , have

The processor device 4 has a function of performing image processing, but also has other functions. The processor device 4 has an actuator controller 25 , an image processor 30 and a controller 39 . The image display device 5 displays the image signal generated by the processor device 4 as an endoscopic image.

The endoscope 2 has an elongated insertion section 6 to be inserted into the subject, and an operation section 7 provided at the rear end of the insertion section 6 . A light guide cable 8 extends outward from the operating portion 7 . One end of the light guide cable 8 is detachably connected to the light source device 3 via a connecting portion 8a. The light guide cable 8 has a light guide 9 inside. A portion of the light guide 9 is arranged inside the insertion portion 6 .

The light source device 3 incorporates a lamp 11 such as a xenon lamp as a light source. The light source is not limited to the lamp 11 such as a xenon lamp, and a light emitting diode (abbreviated as LED) may be used. Illumination light generated by the lamp 11 , for example, white light, passes through the diaphragm 12 and is adjusted in amount. The illumination light is condensed by the condenser lens 13 and enters the incident end surface of the light guide 9 . The aperture diameter of the diaphragm 12 can be changed by the diaphragm driver 14 .

The light guide 9 transmits illumination light generated by the light source device 3 to the distal end portion 6 a of the insertion portion 6 . The transmitted illumination light is emitted from the tip surface of the light guide 9 . An illumination lens 15 is arranged at the distal end portion 6a so as to face the distal end surface. The illumination lens 15 emits illumination light through an illumination window 15a. As a result, the site to be observed inside the subject is illuminated.

An observation window 20 is provided in the distal end portion 6a next to the illumination window 15a. Light from the site to be observed passes through the observation window 20 and enters the distal end portion 6a. An objective optical system OBL is arranged behind the observation window 20 . Objective optical system OBL is composed of lens group 16 and optical path splitter 120 .

The lens group 16 has a lens 16 a and a lens 21 . The lens 21 is movable along the optical axis. Focusing is thereby performed. An actuator 22 is arranged to move the lens 21 .

One imaging element 122 (not shown) is arranged in the optical path splitting section 120 . Two optical images are simultaneously formed on the light receiving surface of the imaging element 122 . The two optical images are captured by the imaging device 122 .

The operation unit 7 is connected to the processor device 4 via a cable 24 . A signal connector 24 a is provided at a connection point with the processor device 4 . Various information is communicated between endoscope 2 and processor unit 4 via cable 24 . The signal connector 24 a has a correction parameter storage section 37 .

The correction parameter storage unit 37 stores (information on) correction parameters used for image correction. Correction parameters are different for each endoscope. Assume that an endoscope having unique endoscope identification information is connected to processor unit 4 . In this case, a correction parameter unique to the connected endoscope is read from the correction parameter storage unit 37 based on the endoscope identification information. The image is corrected in the image correction processing unit 32 based on the read correction parameters. The presence or absence of correction is performed by the control unit 39 .

Control of the actuator 22 is performed by an actuator control section 25 . Therefore, the actuator 22 and the actuator control section 25 are connected via a signal line 23 . The imaging device is also connected to the image processor 30 via a signal line 27a. A signal from the imaging device is input to the image processor 30 . Information on the switch 26 provided on the operation unit 7 is also transmitted to the processor device 4 via the signal line 27a.

If the optical path length in the first optical path is slightly different from the optical path length in the second optical path, two in-focus optical images are formed in front of and behind the imaging plane. The amount of deviation of the optical image with respect to the imaging plane is slight. As a result, two optical images are formed on the image pickup plane in a state in which only a part of the area is in focus.

The two optical images are picked up by an imaging device. An image signal obtained by imaging is input to the image processor 30 via the signal line 27a. The image processor 30 has an image reading section 31 , an image correction processing section 32 , an image synthesis processing section 33 , a post-stage image processing section 34 , an image output section 35 and a light control section 36 .

The image reading unit 31 reads image signals of a plurality of images from the input image signals. Here, both the number of optical images and the number of images are two.

A geometric difference may occur in an optical system that forms two optical images. Geometrical differences include relative deviations (differences) between two optical images, such as magnification deviations (differences), positional deviations (differences), and rotational deviations (differences). It is difficult to completely eliminate these differences when manufacturing the objective optical system OBL. However, if the amount of deviation (difference) between them becomes large, for example, the composite image appears double. Therefore, it is preferable that the image correction processing unit 32 corrects the geometrical difference described above.

The image correction processing unit 32 performs image correction on the two read images. In the image correction processing unit 32, for example, a process of matching at least one of the relative magnification difference, positional difference, and rotational difference between the two images is performed.

Furthermore, the image correction processing unit 32 performs color tone correction. Therefore, the image correction processing section 32 has a color tone correction section (not shown). Color tone correction is a process of substantially matching the relative brightness and saturation of two images in at least one arbitrary specific wavelength band. Color tone correction may be performed by the image correction processing section 32 without providing the color tone correction section.

The image correction processing unit 32 changes the brightness of one of the two images so that it substantially matches the brightness of the other image. Further, the image correction processing unit 32 changes the saturation of one image so as to substantially match the saturation of the other image.

As described above, in the method of acquiring an image with a large depth of field, only in-focus areas are extracted from a plurality of images, and the extracted areas are combined. In the endoscope system of this embodiment, it is possible to reduce differences in brightness and differences in color tone between a plurality of images. Therefore, unevenness in brightness and difference in color tone can be reduced in the combined image.

Further, in a method for improving the color reproducibility of an image, image synthesis is performed using two images. If the two optical images have a difference in brightness or a difference in color tone, the two images obtained by imaging also have a difference in brightness or a difference in color tone. In the endoscope system of the present embodiment, even if there are differences in brightness and color tones between a plurality of images, the differences in brightness and color tones can be reduced. Therefore, it is possible to further improve the color reproducibility of the combined image.

The image synthesis processing unit 33 first compares the spatial frequency characteristic values using the two images. This comparison is made for each spatially identical pixel region in the two images. Subsequently, a pixel region having a relatively higher spatial frequency characteristic value is selected. Then, one image is generated using the selected pixel area. Thus, one composite image is generated from two images. If the difference between the spatial frequency characteristic values of the two images is small, the composite image may be generated after performing composite image processing in which each image is added with a predetermined weight.

The post-stage image processing unit 34 performs image processing such as edge enhancement and gamma correction on the composite image. The image output unit 35 outputs the processed image to the image display device 5 .

The dimming unit 36 generates a dimming signal for dimming to a reference brightness from the image read by the image reading unit 31 . The dimming signal is output to the diaphragm driver 14 of the light source device 3 . The diaphragm driver 14 adjusts the opening amount of the diaphragm 12 so as to maintain the reference brightness according to the dimming signal.

Note that the image calibration device described above may simultaneously satisfy a plurality of configurations. This arrangement is preferable for obtaining a good endoscope optical system. Moreover, the combination of preferable configurations is arbitrary.

Moreover, the present embodiment is not limited to the calibration device for an endoscope system that expands the depth of field. For example, it is also useful for an image calibration device for an optical device that performs stereoscopic viewing from a left-eye image and a right-eye image.

Although various embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and can be implemented by appropriately combining the configurations of these embodiments without departing from the spirit of the present invention. Forms are also within the scope of the present invention.

(Appendix)
The invention having the following configuration is derived from these examples.
(Appendix 1)
In order from the object side, an objective optical system, an optical path splitting unit that splits the subject image obtained by the objective optical system into two optical images with different optical path differences, and an imaging that acquires two images from the two optical images. An image calibration method for an optical device comprising:
providing a first chart, a second chart and a third chart;
irradiating the first chart, the second chart, and the third chart with illumination light;
placing the first chart, the second chart and the third chart;
a first calculation step of calculating spatial frequency characteristic values of each of the first two images obtained by imaging the first chart;
A second image obtained by imaging the second chart with the optical device in a state where the second chart is arranged at a position where the spatial frequency characteristic values of the two first images are equivalent. a second calculation step of calculating a difference in relative positional relationship between the two images;
a third calculation step of calculating, for each RGB signal, a difference in relative brightness ratio between two third images obtained by imaging the third chart illuminated by the light source with the optical device; ,
a parameter calculation step of calculating a first correction parameter for correcting the difference in the positional relationship and a second correction parameter for correcting the difference in the brightness ratio;
An image calibration method characterized by comprising:
(Appendix 2)
An endoscope system having a correction parameter storage unit storing a first correction parameter for correcting the difference in the positional relationship according to additional item 1 and a second correction parameter for correcting the difference in the brightness ratio.

As described above, according to the present invention, when combining two images in an endoscope system, image calibration is performed to reduce deterioration in image quality and artifacts (noise) of the combined image by obtaining optimum correction parameters in advance. It is useful for application devices.

L1 to L10 lens AX optical axis I image plane OBL objective optical system 1 endoscope system 4 processor device 5 image display device 100 image calibration device 101 first chart 102 second chart 103 third chart 104, 105 stage 106, 107 holder 108 calibration processor 109 storage unit 110 first calculator 111 second calculator 112 third calculator 113 parameter calculator 114 driver 115 light source 116, 116a, 116b light guide 120 optical path splitter 122 Imaging device 200 optical device

Claims (7)

In order from the object side, an objective optical system, an optical path splitting unit that splits the subject image obtained by the objective optical system into two optical images with different optical path differences, and an imaging that acquires two images from the two optical images. An image calibration device for an optical device comprising:
a first chart;
a second chart;
a third chart;
a light source that irradiates the first chart, the second chart, and the third chart with illumination light;
a stage on which the first chart, the second chart and the third chart are placed;
a calibration processor, wherein the calibration processor
a first calculator that calculates spatial frequency characteristic values of each of the first two images obtained by imaging the first chart;
The second chart is imaged by the optical device while the second chart is arranged at a position of the first chart where the spatial frequency characteristic values of the first two images are equivalent. a second calculator that calculates the difference in the relative positional relationship between the second two images obtained by
a third calculator that calculates, for each RGB signal, a difference in relative brightness ratio between two third images obtained by imaging the third chart illuminated by the light source with the optical device; ,
a parameter calculation unit that calculates a first correction parameter for correcting the difference in the positional relationship and a second correction parameter for correcting the difference in the brightness ratio;
An image calibration device comprising: The calibration processor further includes a drive unit that changes the viewing distance of the first chart in arbitrary steps by moving the stage with respect to the optical device;
a storage unit storing a still image of the first chart for each observation distance;
The first calculation unit calculates a spatial frequency characteristic value of each of the two images of the first chart calculated from the still image of the first chart stored in the storage unit, and calculates the spatial frequency characteristic value calculating an observation distance at which the characteristic value is the same for the two images of the first chart;
the stage has a holder for placing the second chart at the observation distance calculated by the first calculator;
The storage unit stores a still image of the second chart,
The parameter calculation unit calculates the first correction parameter based on the still image of the second chart and the difference in the positional relationship calculated by the second calculation unit. 2. The image calibration device according to 1. The first chart is a chart for obtaining a spatial frequency characteristic value, the first chart has a chart pattern including black and white edge lines,
3. The image calibration apparatus according to claim 2, wherein the first chart is arranged so that the center of the field of view of the two images and the black-and-white edge ridge line intersect. 2. The image calibration apparatus according to claim 1, wherein said second chart is a grayscale checker pattern in which at least two or more areas having different gradations are randomly arranged. 2. The image calibration apparatus according to claim 1, wherein the parameter calculator calculates the first correction parameter using one of the two images of the second chart as a reference. 3. The third chart is an integrating sphere, and the spectral characteristics of illumination light guided to the integrating sphere are the same as the spectral characteristics of a light source when using the optical device. 2. The image calibration device according to 1. The optical device is an endoscope system capable of switching between different observation modes, wherein illumination light from the light source is switched for each observation mode of the endoscope system,
2. The image calibration apparatus according to claim 1, wherein the parameter calculator calculates the second correction parameter for each observation mode.

Images