KR20190063027A - A method for auto-focus controlling in a line-scanning confocal microscopy and the apparatus therefor - Google Patents

Abstract

The present invention provides an autofocus detection method in a laser scanning confocal microscope using a rolling shutter sensor. The present invention reduces the complexity of the entire system by combining a part of optical systems of an autofocus detection module with a basic microscope system, and automatically analyzes only the intensity of light in an acquired image to automatically reduce a focus detection time.

Description

[0001] The present invention relates to a method and apparatus for automatic focusing of a line-scanning type confocal microscope,

The present invention relates to a method and apparatus for automatic focusing in a confocal microscope.

The conventional auto-focus controlling method used in the conventional microscope is divided into two methods. One is an image (image) analysis based autofocusing method, and the other is an automatic focusing method using a laser. An image analysis based auto focus adjustment method is a method of detecting and adjusting an optimal focus position by analyzing an image of a specimen or a specimen for observation. Since the image analysis method can perform the automatic focus adjustment only by the SW (software) algorithm including obtaining the image of the object to be measured and analyzing the contrast of the object, a separate optical system for automatic focus adjustment is required There is an advantage that it is not. In addition, there is an advantage that the error rate is very small in adjusting the focus position according to the utility degree of the image analysis algorithm. However, since the microscope is sensitive to changes in the number of micrometers, it is necessary to acquire a depth image divided into fine units in order to find an accurate focal position, to find a sample at a random position, It takes a lot of time and slows down. Further, due to the physical thickness error of the slide on which the object is placed or the plate itself, there may be a difference of up to several millimeters in the scanning area for searching for the focus position in the course of changing the sample position. Therefore, in order to compensate for this physical error, it is necessary to set the image acquisition area in the depth direction to be wide, and in this case, the auto focus adjustment becomes slower.

Another method is to construct a separate module by using a light source having a wavelength band (for example, near-infrared ray) other than the fluorescent light source used in a fluorescence microscope, and to transmit the light reflected from the object to be measured to a separate sensor It is a method to receive and analyze. For example, the laser auto-focusing method is a method in which light reflected from a target object is passed through a slit or a pin-hole, and intensity is detected by a PD (photo diode) Check. Such a laser auto focus adjustment method requires a space for configuring an autofocus module in an area outside the basic confocal microscope system space, but it does not require image analysis and can perform fast scanning, have. Further, even if the position of the plate or the stage on which the object is placed due to the external vibration is able to be scanned quickly over a wide area, the plate on which the sample including the object, which occurs up to the maximum number of millimeters, Can overcome the problem of the physical thickness error of the substrate. This method can be applied not only to the actual focal position of the object to be measured but also to the interface where the refractive index of light changes, that is, the point where the optical waveguide medium changes, for example, the interface between the sample plate or the slide glass, An interface between the sample and the sample, and an interface between the sample and the cover glass as reference positions, and has an actual focal position and an error of about 1 mm or less depending on the detected interface. Therefore, it is necessary to move the object at a predetermined offset distance from the detected interface or reference position to the focal point of the measurement object. In this case, the offset distance is an initial set value, and is given as a fixed value irrespective of the field of view or the field of view (FOV), so that a focal position error occurs depending on the distribution and shape of the object.

In recent years, a method has been proposed that combines the above-mentioned image (image) analysis-based auto focus adjustment and the laser scanning speed of the laser auto focus. For example, after the interface is first detected by the laser autofocus method, a predetermined offset is shifted from this position, and an image obtained by scanning the specific area in fine units from the shifted offset position is further analyzed to determine the focal point of the measurement object There is a method of detecting and adjusting the position. As another example, a beam pattern, in which the light of the light source for automatic focus adjustment is reflected from the object and is reflected, may be used as an imaging device in a basic microscopic configuration, for example, a CCD Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor). In this case, by analyzing only a part of the area where the beam pattern is not reflected, the scanning and analysis time of the area including the physical thickness error range of the sample plate can be reduced. Thus, although various methods have been developed to reduce the time required for autofocus in a confocal microscope, satisfactory methods and structures are still required.

Accordingly, the present inventors have endeavored to find a method for more effectively performing autofocus control in a confocal microscope, and as a result, the present invention has been completed.

Korea Patent No. 10-0602915 Korea Patent No. 10-0781095

SUMMARY OF THE INVENTION It is an object of the present invention to provide a method of automatically adjusting focus in a confocal microscope.

It is another object of the present invention to provide a confocal microscope system equipped with an autofocusing apparatus.

It is also an object of the present invention to provide a system for automatic focusing of an optical device.

In order to achieve the above object, the present invention provides an automatic focusing method using a line scanning confocal microscope, which improves the sample scanning speed of a laser scanning confocal microscope.

In one example of the present invention, the automatic focus adjustment method of the present invention is a method for adjusting a focus of a confocal microscope comprising a system constituent part including a light source part, a beam scanning part, a confocal microscope having a detection part and an objective lens part, a system control part and a signal processing part In an autofocus system, i) calculating and storing an offset value of a focus position from a reference position; (Ii) scanning the specimen with a beam in the form of a line with respect to a predetermined area based on the reference position, performing beam scanning, and outputting the image signal incident through the objective lens in a two-dimensional array Generating a scanning image by separating and aligning the generated scanning image by passing the scanning image through at least one dichroic beam splitter to generate a beam scanning pattern; Iii) acquiring an image by analyzing the separated and aligned beam scanning pattern with an image sensor synchronized with the beam scanning unit; Iv) estimating a distance to a focus position by analyzing line sharpness or line intensity or both in the acquired image and moving the objective lens unit in the Z-axis direction; And v) determining whether to move to a new field of view, and selectively performing the image acquisition and analysis repeatedly.

In another example of the present invention, the scanning of ii) is performed with a scanning mirror, and the image sensor is a camera including a rolling shutter synchronized with the scanning mirror, A confocal microscope system for eliminating depth-direction (Z-axis direction) noise is provided, and a method of adjusting the autofocus that is specialized for the present system is provided.

In another embodiment of the present invention, the object to be measured is mounted on a stage provided with a module including at least one objective lens, and the stage is moved in the z-axis direction, And a signal obtained by removing the depth direction noise is obtained through the shutter at the front end of the image sensor of the corresponding image.

In another example of the present invention, the beam scanning operation and the camera rolling shutter operation are performed in opposite directions to each other to acquire and analyze a line-shaped image at the center of the field of view (FOV) The method comprising:

The present invention also relates to a light source unit which emits a line-shaped beam; Adjusting the path of the beam emitted from the light source unit, irradiating the beam onto a specimen mounted on a stage, and outputting the image signal incident through the objective lens to a pixel in a two- A beam scanning unit configured to generate a scanning image; An objective lens for guiding light passing through the beam scanning unit to the object so that a focal point of the beam is formed on the object, and a stage on which a module including the objective lens is mounted and on which an object to be measured is mounted An objective lens unit; And a detector for analyzing the beam scanning pattern with an image sensor synchronized with the beam scanning unit to acquire and analyze an image, wherein the automatic focusing is performed.

One example of the present invention is a confocal microscope in which autofocus is performed, in which the beam in the form of a line is a laser beam and the image sensor comprises a CMOS image sensor and a rolling shutter camera / RTI >

In another example of the present invention, the light source section is provided with a light source module that provides at least one wavelength, an excitation filter, and a beam expander, in sequence.

In another embodiment of the present invention, the beam scanning unit of the present invention includes a first dichroic mirror 130 selected from a galvanometer mirror 132, a polygonal rotation mirror, and microelectro-mechanical systems (MEMS) For example, if the first dichroic mirror is a rotating polygon mirror that scans the X-axis, the second dichroic mirror is a galvanometer mirror and includes a dichroic mirror and a second dichroic mirror, - scanning the axis, wherein the autofocus is performed.

In yet another example of the present invention, the beam scanning section is designed to further include a relay optical system for guiding a beam passed through either the first dichroic mirror or the second dichroic mirror to the other .

In another example of the present invention, an autofocus system of a confocal microscope composed of an optical system component, a system controller, and a signal processor, the optical system component comprising: a light source for emitting a beam in the form of a line A light source; Adjusting a path of a beam emitted from the light source unit, irradiating the beam onto a specimen mounted on a stage, generating a beam scanning pattern from an image signal incident through the objective lens, A beam scanning unit for scanning the beam spot; An objective lens for guiding light passing through the beam scanning unit to the object so that a focal point of the beam is formed on the object, and a stage on which the object to be measured is mounted, An objective lens unit; And a detection unit including a rolling shutter camera as the image sensor; The system control unit includes a computer-readable recording medium for synchronizing the scanning speed of the line-shaped beam and the shutter speed of the rolling shutter camera so that the direction in which the pixels of the camera sensor are synchronized and the scanning direction of the line beam are reversed Wherein the signal processing unit includes a computer-readable recording medium for limiting the area of the image to be analyzed by allowing the signal to be received only at the center of the field of view (FOV), and the line sharpness or line strength And the focal distance is measured.

In the present invention, the reference position means an interface (reference position) at which the refractive index of light irradiated to an object other than the actual focal position of the measurement object changes.

In the present invention, an object to be measured is understood to have the same meaning as a measurement object, sample, or specimen, and means an object to be observed with a microscope.

In the present invention, a beam expander is a component or a device that receives incident light into the system and expands the incident beam to a larger extent upon output.

In the present invention, a beam splitter is a part or apparatus used to split an image scanned by a beam scanning unit into two beams according to a specified ratio, or to combine two different beams into a single beam.

In the present invention, the z-axis direction means a direction perpendicular to the plane on which the measurement object is placed, and means a direction parallel to the traveling direction of the beam or light incident on the object.

In the present invention, the X-axis direction and the Y-axis direction refer to axes forming a mutually perpendicular relationship on the stage surface.

In the present invention, synchronization means that two or more parts or devices are simultaneously driven and linked in real time.

In the present invention. Field of view (FOV) refers to a specific area of an object under a microscope.

The automatic focusing method of the confocal microscope according to the present invention analyzes the fixed central region of the field of view, thus shortening the time required for analysis and processing and providing a result with improved accuracy. According to the automatic focus adjustment method of the present invention, since the focus position is automatically analyzed by a simple calculation, the time required for the automatic focus adjustment depends only on the movement time of the stage on which the measurement object is placed. In addition, since the line pattern obtained by the method of the present invention is driven to be synchronized with the rolling shutter of the image sensor, the beam from the area other than the reference position is removed as noise and only the signal at the desired position is confirmed and analyzed in real time .

1 schematically shows a system configuration of the present invention composed of an optical system configuration unit, a system control unit, and a signal processing unit.
2 and 3 show the optical system constituent part of the present invention.
FIG. 4 shows a general image obtained by a line scanning system control method using a rolling shutter camera.
Figure 5 shows the acquisition of images for autofocus in the system of the present invention.
FIG. 6 shows an operation sequence of the signal processing unit of the present invention.

Hereinafter, the present invention will be described based on the drawings and embodiments. The drawings and embodiments of the present invention are not intended to be illustrative of the present invention, and the scope of the present invention is not limited thereto.

1, the system of the present invention comprises an optical system configuration unit 10, a system control unit 20, and a signal processing unit 30. The optical system configuration unit includes a light source unit, a beam scanning unit, Focus microscope. The system control unit controls the optical system configuration unit to obtain an image for autofocus analysis and adjustment, and controls the optical system configuration unit according to focal position information using an image image acquired from the optical system configuration unit. The signal processing unit analyzes the image (signal) obtained from the optical system construction unit to obtain focal position information and provides the focal position information to the system control unit. The system control unit and the signal processing unit may include a computer readable recording medium having recorded therein a system control and signal processing program, respectively.

The optical system constitution of the present invention will be described with reference to Figs. 2 and 3. Fig. In the present invention, the light source unit 150 emits the beams necessary for line-shaped beam scanning and image analysis image sensors in a confocal microscope. The light source unit 150 includes a light source module 151 that provides one or more short wavelength laser beams, an exc filter 152, and a beam expander 152 to emit, for example, expander 154 may be sequentially provided along the traveling direction of the beam. The beam scanning unit 130 of the present invention may be provided after the beam expander and the one or more short wavelength lasers emitted from the light source unit 130 may be provided with first and second dichroic mirrors 134 , 132, and the beam path can be changed to a specific direction. The first dichroic mirror and the second dichroic mirror may include a galvanometer mirror 132, a polygonal rotation mirror, an acoustic optical deflector (AOD), a digital micromirror device (DMD) And microelectro-mechanical systems (MEMS). ≪ / RTI > For example, if the first dichroic mirror is a rotating polygon mirror scanning the X-line, the second dichroic mirror may be a galvanometer mirror scanning the Y-line. The beam whose path is changed by the beam scanning unit 130 proceeds to the objective lens 142. The objective lens 142 passes through the beam scanning unit 130 so as to form a focus of the beam on a target object Refract a beam to face the object. In the present invention, the beam scanning unit 130 allows the object 141 to be scanned with a line-shaped beam while the stage 141 or the plate on which the object is mounted moves along the z-axis direction. In the present invention, the dichroic mirror 134 may be provided with means for driving it, for example, a driving unit equipped with a motor. The beam scanning unit 130 may further include a relay optical system for transmitting a beam passing through one of the dichroic mirror 134 and the galvanometer scanner 132 to the other Of course.

2 and 3, the stage 141 on which the object to be measured is placed further includes driving means capable of moving the object in the horizontal (xy-axis) and vertical (z-axis) directions . In the present invention, the driving unit may be a conveyor system including a motor or a roller, but is not limited thereto. In the present invention, as the stage moves in the z-axis direction, the distance between the objective lens 142 and the object is adjusted, so that auto focus adjustment is performed. A piezo transducer including a piezoelectric element may be further mounted for vertical movement of the stage 141 in the z-axis direction.

2 and 3, the detection unit 110 of the present invention sequentially includes a camera having an EM filter module 113, a camera barrel 112, and a rolling shutter 111. The beam or light reflected from the measurement object passes through the objective lens 142 and the beam scanning unit 130 and is transmitted to the detection unit 110. The beam spiller of the beam scanning unit 130 generates a scanned beam pattern, and the generated beam pattern is transmitted to a rolling shutter camera synchronized with the beam scanning unit through an EM filter (Emission filter) and a camera barrel, do.

The system controller of the present invention will be described with reference to Figs. 4 and 5. Fig. Figure 4 illustrates system control in a general image acquisition scheme. In FIG. 4, the number of pixels of the rolling shutter camera sensor is 2048 * 2048. The horizontal pixels are activated substantially simultaneously and are activated sequentially with a delay of the interval value set in the case of the vertical direction. In addition, the active state of the pixels per line is maintained during the set camera exposure time, and deactivation is sequentially performed at the end of the exposure time. 4, the confocal method using a rolling shutter camera is a line scanning type microscope system. In this system, a line-shaped beam is scanned in the x-axis or y-axis direction to photograph one field of view (FOV) At this time, the scanning speed of the beam is synchronized with the shutter speed (activation speed of pixels) to receive a signal only for an area where light is shining on the sample and to remove noise from the other area. As shown in Fig. 4, the slope of the pixel active region and the yellow region (line beam illumination region) of Fig. 4 varies depending on the scanning speed (= shutter speed). The pixel activation area varies according to the input parameter (see the third in the right side of Fig. 4). The speed synchronization does not match the tilt (absolute speed) equally, but is characterized in that the timing of the start and end of overlapping areas is matched (see the red dotted line in the upper right corner of Fig. 4). Therefore, the area where the two areas overlap and the actual signal is received will be the area indicated by the black dotted line. The line beam indicated by yellow has a fixed thickness (vertical direction in FIG. 4) depending on the optical system configuration. While the longitudinal direction (the horizontal direction in Fig. 4) represents the time (speed). Actually, however, as shown in FIG. 4, a plurality of rows can not exist at the same time, and a portion overlapped with an actual pixel activation region during scanning becomes an area indicated by a black dotted line (Actual signal acquisition region) in FIG. The smaller the difference between the black dotted line region and the gray region, the more pixels are activated only when there is light, and the noise removal rate (confocal effect) increases. However, the signal / noise ratio (SNR) is reduced because the intensity of the signal decreases as the overlapping region is smaller. Therefore, it is necessary to adjust the beam scanning speed and the pixel activation time appropriately. The sensor size used in FIG. 4 was 13.312 mm * 13.312 mm, and the thickness of the line beam measured was FWHM (full width harf maximum) = about 1.7 μm. The system controller shown in FIG. 4 differs in the size of pixels of an image acquired according to an optical system constituting the system. For example, when using a 20 × objective lens, the pixel size of the final image is about 0.585 μm , Which means that the line beam is simultaneously illuminating a 4-pixel line in a fixed state. In this case, when the line exposure time is set to about four times the interval time, the optimum SNR can be obtained.

Figure 5 shows a method for acquiring images for autofocus analysis in the system of the present invention. 5, by synchronizing the scanning speed of the line beam and the shutter speed to obtain the confocal effect, and by reversing the direction in which the pixels of the camera sensor are synchronized with the scanning direction of the line beam, It is possible to limit the area of the image to be analyzed. At this time, the wavelength of the light source used is the near infrared region. The obtained signal comes from the interface where the refractive index is different, and can be referred to for finding the focal position of the actual sample. The speed at which the rolling shutter operates and the speed at which the scanning mirror is synchronized are synchronized as described above, but can be received in a signal confined to the central region in the vertical direction of the pixel by being in opposite directions. At this time, the thickness of the overlapping region can be adjusted by adjusting the pixel active interval time and the exposure time. The maximum value of the area where the thickness overlaps is the thickness of the illuminating line beam, which is determined by the optical system constituting the system of the present invention. As described above, the system controller of the present invention processes a signal obtained by receiving a signal only in the central region of the field of view according to the present invention in the signal processing unit, and analyzes the intensity only without performing a complicated operation to find a focal distance . The reason why the image analysis can be performed only by the intensity is that the image acquired by the system of the present invention is a confocal image obtained by synchronizing the scanning speed and the shutter speed of the line beam. As another example of the present invention, it is also possible to obtain a line image by photographing the line beam while fixing it in the center without scanning the line beam, and to find the focus position by analyzing line sharpness and intensity.

A process of performing auto focus adjustment in the signal processing unit according to FIG. 6 will be described. According to the present invention, a system of the present invention composed of an optical system component system controller and a signal processor provides a method of autofocusing comprising the following steps:

i) calculating and storing an offset value of the focus position from a reference position;

(Ii) irradiating a line-shaped beam to a measurement region with respect to a predetermined region with reference to the reference position, detecting an image sensor from a beam scanning pattern reflected from the object, Acquiring an image through the image processing unit;

Iii) analyzing line sharpness or line intensity in the acquired image and moving the stage in the z-axis direction by estimating the focal distance; And

Iv) confirming whether the moving picture is moved to a new field of view (FOV), repeating the image acquisition and analysis after moving according to need.

According to the method of the present invention, in the image acquiring and analyzing step, the reference with the highest line sharpness can be set as the reference of the auto focus position. In the absence of beam scanning, a terret including at least one objective lens is provided, and as the stage on which the object to be measured is placed moves up and down in the z-axis direction, The camera shutter mode of the image sensor may be a global shutter (open aperture) mode.

According to another method of the present invention, the step (ii) includes the steps of using a line-shaped beam, beam scanning a target object using a scanning mirror, and moving a rolling shutter, synchronized with the beam scanning, To obtain the image to be analyzed. The step of analyzing the line sharpness or the line intensity from the image of iii) may include the step of measuring the position of the stage while moving the stage on which the object to be measured is mounted in the z- Value and the corresponding image can be acquired and analyzed at the same time. In the image acquiring and analyzing process of the present invention, the movement of the scanning mirror and the execution of the camera rolling shutter are set in directions opposite to each other, so that a line-shaped image can be obtained and analyzed from the center of the field of view. Thus, in the present invention, the camera shutter mode of the image sensor may be a rolling shutter mode synchronized with the scanning mirror. In the automatic focus adjustment process of the present invention, since the line-shaped beam pattern is synchronized with the rolling shutter and synchronized, the beam transmitted from the region other than the desired reference position is removed as noise, 3-dimensional analysis and auto-focus detection are possible.

In the method of the present invention, the process of detecting the reference position may be preceded. Specifically, first, line-shaped beam scanning is performed on the object in the z-axis direction, and at the same time, images synchronized with the object are obtained through the image sensor. The line sharpness and strength are analyzed from the obtained image, do. Next, after the stage is moved to the estimated reference position, the position value offset is calculated and stored after manually or automatically moving the stage to the focal position.

The present invention can be usefully used for auto focus detection and adjustment of confocal microscopes. Therefore, the utilization of observation equipment including confocal microscope can be increased and it can be widely used in related industries.

10: Optical system component
20:
30: Signal processor
110: detecting unit;
111: a camera including a rolling shutter;
112: tube lens (camera lens barrel); 113: EM filter
130: beam scanning unit;
131: beam shaper for line beam;
132: galvanometer scanning mirror; 133: scan lens barrel; 134: Dichroic mirror;
141: stage; 142: Objective lens

Claims (10)

A light source unit for emitting a line-shaped beam;
Adjusting a path of a beam emitted from the light source unit, irradiating the beam onto a specimen mounted on a stage, generating a beam scanning pattern from an image signal incident through the objective lens, A beam scanning unit for scanning the beam spot;
An objective lens for guiding light passing through the beam scanning unit to the object so that a focal point of the beam is formed on the object, and a stage on which the object to be measured is mounted, An objective lens unit; And
And a detector for acquiring and analyzing the image by analyzing the beam scanning pattern with an image sensor synchronized with the beam scanning unit.
Confocal microscope where autofocus is performed.  The confocal microscope according to claim 1, wherein the line-shaped beam is a laser beam and the image sensor comprises a CMOS image sensor and a rolling shutter camera. 3. The apparatus according to claim 1 or 2, wherein the light source section is provided sequentially with a light source module providing at least one wavelength, an excitation filter, a beam expander and a beam splitter, Confocal microscope. The confocal microscope according to claim 3, wherein the beam scanning unit includes a first dichroic mirror and a second dichroic mirror. 5. The apparatus according to claim 4, wherein the beam scanning section further comprises a relay optical system for guiding a beam passed through either the first dichroic mirror or the second dichroic mirror to the other, This is a confocal microscope being performed. In an autofocus microscope autofocus system composed of a light source system constituent part including a light source part, a beam scanning part, a confocal microscope composed of a detection part and an objective lens part, a system control part and a signal processing part,
i) calculating and storing an offset value of the focus position from a reference position;
Ii) generating a beam scanning pattern from an image signal incident through the objective lens by irradiating the line-shaped beam with respect to the reference position on the measurement object to perform beam scanning;
Iii) analyzing the beam scanning pattern with an image sensor synchronized with the beam scanning unit to acquire an image;
Iv) estimating a distance to a focus position by analyzing line sharpness or line intensity or both in the acquired image and moving the objective lens unit in the Z-axis direction; And
V) determining whether to move to a new field of view, and repeatedly performing said image acquisition and analysis;
A method for automatic focusing of a confocal microscope. 7. The method of claim 6, wherein the scanning of ii) is performed with a scanning mirror, wherein the image sensor comprises a rolling shutter camera synchronized with the scanning mirror. Focus adjustment method. 8. The method according to claim 7, wherein the object is mounted on a stage having at least one objective lens, and moving the stage in the Z-axis direction causes an image corresponding to the stage position Is obtained simultaneously with the beam scanning. 9. The method of claim 8, wherein the beam scanning movement by the scanning mirror and the camera rolling shutter movement are performed in opposite directions to each other to acquire and analyze a line-shaped image at the center of the field of view, Focus adjustment method. An autofocus system of a confocal microscope comprising an optical system component, a system controller and a signal processor,
The optical system constructing unit,
A light source unit for emitting a line-shaped beam;
Adjusting a path of a beam emitted from the light source unit, irradiating the beam onto a specimen mounted on a stage, generating a beam scanning pattern from an image signal incident through the objective lens, A beam scanning unit for scanning the beam spot;
An objective lens for guiding light passing through the beam scanning unit to the object so that a focal point of the beam is formed on the object, and a stage on which the object to be measured is mounted, An objective lens unit; And
And a detection unit including a rolling shutter camera as the image sensor;
The system control unit includes:
A computer-readable recording medium which synchronizes the scanning speed of the line-shaped beam with the shutter speed of the rolling shutter camera and controls the direction in which the pixels of the camera sensor are synchronized and the scanning direction of the line beam to be reversed, FOV) to limit the area of the image to be analyzed,
The signal processing unit,
And a computer-readable recording medium, wherein the focal length is measured by analyzing the line sharpness or line strength of the obtained image,
Focusing system of confocal microscope.

Images