JP2020124569A - Image processing apparatus, image processing method and program - Google Patents

Abstract

To provide a mechanism which can efficiently perform comparison with time in the fundus oculi of an eye to be examined.SOLUTION: An image processing apparatus includes: an image acquisition part 110 which acquires the first image obtained by imaging a region in the prescribed size in the fundus oculi of an eye to be examined and the second image obtained by imaging a region narrower than the region in the prescribed region in association with each other for each set based on the imaging timing; a positioning part 141 which performs the first positioning being the positioning between the first image and the second image constituting the same set and the second positioning being the positioning between the first images included in the different sets to generate position information related to the positioning; and an overlapping part 142 which generates an overlapped image by overlapping the second image included in the first set and the second image included in the second set acquired after the first set with the first image included in the first set or the first image included in the second set on the basis of the position information generated by the positioning part 141.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image processing apparatus and an image processing method for processing an image of a fundus of an eye to be inspected over time, and a program for causing a computer to execute the image processing method.

The photoreceptor cells of the retina located at the fundus of the eye to be examined play an important role in the visual function, which is considered to be an important function among the human five senses. In recent years, by applying aberration correction technology, a technology for depicting the fundus retina in high definition has been constructed, and it is expected to discover new knowledge about visual function and build diagnostic value.

An adaptive optics scanning laser ophthalmoscope (hereinafter, referred to as “AO-SLO”) device for adaptive optics is a telescope technique that acquires a clear star image by compensating for atmospheric fluctuations. It is an ophthalmoscope applied to the eye. The technique of this AO-SLO device has made it possible to resolve each photoreceptor cell of the retina of the eye to be examined. It is said that photoreceptor cells are present at approximately 2 μm intervals in the vicinity of the fovea, which has the highest density. Currently, development is being continued to improve the resolution for the purpose of imaging even the photoreceptor cells of the fovea.

In order to link the visualized photoreceptor cells to clinical value, there is a demand for a technique capable of observing photoreceptor cells in the same region over time and quantitatively analyzing the change. An image captured by an AO-SLO device (hereinafter referred to as "AO-SLO image") often has a small angle of view (a narrow imaging range) because of its large resolution. Therefore, in order to perform comparison over time, it is necessary to first acquire a plurality of AO-SLO images in which the same position is captured, and further perform detailed alignment between the acquired images. As a conventional technique related to this, for example, there is a technique disclosed in Patent Document 1.

Specifically, in Patent Document 1, in order to compare the tomographic images of the retina by OCT (Optical Coherence Tomography) with each other over time, a surface image of the fundus (fundus image) is acquired at the same time as the tomographic images, and first, the surface images are compared with each other. Is described, and thereafter, the tomographic images associated with the respective surface images are associated with each other and the temporal comparison is performed.

JP, 2007-252692, A

However, the technique described in Patent Document 1 has a problem in that it is necessary to first specify surface images to be aligned with each other, and thus it is not possible to efficiently perform temporal comparison on the fundus of the eye to be inspected.

The present invention has been made in view of such problems, and an object thereof is to provide a mechanism capable of efficiently performing temporal comparison on the fundus of the eye to be inspected.

The image processing apparatus of the present invention is a first image obtained by photographing an area of a predetermined size on the fundus of the eye to be inspected, and an area of the eye fundus of the eye to be inspected, the area having the predetermined size. Image acquisition means for acquiring a second image obtained by photographing a smaller area than each other in association with each set based on the photographing time, and the first image and the above-mentioned first image forming the same set. The first alignment, which is the alignment with the second image, and the second alignment, which is the alignment of each of the first images included in the different sets, are performed to perform the first alignment. Of the first set, which is the previously acquired set, on the basis of the position alignment unit that generates position information related to the position alignment and the second position alignment, and the position information generated by the position alignment unit. The second image included and the second image included in the second set, which is the set acquired after the first set, are set to the first image included in the first set. A superimposing unit that superimposes the image or the first image included in the second set to generate a superimposed image.
The present invention also includes an image processing method by the above-described image processing apparatus, and a program for causing a computer to execute the image processing method.

According to the present invention, the temporal comparison of the fundus of the eye to be inspected can be efficiently performed.

It is a block diagram which shows an example of schematic structure of the ophthalmologic apparatus containing the image processing apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows the 1st Embodiment of this invention and which shows typically several AO-SLO images and WF-SLO images produced|generated by the imaging device of FIG. It is a figure which shows the 1st Embodiment of this invention and shows the fixation lamp map for operating the lighting position of the fixation lamp in the imaging device of FIG. 6 is a flowchart showing an example of a processing procedure of an image processing method by the image processing apparatus according to the first embodiment of the present invention. 6 is a flowchart showing an example of a detailed processing procedure of in-set position alignment processing in step S430 of FIG. 4. FIG. 5 is a diagram showing the first embodiment of the present invention and schematically showing an example of a superimposed image generated in step S450 of FIG. 4. It is a figure which shows the 1st Embodiment of this invention and which shows typically the example of a display of the time-aligned analysis target AO-SLO image group etc.. 9 is a flowchart showing an example of a processing procedure of an image processing method by the image processing apparatus according to the second embodiment of the present invention.

Hereinafter, modes (embodiments) for carrying out the present invention will be described with reference to the drawings.

(First embodiment)
First, a first embodiment of the present invention will be described.

In the first embodiment, a method of acquiring an image of a retina located on the fundus of the eye to be inspected over time by an imaging device and then performing alignment between two or more images capturing the same position explain. Specifically, each AO-SLO image is aligned with a wide-angle SLO image (hereinafter referred to as “WF-SLO image”) acquired as the same set (for example, taken on the same day). To do. Further, each of the images included in different sets (for example, sets acquired on different days) is aligned. Then, based on the position information of these images, a plurality of AO-SLO images acquired as different sets (for example, captured on different days) are selected from among a plurality of WF-SLO images acquired as different sets. The superimposed image is generated by superimposing it on one WF-SLO image.

Then, in the first embodiment, when the temporal evaluation area is selected as an area to be analyzed from the superimposed image, all AO-SLO images including the temporal evaluation area are displayed. Then, when an image group to be compared in detail is selected from the displayed AO-SLO image group, more precise detailed alignment is performed between the selected images.

Thus, in the first embodiment, by superimposing the AO-SLO image groups acquired as different sets (acquired at different times) on one WF-SLO image to generate a superimposed image, The temporal comparison of the fundus of the eye to be inspected can be efficiently performed. Further, it becomes possible to select a temporal evaluation region from the superimposed images, and accordingly, it is possible to efficiently select images to be compared. Further, by performing the detailed registration only for the image group including the temporal evaluation region, it is possible to reduce the error in the registration processing.

[Schematic configuration of ophthalmic device]
FIG. 1 is a block diagram showing an example of a schematic configuration of an ophthalmologic apparatus 10 including an image processing apparatus 100 according to the first embodiment of the present invention.
As shown in FIG. 1, the ophthalmologic apparatus 10 according to the present embodiment includes an image processing apparatus 100, a photographing apparatus (for example, AO-SLO apparatus) 200, an information input apparatus 300, a display apparatus 400, and a database 500. It is configured.

The imaging device 200 is a device that images a retina located on the fundus of the eye to be inspected and generates a planar image such as an AO-SLO image or a WF-SLO image with a wide angle of view. Here, the WF-SLO image corresponds to a first image obtained by photographing an area of a predetermined size on the fundus of the eye to be inspected. Further, the AO-SLO image corresponds to a second image obtained by photographing a region that is a region of the fundus of the eye to be inspected and is narrower than the region of the predetermined size.

The information input device 300 is a device for inputting various information to the image processing device 100.

The image processing apparatus 100 is an apparatus that performs image processing on a planar image such as an AO-SLO image or a WF-SLO image obtained by imaging the retina located on the fundus of the eye to be inspected by the imaging device 200. As shown in FIG. 1, the image processing apparatus 100 is configured to include an image acquisition unit 110, an information acquisition unit 120, a control unit 130, an image processing unit 140, a storage unit 150, and an output unit 160. ..

The image acquisition unit 110 performs a process of acquiring a planar image such as an AO-SLO image or a WF-SLO image obtained by imaging the retina located on the fundus of the eye to be inspected by the imaging device 200. Here, in addition to the case where the plane image is directly acquired from the photographing apparatus 200, the case where the plane image previously obtained by the photographing apparatus 200 and stored in the database 500 after image processing is obtained from the database 500 is also available. Including. For example, the image acquisition unit 110 is a WF-SLO image (first image) obtained by capturing an area of a predetermined size in the fundus of the eye to be inspected, and the area in the fundus of the eye to be inspected. An AO-SLO image (second image) obtained by photographing an area narrower than a predetermined size is acquired in association with each set based on the photographing time.

The information acquisition unit 120 performs a process of acquiring various types of information from the image capturing device 200 and the information input device 300. For example, the information acquisition unit 120 acquires information regarding the eye to be inspected from the imaging device 200, or acquires input information of the examiner from the information input device 300. Further, the information acquisition unit 120 performs a process of acquiring a temporal evaluation area, which is an area in which temporal evaluation is performed, from the superimposed image generated by the superimposing unit 142 described below. The information acquisition unit 120 that performs the process of acquiring the temporal evaluation area constitutes an area acquisition unit.

The control unit 130 centrally controls the operation of the image processing apparatus 100.

The image processing unit 140 processes the planar image such as the AO-SLO image or the WF-SLO image acquired by the image acquisition unit 110 under the control of the control unit 130. As shown in FIG. 1, the image processing unit 140 includes a position alignment unit 141, a superposition unit 142, an image selection unit 143, a region extraction unit 144, and an analysis unit 145.

The alignment unit 141 performs alignment between the frames of the AO-SLO image and the WF-SLO image acquired by the image acquisition unit 110, the AO-SLO image and the WF-SLO image, and the WF acquired over time. -Alignment processing is performed between SLO images and between AO-SLO images. When the various alignment processes are performed, the alignment unit 141 generates position information related to the various alignments. For example, the alignment unit 141 performs the first alignment that is the alignment of the WF-SLO images and the AO-SLO images that configure the same set, and the WF-SLO images that are included in the different sets. The second alignment, which is alignment, is performed. Then, the alignment unit 141 generates position information related to the first alignment and the second alignment. Further, the alignment unit 141 further performs the third alignment, which is the detailed alignment in the group of AO-SLO images selected by the image selection unit 143 described later, and outputs the position information related to the third alignment. To generate.

The superimposing unit 142 performs a process of superimposing the plurality of AO-SLO images associated by the alignment unit 141 on the reference WF-SLO image to generate a superimposed image. For example, the superimposing unit 142 is acquired after the AO-SLO image and the first set included in the first set, which is the previously acquired set, based on the position information generated by the position adjusting unit 141. And a AO-SLO image included in the second set, which is a second set, are superimposed on the WF-SLO image included in the first set or the WF-SLO image included in the second set to generate a superimposed image. To do.

The image selection unit 143 performs an AO-SLO image that performs detailed alignment (detailed temporal analysis) from the group of AO-SLO images including the temporal evaluation region acquired from the superimposed image by the information acquisition unit 120. The process of selecting the group of is performed.

The area extraction unit 144 extracts a common area which is an area (area with overlap) commonly included in the group of AO-SLO images subjected to the third alignment, which is the detailed alignment by the alignment unit 141. Perform processing to

The analysis unit 145 performs analysis processing such as photoreceptor cell extraction of the eye to be inspected on the AO-SLO image. Furthermore, the analysis unit 145 performs analysis processing such as photoreceptor cell extraction of the eye to be inspected and temporal comparison on the common region extracted by the region extraction unit 144.

The storage unit 150 stores programs and various information necessary for the control unit 130 to perform processing. Further, the storage unit 150, under the control of the control unit 130, obtains the planar image acquired by the image acquisition unit 110, various information acquired by the information acquisition unit 120, and further the result of the processing of the image processing unit 140. Stores various information obtained.

Based on the control of the control unit 130, the output unit 160 has obtained the planar image acquired by the image acquisition unit 110, various information acquired by the information acquisition unit 120, and the result of the processing of the image processing unit 140. Various information and the like are output to the display device 400 for display and also output to the database 500 for storage.

The display device 400 performs a process of displaying a plane image output from the output unit 160, various types of information, and the like.

The database 500 performs a process of saving the planar image output from the output unit 160, various information, and the like.

[Plane image]
FIG. 2 shows the first embodiment of the present invention, and is a diagram schematically showing a plurality of AO-SLO images and WF-SLO images generated by the image capturing apparatus 200 of FIG. In the imaging device 200, different positions of the retina located on the fundus of the eye to be inspected can be imaged by changing the lighting position of the fixation lamp and taking an image of the eye to be inspected in a different state.

Here, FIG. 2 shows a WF-SLO image 210 having a wide angle of view and a plurality of AO-SLO images 220 generated by the image capturing apparatus 200. A set of the WF-SLO image 210 and the AO-SLO image 220 as shown in FIG. 2 is acquired by one shooting. In the present embodiment, by observing the same eye to be examined with time, a set as shown in FIG. 2 is acquired multiple times.

FIG. 3 shows the first embodiment of the present invention, and is a diagram showing a fixation lamp map for operating the lighting position of the fixation lamp in the image capturing apparatus 200 of FIG.
First, the image capturing apparatus 200 turns on the fixation lamp with the center position of the fixation lamp map shown in FIG. 3 selected. At this time, if the eye to be inspected that gazes at the presented fixation lamp is imaged, it is possible to image the vicinity of the macula of the eye to be inspected.

Here, the WF-SLO image 210 shown in FIG. 2 is an optical system different from the case of acquiring an AO-SLO image in the image capturing apparatus 200, and has a low resolution and a wide angle of view acquired without using an adaptive optical system. It is an image. The WF-SLO image 210 is an image for obtaining an entire image of the retina by photographing a wide area of the retina located at the fundus of the eye to be inspected.

As shown in FIG. 2, by aligning the AO-SLO image 220 having a narrow angle of view with the WF-SLO image 210 and associating them with each other, the position of the AO-SLO image 220 having a narrow angle of view in the entire retina of the eye to be inspected. Is shown. Hereinafter, in the present embodiment, the WF-SLO image 210 is assumed to have an image size of 8 mm×6 mm and a pixel size of 533 pixels×400 pixels. In addition, the AO-SLO image 220 has three types of shooting area sizes of 1.7 mm×1.7 mm, 0.82 mm×0.82 mm, and 0.34 mm×0.34 mm, and the pixel sizes are all common. There are three types of resolution, 400 pixels x 400 pixels. Here, the AO-SLO image in which the size of the imaging region is 1.7 mm×1.7 mm is the AO-SLO image 220-L having the L angle of view. Further, an AO-SLO image in which the size of the imaging region is 0.82 mm×0.82 mm is set as an AO-SLO image 220-M having an M field angle. In addition, an AO-SLO image in which the size of the shooting region is 0.34 mm×0.34 mm is referred to as an AO-SLO image 220-S having an S angle of view.

Further, the WF-SLO image 210 and the AO-SLO image 220 described here are acquired as moving images including a plurality of frames, and the number of frames configured by the frame rate and the shooting time changes. In this embodiment, the AO-SLO image 220 has a frame rate of 32 frames per second and a shooting time of 1 second, and is composed of 32 frames. The WF-SLO image 210 has a frame rate of 14 frames per second and a shooting time of 1 second, and is composed of 14 frames. In the photoreceptor cell analysis described below, the AO-SLO image 220-S with the S angle of view is set as the analysis target. In addition, the WF-SLO image 210 and the AO-SLO image 220 are referred to as plane images.

[Shooting protocol]
The imaging protocol of the eye to be inspected varies depending on the disease or the like to be noticed in the eye to be inspected, but as the standard protocol, first, the WF-SLO image 210 centering on the macula is imaged, and then a plurality of positions of the retina are different in resolution. Photographing is performed while combining the AO-SLO images 220. In addition, when observing the same eye to be examined over time, it is common to take an image using the same protocol, but multiple images are taken while gradually changing the position only around the diseased part of interest. In some cases.

[Processing Procedure of Image Processing Device 100]
FIG. 4 is a flowchart showing an example of a processing procedure of an image processing method by the image processing apparatus 100 according to the first embodiment of the present invention.

<Step S410>
First, in step S410, the information acquisition unit 120 acquires the information on the eye to be inspected as the patient information, which is input from the information input device 300. Then, the information acquisition unit 120 stores the acquired information about the eye to be inspected in the storage unit 150 via the control unit 130. Here, the information on the eye to be inspected includes information on the person to be inspected such as the ID of the person to be inspected and the date of birth, information on measurement data such as the axial length of the eye to be inspected, and images taken in the past. Information is included.

<Step S420>
Subsequently, in step S420, the image acquisition unit 110 acquires, over time, the planar image of the retina located on the fundus of the eye to be inspected (WF-SLO image 210 and AO-SLO image) from the imaging apparatus 200 or the database 500. 220) to obtain multiple sets. Then, the image acquisition unit 110 stores the plurality of sets in the acquired planar image in the storage unit 150 via the control unit 130. Here, in the present embodiment, a set of planar images means, for example, a WF-SLO image 210 and an AO-SLO image 210 taken on the same day when the same eye is imaged over time. An image group composed of the images 220 will be referred to. However, for example, in the case where images are taken before and after the treatment for comparison, the images taken on the same day may be treated as different sets before and after the treatment.

<Step S430>
Subsequently, in step S430, for each set of the plurality of sets in the plane image acquired in step S420, the alignment unit 141 sets the WF-SLO image 210 and the AO-SLO image 220 included in the same set. Align the. At this time, the alignment unit 141 generates position information regarding alignment in each set. The in-set alignment in step S430 corresponds to the first alignment, and the detailed processing of step S430 will be described below with reference to FIG.

FIG. 5 is a flowchart showing an example of a detailed processing procedure of the in-set alignment processing in step S430 of FIG.

<<Step S431>>
First, in step S431, the alignment unit 141 performs inter-frame alignment of the AO-SLO image or the WF-SLO image. As described above, the AO-SLO image is composed of 32 images and the WF-SLO image is composed of 14 images. These images are taken at the same position on the retina of the eye to be inspected, but there are displacements and intra-image distortions due to involuntary eye movements. Therefore, it is necessary to align the frames to correct the distortion and the positional deviation. Such a positional shift or distortion within a frame has a greater effect on an AO-SLO image having a narrower shooting range than on a WF-SLO image which shoots a wide range. Therefore, the processing procedure for the AO-SLO image will be described.

First, the frame with the least distortion is selected from the 32 frames as a reference frame, and then the other frames are aligned with the reference frame. This reference frame can be selected by the user or automatically by using the image quality index or the like. Although a plurality of methods are known as methods for aligning the reference frame with the other frames, the alignment using the phase-only correlation method is used here. In many cases, the AO-SLO image obtained by photographing the retina does not include a feature such as a landmark in the image. Therefore, it is necessary to use a method that enables highly accurate alignment even from the arrangement information of photoreceptor cells. Because there is.

Specifically, the alignment unit 141 selects one frame other than the reference frame (hereinafter, the selected frame is referred to as “current frame”), and uses the phase-only correlation method for the entire reference frame and current frame. Then, the parallel movement amount is calculated. Next, the alignment unit 141 obtains an area where the reference frame and the current frame overlap in the state of parallel movement, and sets a plurality of small areas in the area. Here, the alignment unit 141 sets a total of 36 small regions of 64 pixels×64 pixels, 6 in each of the vertical and horizontal directions. Then, the alignment unit 141 uses the phase-only correlation method again with respect to the images included in the reference frame and the current frame corresponding to each small area, and obtains the positional deviation (parallel movement amount) between the small areas. .. Then, the alignment unit 141 finds the affine transformation that optimizes the parallel movement at 36 locations, and applies it to the current frame. Further, the alignment unit 141 also finely corrects the deformation of each small area obtained by the affine transformation and the parallel movement amount obtained above if there is a difference, and corrects the position based on the obtained information. The generated image. Then, the alignment unit 141 performs the above-described processing on all 31 frames other than the reference frame, so that the AO-SLO image aligned with the reference frame (hereinafter, referred to as “aligned AO-SLO image”). Get). Further, the alignment unit 141 also acquires the aligned WF-SLO image for the WF-SLO image by the same method as the AO-SLO image.

<<Step S432>>
Subsequently, in step S432, the alignment unit 141 generates a superimposed image using the aligned AO-SLO image (hereinafter, referred to as “superimposed AO-SLO image”). Specifically, the alignment unit 141 accumulates the pixel value of each corresponding frame with respect to each pixel (400 pixels×400 pixels) of the reference frame, and acquires an average value. Here, the alignment unit 141 obtains an average value only for pixels having a corresponding pixel value, because there is a frame in which a corresponding pixel does not exist for a certain pixel due to a positional shift. Further, the alignment unit 141 also generates a superimposed WF-SLO image for the WF-SLO image by the same method as that for the AO-SLO image.

<<Step S433>>
Subsequently, in step S433, the alignment unit 141 performs alignment between the superimposed WF-SLO image and the superimposed AO-SLO image of the same set acquired in step S432. Then, the position alignment unit 141 acquires position information regarding the position of each AO-SLO image on the WF-SLO image.

Unlike the inter-frame alignment in step S431, alignment between images having different resolutions is required here. In this case, since the images with similar resolutions are more accurate in the alignment, first, the overlay WF-SLO image and the overlay AO-SLO image with the lowest resolution L-view angle are aligned. , The position information corresponding to the WF-SLO image of the AO-SLO image with the L angle of view is acquired. After that, a composite image in which the superimposed AO-SLO image having the L field angle is fitted to the corresponding position of the superimposed WF-SLO image is generated.

When there are a plurality of WF-SLO images in the same set, the alignment unit 141 determines a reference WF-SLO image (hereinafter, referred to as “reference WF-SLO image”) and superimposes them. The position of the superposed AO-SLO image with the L angle of view is aligned with the WF-SLO image. Then, the alignment unit 141 aligns the superposed AO-SLO images of all L angles of view with the composite image generated in the previous step, and fits the composite image in the corresponding position to update the composite image. When the association of the superposed AO-SLO images with the L field angle is completed, the superposed AO-SLO image with the next lower resolution M field angle is processed as in the case of the L field angle to update the composite image. .. After that, the registration of the superimposed AO-SLO images of the S angle of view is performed in the same manner.

Although there are a plurality of methods for alignment, alignment is performed also in step S433 using the phase-only correlation method as in step S431. However, all planar images (superimposed WF-SLO image and superposed AO-SLO image of L-field angle, M-field angle, and S-field angle) are L-sized in order to correspond to alignment between images having different resolutions. The resolution is 4.25 μ/pixel, which is the same as the angle of view. Specifically, in the case of the superposed AO-SLO image of the M field angle and the S field of view, downsampling is performed to 2.07 times and 5 times respectively, and in the case of the superposed WF-SLO image, it is approximately 3.5 times. Upsample to. Further, in order to adjust the difference in contrast depending on the angle of view, preprocessing by a low-pass filter or a high-pass filter is performed.

The processes in steps S431 to S433 described above are performed for all the sets acquired in step S420 to perform alignment within each set, and the process in step S430 in FIG. 7 ends.

Here, a case has been described in which a plurality of sets of planar images are acquired when performing temporal comparison, and alignment is performed within each set. However, in actuality, when each set is acquired, alignment between images may be performed for the purpose of browsing and analyzing images in the same set. In that case, since the information is stored in the database 500 in the state where the process of step S433 is completed, the information of the alignment performed in step S430 is also acquired when the set of the planar images is acquired in step S420. The processing of step S430 is not performed.

<Step S440>
Subsequently, in step S440, the alignment unit 141 aligns the reference WF-SLO images of each set in order to align the plurality of sets in the plane image acquired in step S420. At this time, the alignment unit 141 generates position information regarding alignment of the reference WF-SLO images of each set. The inter-set alignment in step S440 corresponds to the second alignment.

Here, the alignment unit 141 selects a reference set from a plurality of sets acquired over time for the same eye to be examined (hereinafter, the selected set is referred to as a “baseline set”). Referred to). This baseline set is a reference set for comparison with time, and may select the newest set or the oldest set of the planar images acquired for the same eye to be examined. In some cases. Alternatively, the user can make a selection according to the progress of a disease or the like.

The alignment of the reference WF-SLO images described above is performed by comparing the reference WF-SLO images included in the baseline set (hereinafter, referred to as "baseline WF-SLO image") with the reference WF- of each planar image set. This is performed by acquiring the positional relationship of the SLO image. Specifically, the alignment unit 141 cuts out a region of 400 pixels×400 pixels from each of the baseline WF-SLO image and one set of reference WF-SLO images, and calculates the average value of the luminance histogram of each image. After adjusting the contrast so that the variances are the same, the parallel movement amount is acquired using the phase-only correlation method. Then, the alignment unit 141 performs the same processing on the reference WF-SLO images of all the planar image sets, and acquires the relative position (parallel movement amount) of each set with respect to the baseline set. Then, the alignment unit 141 stores the acquired relative positions of the plurality of sets in the storage unit 150 via the control unit 130.

<Step S450>
Subsequently, in step S450, the superimposing unit 142 causes the result of the alignment (first alignment) in the same set in step S430 and the alignment between the plurality of sets in step S440 (second alignment). Based on the result of (3), all AO-SLO images are superimposed on the baseline WF-SLO to generate a superimposed image. Then, the control unit 130 controls the display device to display the superimposed image generated by the superimposing unit 142 via the output unit 160.

Here, it is assumed that the superimposed image is generated using only the image of the view angle that is the target of the photoreceptor cell analysis. In the present embodiment, only the S angle of view is the analysis target, so a case will be described in which a superimposed image is generated using only the S angle of view.

First, the superimposing unit 142, for the AO-SLO image of the S angle of view included in the baseline set of the plurality of sets of planar images acquired in step S420, calculates each of the superimposed AO-SLO images obtained in step S433. The relative position information of the image with respect to the baseline WF-SLO image is acquired. Here, in the baseline set, since the reference WF-SLO image in step S433 matches the baseline WF-SLO image, the position information acquired in step S433 can be used as it is.

Next, the superimposing unit 142 acquires the relative position information acquired in step S440 with respect to the AO-SLO image having the S angle of view included in a set other than the baseline set among the plurality of sets of planar images acquired in step S420. Then, the relative position information acquired in step S433 is acquired. Specifically, the superimposing unit 142 acquires relative position information between the baseline WF-SLO image and the reference WF-SLO images of each set in step S440, and the reference WF-SLO image and each superimposed AO-SLO image are acquired. The relative position information of is acquired in step S443. Then, the superimposing unit 142 acquires relative position information between the baseline WF-SLO image and each superimposed AO-SLO image using these relative position information. In the present embodiment, only the parallel movement position is acquired as the relative position between the baseline WF-SLO image and the reference WF-SLO image of each set, and the parameters such as rotation and shear acquired from the affine transformation are not considered. However, these may be taken into consideration.

Next, the superimposing unit 142 changes the pixel value, for example, by reflecting the amount of overlapping images, based on the position information of the superposed AO-SLO images of all S field angles of interest with respect to the baseline WF-SLO images. The superimposed image thus generated is generated. Specifically, when generating the superimposed image, the superimposing unit 142 first obtains an area on the baseline WF-SLO image in which the image exists from the position information of the superimposed AO-SLO image having the S field angle, For each pixel, the number of AO-SLO images superimposed on it is acquired. Next, the luminance value of the baseline WF-SLO image is converted for display. Although there are a plurality of conversion methods, here, the maximum brightness value and the minimum brightness value of the baseline WF-SLO image are acquired, and the difference is divided into 255 to set the pixel depth to 8 bits. When such conversion is performed, the minimum luminance value of the baseline WF-SLO image is 0 and the maximum luminance value is 255. Then, the superimposing unit 142 sets the converted luminance value of the corresponding pixel of the baseline WF-SLO image when the pixel value is 0 (there is no superimposed AO-SLO image), and the pixel value is 0. In other cases, a brightness value that reflects the number of superimposed images is set. For example, when the number of superposed AO-SLO images of all S field angles is N, and the number of superposed AO-SLO images of S field angle present in a certain pixel is M, the superposition unit 142 determines that The brightness value I is set as in the following expression (1).

Here, various methods can be considered for determining the value of I. For example, N may be the maximum value of the number of AO-SLO images to be superimposed on the pixel, instead of the number of superimposed AO-SLO images of all S field angles. In addition, a method in which N is set to a predetermined number (for example, 30) and I is set to depth when M is larger than N is also considered.

FIG. 6 shows the first embodiment of the present invention, and is a diagram schematically showing an example of the superimposed image 600 generated in step S450 of FIG. In the superimposed image 600 shown in FIG. 6, an example in which only the number of superimposed AO-SLO images is reflected in gray scale is shown, but various methods other than this method can be considered. For example, it is possible to display in color the large number of superimposed AO-SLO images. Further, for example, it is possible to change the color for each set including the superimposed AO-SLO image and generate the superimposed image.

Then, the control unit 130 controls the storage of the superimposed image generated by the superimposing unit 142 in the storage unit 150 and the control of displaying the superimposed image on the external display device 400 via the output unit 160.

<Step S460>
Subsequently, in step S460, the information acquisition unit 120 performs a process of acquiring the temporal evaluation area selected by the user from the information input device 300. Then, the information acquisition unit 120 stores the acquired temporal evaluation area in the storage unit 150 via the control unit 130. Here, the temporal evaluation region is, for example, a region selected by the user on the superimposed image displayed on the display device 400 in step S450 as a region to be temporally evaluated using a mouse or the like. More specifically, the user can click one point on the superimposed image displayed on the display device 400 to obtain the coordinate, or specify a region on the superimposed image and select it. It is also possible to do so. Furthermore, it is also possible to automatically acquire the temporal evaluation region from the superimposed image without depending on the user's input. For example, the information acquisition unit 120 acquires, as the temporal evaluation area, the area with the largest number of superimposed images on the superimposed image, or acquires the area with the largest number of superimposed images belonging to different sets of planar images as the temporal evaluation area. It is also possible to do so. Further, the information acquisition unit 120 weights the image quality such as the contrast of each image when counting the number of images in such a manner, and acquires an area with a large number of images whose image quality is higher than a predetermined threshold value as a temporal evaluation area. It is also possible to do so.

<Step S470>
Subsequently, in step S470, the image selection unit 143 selects the overlay AO-SLO including the temporal evaluation region acquired in step S460 from the overlay AO-SLO images of all S field angles generated in step S432. Select an image. After that, the control unit 130 controls to display the list of the group of the superimposed AO-SLO images selected by the image selection unit 143 on the external display device 400 through the output unit 160.

Further, the image selecting unit 143 selects an image group (hereinafter, referred to as “time-dependent analysis target”) from among the group of the superimposed AO-SLO images including the time-dependent evaluation area acquired in step S460, for performing the detailed alignment for the time-dependent analysis. "AO-SLO image group"). This selection can be selected by the user or can be automatically selected by the image selection unit 143 according to an algorithm. For example, when the user makes a selection, the user selects the AO-SLO image group for temporal analysis from the list of all superimposed AO-SLO images including the displayed temporal evaluation region in consideration of the shooting date, image quality, and the like. The mode is adopted. In addition, when the image selection unit 143 automatically selects, an image with the highest image quality is selected for each shooting date, or an image including the temporal evaluation area acquired in step S460 in the center is selected for each shooting date. A mode of selecting as an image is adopted. Then, the image selection unit 143 stores the selected temporal analysis target AO-SLO image group in the storage unit 150 via the control unit 130.

<Step S480>
Subsequently, in step S480, the alignment unit 141 performs detailed alignment between the temporal analysis target AO-SLO image groups selected in step S470. At this time, the alignment unit 141 generates position information regarding alignment of the AO-SLO image group for temporal analysis. The detailed alignment in step S480 corresponds to the third alignment.

Here, a case where the phase-only correlation method similar to the in-set registration in step S430 is used as a detailed registration method will be described. However, there are a plurality of methods for alignment, and the method is not limited to this method.

Specifically, in step S480, the alignment unit 141 first selects a reference image (hereinafter, referred to as “reference AO-SLO image”) from the temporal analysis target AO-SLO image group. The reference AO-SLO image may be selected according to the object that the user wants to observe, or may be automatically selected by software. As a specific example of the automatic selection, an image having an image quality higher than a predetermined threshold such as a high contrast is selected, and an image including the temporal evaluation region acquired in step S460 in the center is selected.

Next, the alignment unit 141 aligns images other than the reference AO-SLO image included in the temporal analysis target AO-SLO image group with the reference AO-SLO image. Here, the same method as the inter-frame registration in step S431 is used, but pre-processing for adjusting the image between the temporal analysis target AO-SLO images is performed. The reason is that in the case of inter-frame alignment, the properties of the images are similar due to the alignment of the 32 frames acquired in 1 second, whereas the temporal analysis target AO-SLO image group changes with time. This is due to the feature that the image quality varies because it is the acquired superimposed image.

As preprocessing here, contrast adjustment by LOG that is often used for photoreceptor cell observation is performed. More specifically, the luminance histogram of each image is obtained, and the maximum and minimum luminance values obtained by cutting the luminance values of 0.5% above and below the histogram are defined as Imax and Imin, respectively. Then, in the contrast adjustment by LOG, the brightness value I of each pixel is changed as in the following expression (2).

Here, Iprocessed shown in Expression (2) is the pixel value after adjustment. Further, Inorm shown in the equation (2) is a constant that normalizes the maximum value of the luminance to the maximum value allowed by the pixel depth (16 bits or 8 bits), and is shown as the equation (3). It is what is done. Further, a shown in the equations (2) and (3) is a parameter used in the contrast adjustment by LOG, and the smaller the value, the larger the adjustment. In the present embodiment, 5000 is selected from the range of 100 to 5000 suitable for the range of a.

After performing such pre-processing, the alignment unit 141 performs alignment in the same manner as in step S431, and the aligned temporal analysis target AO-SLO images are subjected to the temporal analysis target AO-SLO image group. Generate a group. Further, when the detailed alignment is not successful (for example, when the correlation peak of the phase-only correlation is less than or equal to the threshold), the alignment unit 141 acquires at what stage the error has occurred as error flag information. Then, the alignment unit 141 stores, in the storage unit 150 via the control unit 130, the information of the alignment-completed temporal analysis target AO-SLO image group and the alignment result generated in this way.

<Step S490-1>
Subsequently, in step S490-1, the region extraction unit 144 extracts a common region which is a region commonly included in the aligned temporal analysis target AO-SLO image groups obtained as a result of the detailed alignment in step S480. To do.

<Step S490-2>
Subsequently, in step S490-2, the control unit 130 sets the common area extracted in step S490-2 together with the position-adjusted temporal analysis target AO-SLO image group obtained in step S480 and the information of the position alignment result. The control for displaying on the display device 400 is performed.

FIG. 7 shows the first embodiment of the present invention, and is a diagram schematically showing a display example of a registered temporal analysis target AO-SLO image group and the like. In FIG. 7, the common area 720 is displayed while the aligned temporal analysis target AO-SLO image groups 711 to 714 are displayed side by side.

Further, a region 730 for displaying error flag information and the like acquired in the detailed alignment stage is added to each image. Specifically, in FIG. 7, when the image is the reference AO-SLO image (displayed as “reference”), when the detailed alignment has been performed but the movement amount is larger than the reference AO-SLO image ( "Displayed as "large displacement") is described.

The reason for excluding the images having a large movement amount from the analysis target is that the size of the common region 720 becomes extremely small when these images are included. In this embodiment, when the center position of the image is 40% or more away from the center of the reference AO-SLO image, it is excluded from the analysis target. Further, in the error flag information acquired in step S480, the process proceeds up to the parallel movement of the entire image, but in the case of the image in which the distortion correction portion such as the affine transformation after that has an error, the parallel movement is performed. Only the movement is displayed (displayed as “shift”). This is because although this image is roughly aligned, it is not possible to perform alignment at the level of individual photoreceptor cells, and it is possible to compare photoreceptor cell densities within selected areas, but individual It indicates that it is not at the level to judge whether the photoreceptor cells have recovered or disappeared.

The control unit 130 controls the storage unit 150 to store the position-adjusted temporal analysis target AO-SLO image group thus obtained, information on the position alignment result, and the common area 720, and also to output external data through the output unit 160. The control for displaying on the display device 400 is performed. At the same time, the output unit 160 outputs and stores the acquired information and the like in the database 500. Although not shown, information about the date and time of each shooting is also displayed near each displayed AO-SLO image.

When the process of step S490 ends, the process of the flowchart illustrated in FIG. 4 ends.

In the image processing apparatus 100 according to the first embodiment, the AO-SLO images acquired over time are aligned with each other and superimposed on the reference WF-SLO image to generate a superimposed image. ..
According to such a configuration, for example, it is possible to grasp a region in which the AO-SLO image is repeatedly captured over time, and thus it is possible to efficiently perform temporal comparison on the fundus of the eye to be inspected. Furthermore, for example, by selecting an image from the periphery of the area where the AO-SLO image is repeatedly captured and performing detailed alignment, it is possible to perform a temporal comparison on many images.

(Second embodiment)
Next, a second embodiment of the present invention will be described.

In the first embodiment, the AO-SLO images acquired over time are aligned with each other and superimposed on the reference WF-SLO image to generate and display the superimposed image, and the image is captured based on the superimposed image. The method of acquiring a region with a large number of printed images as a temporal evaluation region has been described. Furthermore, in the first embodiment, an AO-SLO image including a temporal evaluation region is selected from a plurality of AO-SLO images, and detailed alignment of these images is performed to extract and display a common region that actually overlaps. I also explained how.

In the second embodiment, the common area of each image is extracted from the temporal analysis target AO-SLO image group for which the detailed alignment of step S480 in the first embodiment is performed, and the periodic structure evaluation and A method for performing analysis such as detection of photoreceptor cells will be described.

The schematic configuration of the ophthalmologic apparatus according to the second embodiment is the same as the schematic configuration of the ophthalmologic apparatus 10 according to the first embodiment shown in FIG. 1, so description thereof will be omitted.

[Processing Procedure of Image Processing Device 100]
FIG. 8 is a flowchart showing an example of a processing procedure of an image processing method by the image processing apparatus 100 according to the second embodiment of the present invention. Here, it is assumed that the processes of steps S410 to S490-1 shown in FIG. 4 have been completed at the “start” stage of the flowchart shown in FIG.

<Step S810>
After performing the processes of steps S410 to S490-1 shown in FIG. 4, in step S810, the analysis unit 145 first sets the aligned temporal analysis target AO-SLO image group obtained in step S480 and step S490-1. The information on the common area extracted in step 3 is acquired from the database 500. Then, the analysis unit 145 generates a spatial frequency conversion image of the region extracted as the common region from each of the aligned temporal analysis target AO-SLO images.

Specifically, first, a square image area having an image size that is an integer that is the Nth power of 2 and is larger than the width and height of the common area is set, and the extraction area is placed at the center thereof. This is called a spatial frequency conversion extracted image, and its size is 512 pixels×512 pixels here. At this time, the average value of all the pixel values in the common area is calculated, and the average value is subtracted from each pixel value in the common area, and the pixel value not corresponding to the common area is set to 0. Furthermore, in order to prevent the influence of discontinuity at the boundary of the common area, processing with a window function is performed. Here, a Hann window having a size of 8 pixels is used. An FFT process (fast Fourier transform) is performed on each of the frequency conversion extracted images thus generated to generate a power spectrum image in which each pixel value is the magnitude of the FFT amplitude. Here, since the power spectrum image is known to have a ring-shaped structure that reflects the cycle of photoreceptor cells, the radius of the ring is obtained from the power spectrum image, and the cycle of photoreceptor cells is roughly estimated from the radius of the ring. There are multiple methods to obtain the radius of the ring, but here, the image center of the power spectrum image is set as the origin, the pixel values of the power spectrum image are integrated in the angular direction, and the integrated value becomes the largest value in the radial direction. Let the location be the radius of the ring.

The value of each pixel value in the power spectrum image shows how strong the structure of the cycle of photoreceptor cells in the corresponding direction is, but here, because the direction of photoreceptor cells is not considered in a particular direction. , The angle components related to the direction are integrated, and attention is paid only to the size of the periodic structure. When the ring radius on the power spectrum image obtained by the above-described method is r, the intercellular distance L at this time can be roughly calculated as shown in the following formula (4).

Here, the size of the power spectrum image (image_size) is 512 pixels×512 pixels as described above, and the size of the pixel (pixel_size) is 0.85 μ/pixel in the case of the AO-SLO image with the S angle of view.

When the common region of all the aligned temporal analysis target AO-SLO images is the same image, the intercellular distance L is the same for all power spectrum images, but in reality The value of the intercellular distance L is different. This is because clinical changes such as changes in photoreceptor density due to disease progression are correctly reflected, when equal areas are not selected due to misalignment, image quality deterioration and focus position shift. Therefore, it is divided into cases where a correct image cannot be obtained. However, even when the photoreceptor cells disappear due to the progress of the disease, the photoreceptor cells are not rearranged so that the distance L between the photoreceptor cells is uniformly increased, but only the affected photoreceptor cells disappear and other photoreceptor cells disappear. It is known that cells often do not show significant changes. Therefore, an image in which the value of the distance L between photoreceptor cells is greatly changed often indicates some problem (misalignment, failure at the time of photographing, etc.). Therefore, by presenting the value of the distance L between photoreceptors or the index calculated using the value of the distance L between photoreceptors to the user, the validity of the analysis result can be shown to the user.

Then, the control unit 130 controls the storage unit 150 to store the information on the inter-visual cell distance L of the common region acquired by the analysis unit 145 in this way, and also displays the information on the external display device 400 via the output unit 160. Control to display. At the same time, the output unit 160 outputs and stores the information on the distance L between photoreceptors to the database 500. Here, there can be a plurality of ways of presenting the distance L between photoreceptors or the index based on the distance L between photoreceptors to the user. For example, presentation to the comment area (730 in FIG. 7) in FIG. 7 is suitable. ..

<Step S820>
Subsequently, in step S820, the analysis unit 145 corresponds to the reference AO-SLO image among the inter photoreceptor distances L corresponding to the common region of each aligned temporal analysis target AO-SLO image acquired in step S810. The photoreceptor cells are detected based on the distance between the photoreceptor cells. Here, a plurality of methods for detecting photoreceptor cells are known, for example, a method for detecting a local maximum value of the brightness value of an image is known. In this method, two points detected at a distance closer than the size corresponding to the distance between photoreceptors are merged by fusion processing. As a parameter at this time, it is possible to use the distance L between photoreceptors. However, since there are fluctuations in the arrangement of photoreceptor cells, most of the detection points will be fused if all two points with a distance L between photoreceptor cells or less are fused, so in the present embodiment, two points of 0.6L or less are fused. Shall be fused. Then, the control unit 130 stores, in the storage unit 150, the coordinate information of the detection points detected in the common region of each aligned temporal analysis target AO-SLO image acquired in this way by the analysis unit 145. Take control.

<Step S830>
Subsequently, in step S830, the analysis unit 145 performs Voronoi analysis or the like based on the coordinate information of the detection points acquired in step S820, and extracts an index regarding the arrangement of photoreceptor cells. Then, the control unit 130 controls the storage unit 150 to store the index information regarding the arrangement of the photoreceptor cells thus obtained by the analysis unit 145.

<Step S840>
Subsequently, in step S840, the control unit 130 controls to display the detection result of the photoreceptor cells acquired in step S820 and the analysis result acquired in step S830 on the external display device 400 through the output unit 160. At this time, the control unit 130 performs control to perform comparative display so as to make it possible to compare the difference in the analysis results over time between the common areas of the aligned temporal analysis target AO-SLO images.

Here, as the comparison display of the analysis result, it is possible to display the comparison result in the entire common area or only in the selected area selected by the user in the common area. Specifically, it is possible to compare and display changes over time in indices such as photoreceptor cell density and Voronoi's hexagonal ratio calculated in the selected region with a list or graph. It is also possible to indicate whether or not the detection points in the selected area match the detection points in the images acquired at different dates and times by changing the color of the detection points to be displayed.

Furthermore, the display can be changed by comparing the indexes acquired as a result of the Voronoi analysis. For example, the color of the detection point or the color of the Voronoi region is changed according to the rate of change of the value of the closest distance (NND) for comparison display, or the difference in the area of the Voronoi region, the difference in the number of vertices, the eccentricity, or the like It is possible to compare and display according to the change in the difference in the index indicating. Further, instead of the contour contour display of the density, it is also possible to display the contour difference of the density or the hexagon ratio of the Voronoi region. In this case, it is possible to intuitively understand in which region the disease progresses by clearly indicating the region where the change is small and the region where the change is large.

When the process of step S840 ends, the process of the flowchart illustrated in FIG. 8 ends.

According to the second embodiment, the following effects are obtained in addition to the effects of the first embodiment described above. In other words, if you want to compare the state of disease progression etc. using images acquired over time, you can obtain and compare analysis indices that reflect the state of photoreceptor cells between the same imaging areas that have been aligned in detail. However, the comparison results can be displayed so that they can be listed.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. It can also be realized by the processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.
The program and a computer-readable storage medium storing the program are included in the present invention.

It should be noted that the above-described embodiments of the present invention are merely examples of specific embodiments for carrying out the present invention, and the technical scope of the present invention should not be limitedly interpreted by these. Is. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

10 ophthalmologic apparatus, 100 image processing apparatus, 110 image acquisition section, 120 information acquisition section, 130 control section, 140 image processing section, 141 alignment section, 142 superposition section, 143 image selection section, 144 area extraction section, 145 analysis section , 150 storage unit, 160 output unit, 200 photographing device, 300 information input device, 400 display device, 500 database

The image processing apparatus of the present invention is a first image obtained by photographing an area of a predetermined size on the fundus of the eye to be inspected, and an area of the eye fundus of the eye to be inspected, the area having the predetermined size. A second image obtained by photographing a narrower area is acquired in association with each set based on the photographing time and stored in the storage means, and the same image storage means stored in the storage means. and position information on the first image of the second image the performing the set with the first images constituting a first alignment is aligned with the second image of the storage wherein the alignment target with respect to the first image as a reference by performing a second, aligning a registration between the first image included in each of the stored plurality of different sets means the based on the alignment means for generating a position fit allowed information of the first image, and the generated positional information by said alignment means and said alignment information, which is the set that was previously acquired first The second image included in the first set and the second image included in the second set, which is the set acquired after the first set, are aligned. having a superimposition means for generating a superimposed image of the second image included in said second image said second sets included in the.
The present invention also includes an image processing method by the above-described image processing apparatus, and a program for causing a computer to execute the image processing method.

Claims (17)

A first image obtained by photographing an area of a predetermined size in the fundus of the eye to be inspected and an area of the fundus of the eye to be inspected which is narrower than the area of the predetermined size. And an image acquisition unit that acquires the second image obtained by
A first alignment, which is alignment of the first image and the second image forming the same set, and an alignment of each of the first images included in a plurality of different sets. Alignment means for performing a certain second alignment to generate position information relating to the first alignment and the second alignment;
On the basis of the position information generated by the position adjusting means, the second image included in the first set, which is the previously acquired set, and the set acquired after the first set, The second image included in a certain second set is superimposed on the first image included in the first set or the first image included in the second set, and a superimposed image is obtained. Superimposing means for generating
An image processing apparatus comprising: The image acquisition means acquires a plurality of the second images for each of the sets,
The superposed image has a pixel value that reflects the number of overlapping images of the second image included in the first set and the second image included in the second set. 1. The image processing device according to 1. The image acquisition means acquires a plurality of the second images for each of the sets,
The superimposed image has a pixel value that reflects the number of overlaps of each of the second image included in the first set and the second image included in the second set. The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. From the superimposed image, a region acquisition unit that acquires a temporal evaluation region that is a region for performing temporal evaluation,
Image selecting means for selecting the second image group for which detailed alignment is to be performed, from the second image group including the temporal evaluation region;
The alignment means further performs a third alignment which is a detailed alignment in the group of the second images selected by the image selection means, and the third alignment is performed. Area extracting means for extracting a common area which is an area commonly included in the second image group;
Control means for controlling the display of the common area and the result of the third alignment together with the group of the second images on which the third alignment has been performed;
The image processing apparatus according to any one of claims 1 to 3, further comprising: The image processing apparatus according to claim 4, wherein the area acquisition unit acquires, in the superimposed image, an area in which the number of overlapping second images is the largest, as the temporal evaluation area. The image processing according to claim 4, wherein the area acquisition unit acquires, as the temporal evaluation area, an area in the superimposed image in which the number of overlapping second images included in different sets is largest. apparatus. The image processing apparatus according to claim 4, wherein the area acquisition unit acquires, in the superimposed image, an area in which the number of overlapping second images in consideration of image quality is the largest, as the temporal evaluation area. The alignment means, when performing the third alignment, selects a second image including the temporal evaluation region in the center from the group of the second images selected by the image selection means. The image processing apparatus according to claim 4, wherein the image processing apparatus is selected as a reference image. From the superimposed image, a region acquisition unit that acquires a temporal evaluation region that is a region for performing temporal evaluation,
Image selecting means for selecting the second image group for which detailed alignment is to be performed, from the second image group including the temporal evaluation region;
The alignment means further performs a third alignment which is a detailed alignment in the group of the second images selected by the image selection means, and the third alignment is performed. Area extracting means for extracting a common area which is an area commonly included in the second image group;
Analyzing means for analyzing photoreceptor cells of the eye to be examined for the common region,
Control means for controlling the display of the result of the analysis by the analysis means,
The image processing apparatus according to any one of claims 1 to 3, further comprising: The analyzing means, when analyzing the photoreceptor cells, obtains an index based on the periodic structure of the photoreceptor cells with respect to the common region, and analyzes the detection of the photoreceptor cells and the arrangement of the photoreceptor cells based on the indicator. The image processing device according to claim 9, wherein The control means compares and displays the analysis result by the analysis means on the display device between the common regions of the respective second images in the group of the second images on which the third alignment has been performed. The image processing apparatus according to claim 9 or 10, wherein the image processing apparatus controls the image processing to be performed. The image processing apparatus according to claim 11, wherein the control unit performs control to display the difference between the detection results of the photoreceptor cells by the analysis unit in the comparative display. The analysis means is for analyzing the arrangement of the photoreceptor cells using Voronoi analysis,
The image processing apparatus according to claim 11, wherein the control unit controls to display the difference between the results of the Voronoi analysis in the comparative display. The difference in the result of the Voronoi analysis is the difference in the closest distance at each detection point by the Voronoi analysis, the difference in the number of vertices of the Voronoi region by the Voronoi analysis, the difference in the area of the Voronoi region, and the shape of the Voronoi region. 14. The image processing apparatus according to claim 13, wherein the image processing apparatus is at least one of the following differences. The image processing apparatus according to claim 11, wherein the control unit controls the comparison display by displaying the difference in the density or the difference in the hexagon ratio by the analyzing unit on the display device by contour lines. A first image obtained by photographing an area of a predetermined size in the fundus of the eye to be inspected and an area of the fundus of the eye to be inspected which is narrower than the area of the predetermined size. An image acquisition step of acquiring the second image obtained by
A first alignment, which is alignment of the first image and the second image forming the same set, and an alignment of each of the first images included in a plurality of different sets. An alignment step of performing a certain second alignment to generate position information relating to the first alignment and the second alignment;
On the basis of the position information generated in the positioning step, the second image included in the first set, which is the previously acquired set, and the set acquired after the first set, The second image included in a certain second set is superimposed on the first image included in the first set or the first image included in the second set, and a superimposed image is obtained. A superposition step for generating
An image processing method comprising: A program for causing a computer to execute each step in the image processing method according to claim 16.