WO2021255779A1 - Information display method, information display device, and program - Google Patents

Abstract

This information display method executed by a processor includes: determining whether an image is a first fundus image of the right eye of a subject or a second fundus image of the left eye of the subject; and generating a display screen that includes a first field and a second field, the first field being for displaying the first fundus image if the first fundus image has already been captured, and for indicating, if the first fundus image has not been captured yet, that the image has not been captured yet, and the second field being for displaying the second fundus image if the second fundus image has already been captured, and for indicating, if the second fundus image has not been captured yet, that the image has not been captured yet.

Description

Information display method, information display device, and program

The technology of this disclosure relates to an information display method, an information display device, and a program.

U.S. Pat. No. 8414122 discloses a fundus camera that captures the fundus. An interface that is easy for users to use is desired.

The information display method according to the first aspect of the technique of the present disclosure is an information display method performed by a processor, which includes a first fundus image of the right eye of the subject and a second fundus image of the left eye of the subject. The step of determining that the first tongue image has been acquired, and the first to indicate that the first tongue image has not been acquired, the first tongue image has been acquired, and the first tongue image has not been acquired. A display screen including one field and a second field for displaying the second fundus image when the second fundus image has been acquired, and for indicating unacquired when the second fundus image has not been acquired. Includes steps to generate.

The information display device of the second aspect of the technique of the present disclosure includes a memory and a processor connected to the memory, wherein the processor is a first fundus image of the right eye of the subject and the left of the subject. It is determined whether the second fundus image of the eye has been acquired, and if the first fundus image has been acquired, the first fundus image is displayed, and if the first fundus image has not been acquired, it has not been acquired. The first field for indicating the above, and the second field for indicating that the second fundus image has not been acquired, the second fundus image is displayed when the second fundus image has been acquired, and the second field for indicating that the second fundus image has not been acquired. Generate a display screen that includes ,.

The program of the third aspect of the technique of the present disclosure determines whether the first fundus image of the right eye of the subject and the second fundus image of the left eye of the subject are acquired by the computer. If the first fundus image has been acquired, the first fundus image is displayed, and if the first fundus image has not been acquired, the first field for indicating unacquired and the second fundus image have been acquired. In the case of, the second fundus image is displayed, and if the second fundus image has not been acquired, a display screen including a second field for indicating unacquired is generated.

It is a block diagram of an ophthalmology system 100. It is a block diagram of an ophthalmic apparatus 110. It is a functional block diagram of the CPU of the control unit 36 of the ophthalmic apparatus 110. It is a flowchart of the information display process executed by the CPU of the control unit 36 of the ophthalmic apparatus 110. It is a flowchart of the alignment process of step 202 of FIG. 5A is a flowchart of the display process of the shooting preparation screen 300 in step 250 of FIG. 5A. It is a figure which shows the shooting preparation screen 300. It is a figure which shows the front eye part photographing section 310 which displayed the object 320 of an eye. FIG. 3 is another diagram showing an anterior ocular segment imaging section 310 displaying an eye object 320. Yet another diagram showing an anterior ocular segment imaging section 310 displaying an eye object 320. FIG. 3 is another diagram showing an anterior ocular segment imaging section 310 displaying an eye object 320. It is still another figure which shows the anterior segment imaging section 310 which displayed the object 320 of the eye. It is a flowchart of the position adjustment process of the ophthalmologic apparatus 110 when the image of the anterior segment of the eye to be inspected 12 is used instead of the object of the eye. It is a figure which shows the (diagonal) image G0 of a pupil based on the image signal from a stereo camera 15B. It is a figure which shows the front image G1 converted from the (oblique) image G0 of a pupil. It is a figure which shows the anterior eye part photographing section 310 which displayed the front image G1. It is a flowchart of the display process of the shooting result confirmation screen 400 of step 208 of FIG. It is a figure which shows the shooting result confirmation screen 400.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

The configuration of the ophthalmic system 100 will be described with reference to FIG. As shown in FIG. 1, the ophthalmology system 100 includes an ophthalmology device 110, an AI (artificial intelligence) analysis server 120, a management server device (hereinafter referred to as “management server”) 140, and an image display device (hereinafter referred to as “management server”). , "Viewer") 150 and. The ophthalmic apparatus 110 acquires a fundus image. The management server 140 stores the fundus image obtained by photographing the fundus of the patient by the ophthalmologic apparatus 110, corresponding to the information of the right eye or the left eye, the patient ID of the patient, and the patient name. The viewer 150 displays medical information such as a fundus image acquired from the management server 140. There may be a plurality of viewers 150.

The fundus image is transmitted from the ophthalmic apparatus 110 or the management server 140 connected to the network 130 to the AI analysis server 120. The AI analysis server 120 uses an AI algorithm trained by machine learning to perform a process of generating information about the symptoms of the eye to be inspected. For example, diabetic retinopathy AI screening is performed by performing image processing on the fundus image of a subject suspected of having diabetic retinopathy or a subject predicted to have diabetic retinopathy by the algorithm. In addition, the AI analysis server 120 determines the degree of progression of diabetic retinopathy with respect to the left and right fundus images of the subject transmitted from the requester, and transmits the determination result to the requester. In order for the AI analysis server 120 to execute a process of generating information on the symptoms of the eye to be inspected such as diabetic retinopathy, there are fundus images of the left and right eyes to be inspected, and the fundus image to be the target of the process is present. The image quality needs to be a certain level or higher for processing using the AI algorithm.

The ophthalmic apparatus 110, the AI analysis server 120, the management server 140, and the viewer 150 are connected to each other via the network 130. The network 130 may be, for example, a local area network (LAN: Local Area Network) in the hospital. The AI analysis server 120 may exist outside the LAN and may be connected to the ophthalmic apparatus 110, the management server 140, and the viewer 150 via the Internet, for example.

Further, the AI analysis server 120, the management server 140, and the viewer 150 include a computer and a display equipped with a CPU, RAM, ROM, a storage device, and the like.

Next, the configuration of the ophthalmic apparatus 110 will be described with reference to FIG.

The ophthalmology device 110 is a device for photographing the fundus. As the ophthalmic apparatus 110, any specific method may be used as long as the fundus can be photographed. For example, a fundus camera or a scanning laser ophthalmoscope may be used. Hereinafter, the case where the ophthalmologic device 110 is a fundus camera will be described.

When the ophthalmic apparatus 110 is installed on a horizontal plane, the horizontal direction is "X direction", the direction perpendicular to the horizontal plane is "Y direction", and the direction perpendicular to the horizontal direction (X direction) and the vertical direction (Y direction) is "Y direction". "Z direction". When the coordinates in each direction of XYZ are taken, the position of (X, Y, Z) = (0,0,0) is the position on the optical axis of the ophthalmologic apparatus 110, and the position where the luminous flux becomes the thinnest. When the center of the pupil of the anterior segment of the eye to be inspected 12 is located at the correct position for imaging of the ophthalmologic apparatus 110, the direction connecting the center of the pupil of the anterior segment of the eye to be inspected 12 and the center of the eyeball is "Z direction". Matches. The direction from the correct position in the Z direction toward the ophthalmologic device 110 side is defined as the negative direction in the Z direction, and the direction away from the correct position is defined as the positive direction in the Z direction.
The position where the luminous flux becomes the thinnest is an example of the "reference position" of the technique of the present disclosure.

The ophthalmic apparatus 110 includes an optical system for photographing the pupil of the eye to be inspected 12. Specifically, the ophthalmic apparatus 110 includes an objective lens 20, a perforated mirror 22, a relay lens 24, a dichroic mirror 26, a photographing light source 28, an observation light source 30, a focus lens 32, and an image pickup element 34.

The ophthalmology device 110 transfers data to the communication unit 38 for communicating with other devices (120, 140, 150), the stereo cameras 15A and 15B for measuring the pupil center of the eye 12 to be inspected, and the ophthalmology device 110. It includes an input / display unit 40 for inputting data and displaying an image or the like of the eye to be inspected 12 obtained by the ophthalmic apparatus 110.

The ophthalmic apparatus 110 includes a position adjusting device 42 for adjusting the three-dimensional position of the ophthalmic apparatus 110.

The ophthalmologic device 110 is connected to a stereo camera 15A, 15B, a photographing light source 28, an observation light source 30, a drive unit (not shown) of a focus lens 32, an image pickup element 34, a communication unit 38, and a position adjustment device 42, and controls these elements. The control unit 36 is provided. The control unit 36 includes a computer having a CPU, a ROM, a RAM, an input / output port, a storage device, and the like (not shown).

The perforated mirror 22 branches the optical path of the eye to be inspected 12 into the optical path of the photographing light source 28 and the observation light source 30 and the optical path of the image pickup element 34. The dichroic mirror 26 branches the optical path of the photographing light source 28 and the observation light source 30 into an optical path of the photographing light source 28 and an optical path of the observation light source 30.

The photographing light source 28 is a light source for photographing the fundus of the eye, and is, for example, a light source of visible light (photographing light) including a halogen lamp or an LED (Light Emitting Diode). The photographing light source 28 irradiates visible light toward the dichroic mirror 26.

The observation light source 30 is a light source for constantly observing the fundus until the fundus is photographed, and is an infrared ray that emits infrared rays (for example, near infrared rays) (observation light) so that the subject does not feel dazzling. It is a light source. The observation light source 30 irradiates infrared rays toward the dichroic mirror 26. For example, a halogen lamp or LED light source and a visible cut filter 14 that cuts visible light from the light from the light source may be provided, and infrared rays may be emitted toward the dichroic mirror 26.

The dichroic mirror 26 reflects infrared rays and transmits visible light. The visible light from the photographing light source 28 passes through the dichroic mirror 26, the infrared rays from the observation light source 30 reflect the dichroic mirror 26, and these lights pass through the relay lens 24, the perforated mirror 22, and the objective lens 20. The light source is irradiated to the fundus of the eye to be inspected 12 through the pupil of the eye to be inspected 12. The light reflected from the fundus of the eye to be inspected 12 is incident on the image sensor 34 via the objective lens 20, the hole of the perforated mirror 22, and the focus lens 32. The focus lens 32 focuses so that the light reflected from the fundus of the eye 12 to be inspected forms an image on the image pickup device 34. Therefore, the fundus of the eye to be inspected 12 is imaged on the image sensor 34.

The image sensor 34 forms an image of the fundus of the eye 12 to be inspected. The image pickup device 34 is an image pickup device having sensitivity to visible light and near infrared light. The image sensor for visible light and the image sensor for near-infrared light may be provided separately. In this case, a dichroic mirror or the like that separates visible light and near-infrared light is arranged in front of the image sensor.

The ophthalmic apparatus 110 guides a fixed-view lamp (visible light source) (not shown) for guiding the line of sight of the eye to be inspected 12 to the optical path of the image pickup element 34, and the fixed-eye light from the fixed-view lamp to the optical path of the eye to be inspected 12. It is equipped with an optical system.

The stereo cameras 15A and 15B will be explained. In order for the ophthalmic apparatus 110 to image the fundus of the eye 12 to be inspected, the photographing light needs to pass through the pupil. Therefore, it is determined whether the pupil is located at the position where the photographing light passes (that is, the optical axis of the photographing light). Need to be detected. Therefore, it is necessary to measure the position of the center of the pupil. Therefore, the ophthalmic apparatus 110 includes stereo cameras 15A and 15B that capture a predetermined region including the pupil of the eye 12 to be inspected from different directions. The stereo cameras 15A and 15B are cameras for visualizing infrared rays emitted from the pupil of the eye to be inspected 12. The stereo cameras 15A and 15B are composed of a light source that irradiates infrared rays and an image sensor that detects infrared rays.

The stereo cameras 15A and 15B are arranged at a predetermined distance in the Y direction (vertical direction), and the anterior eye portion including the pupil of the eye to be inspected 12 is photographed from different directions from the upper side and the lower side. The stereo cameras 15A and 15B may be arranged at a predetermined distance in the X direction (horizontal direction) instead of the vertical direction.

The stereo cameras 15A and 15B output the image signal obtained by photographing the anterior eye portion to the control unit 36. The control unit 36 forms an image of the anterior eye portion including the pupil by image processing the image signals from the stereo cameras 15A and 15B, and measures the three-dimensional (X, Y, Z) position of the center of the pupil. ..

Instead of the stereo cameras 15A and 15B, a laser range finder may be provided, and the control unit 36 may measure the three-dimensional position of the center of the pupil.

The stereo cameras 15A, 15B, and the laser range finder are examples of the pupil position measuring unit of the technique of the present disclosure.

When the control unit 36 determines that the center of the pupil is not located at the position where the photographing light passes, the position adjusting device 42 controls the control unit 36 so that the center of the pupil is located at the position where the photographing light passes. , Adjust the three-dimensional position of the ophthalmic apparatus 110. The position adjusting device 42 is not limited to adjusting the three-dimensional position of the ophthalmic device 110 under the control of the control unit 36, and the position adjusting device 42 may automatically adjust the three-dimensional position of the ophthalmic device 110. .. The three-dimensional position of the ophthalmic apparatus 110 may be manually adjusted.

Next, with reference to FIG. 3, various functions realized by the CPU of the control unit 36 of the ophthalmic apparatus 110 executing the information display program will be described. The information display program has a lighting function, an alignment function, a focus adjustment function, a shooting function, a display function, a judgment function, and a transmission function. When the CPU of the control unit 36 of the ophthalmic apparatus 110 executes an information display program having each of these functions, the CPU has the lighting unit 50, the alignment unit 52, the focus adjustment unit 54, and the photographing unit, as shown in FIG. It functions as a unit 56, a display unit 58, a determination unit 60, and a transmission unit 62.

Next, the information display program by the CPU of the control unit 36 of the ophthalmic apparatus 110 will be described in detail with reference to FIG. When the CPU of the control unit 36 of the ophthalmic apparatus 110 executes the information display program, the information display process shown in the flowchart of FIG. 4 is realized. When photographing the fundus of the eye to be inspected, first, the user (operator) of the ophthalmologic apparatus 110 inputs the patient ID and the patient name via the input / display unit 40. Then, the subject's eye 12 to be inspected is positioned at the imaging position of the ophthalmic apparatus 110. When the user operates a start button (not shown) displayed on the input / display unit 40, the information display program starts.

When the start button (not shown) displayed on the input / display unit 40 is operated, the observation light source 30 is turned on according to the instruction of the control unit 36, and the image sensor 34 continues to photograph the fundus of the eye to be inspected 12. Therefore, the fundus image of the eye 12 to be inspected continues to be output from the image sensor 34 to the control unit 36. Therefore, the control unit 36 constantly monitors the fundus from the start.

In step 200, the lighting unit 50 lights the fixation lamp (not shown above). As a result, the line of sight of the eye 12 to be inspected is guided in the front direction (optical axis direction (Z direction)) of the ophthalmic apparatus 110. This direction is for photographing the central part of the fundus (macula, posterior pole where the optic nerve head is present). When photographing the peripheral portion of the fundus, the lighting portion 50 turns on the lighting position of the fixation lamp at a position different from the front direction.

In step 202, the alignment unit 52 and the display unit 58 execute the alignment process of the ophthalmic apparatus 110 (see also FIG. 5A), and the positional relationship between the eye to be inspected 12 and the ophthalmologic apparatus 110 is appropriately photographed. Perform alignment processing to adjust the relationship. FIG. 5A shows a flowchart of the alignment process in step 202. As shown in FIG. 5A, in step 250, the alignment unit 52 and the display unit 58 execute a display process of the shooting preparation screen (screen) 300 (see also FIG. 6) on the display of the input / display unit 40. FIG. 5B shows a flowchart of the display process of the shooting preparation screen 300 in step 250.

As shown in FIG. 5B, in step 259, the display unit 58 displays the shooting preparation screen 300 (see also FIG. 6) on the display of the input / display unit 40.

As shown in FIG. 6, the following display items are displayed on the shooting preparation screen 300. The display items include a patient ID display field 302, a patient name display field 304, an acquisition status display section 306 that displays the acquisition status of the fundus image of the right eye, and an acquisition status display section that displays the acquisition status of the fundus image of the left eye. There is 308.
The acquisition status display section 306 and the acquisition status display section 308 are examples of the "first section" and the "second section" of the technique of the present disclosure, respectively.

Further, the display items include an anterior ocular segment photographing section 310 for displaying the anterior segment of the eye to be inspected, and a fundus display section 312 for displaying the fundus of the inspected eye. Further, the display items include a deviation amount display section 314 for displaying the deviation amount from the correct position of the position in the Z direction of the ophthalmic apparatus 110, an imaging button 316, and an inspection end button 318 for ending the information display process. In the anterior ocular segment imaging section 310, an eye object 320 generated based on alignment information, which indicates the positional relationship between the eye to be inspected 12 and the ophthalmologic apparatus 110, acquired by the alignment unit 52 is displayed (this). Will be described later). The anterior ocular segment imaging section 310 is an example of a "third section" of the technique of the present disclosure.

Since the fundus image of the eye to be inspected 12 continues to be output from the image sensor 34 to the control unit 36 as described above, the display unit 58 of the control unit 36 displays the monitored fundus image on the fundus display section 312. ..

FIG. 6 shows a case where the eye to be inspected in the fundus image monitored as described above is the right eye.

In step 261 the alignment unit 52 confirms the acquisition status of the fundus image of the eye to be inspected (whether or not the fundus images of the left and right eye to be inspected for AI screening are acquired). The confirmation process in step 261 is a process for confirming the previous acquisition state of the fundus image of the eye to be inspected.
The fundus image of the right eye to be inspected for AI screening is an example of the "first fundus image" of the technique of the present disclosure, and the fundus image of the left eye to be inspected for AI screening is the "second fundus image" of the technique of the present disclosure. This is an example of a fundus image.

Specifically, whether the display unit 58 stores the fundus image and the fundus image corresponding to the information of whether the fundus image of the right eye or the fundus image of the left eye corresponds to the patient ID in the management server 140. Contact us.

If the fundus of each of the right eye and the left eye of the patient is photographed and the fundus images of the right eye and the left eye are acquired, the patient ID, the information of the right eye, and the fundus image of the right eye correspond. Moreover, the patient ID, the information of the left eye, and the fundus image of the left eye are stored in the storage device of the management server 140 correspondingly. The management server 140 corresponds to the patient ID, the information of the right eye, and the fundus image of the right eye, and transmits the patient ID, the information of the left eye, and the fundus image of the left eye to the ophthalmic apparatus 110. .. The ophthalmologic apparatus 110 that has received the patient ID, the information of the right eye, the fundus image of the right eye, the information of the left eye, and the fundus image of the left eye receives the fundus image of the subject eye as the acquisition state of the fundus image of the right eye and the left eye, respectively. It can be confirmed that the fundus image of the eye has been acquired (taken).

If the fundus of the right eye or the left eye of the patient is photographed and the fundus image of the right eye or the left eye is acquired, the information of the right eye and the fundus image of the right eye correspond to each other corresponding to the patient ID. Alternatively, the information of the left eye and the fundus image of the left eye are stored in the storage device of the management server 140 in correspondence with each other. The management server 140 corresponds to the patient ID, the information of the right eye, and the fundus image of the right eye, or corresponds to the patient ID, the information of the left eye, and the fundus image of the left eye, and transmits the patient ID, the information of the left eye, and the fundus image of the left eye to the ophthalmic apparatus 110. .. The ophthalmologic apparatus 110, which has received the patient ID and the information of the right eye and the fundus image of the right eye, or the information of the left eye and the fundus image of the left eye, sets the fundus of the right eye as the acquisition state of the fundus image of the eye to be inspected. The image has been acquired (photographed), but the fundus image has not been acquired for the left eye (unphotographed), or the fundus image for the right eye has not been acquired (unphotographed), but the fundus for the left eye. It can be confirmed that the image has been acquired (taken).

If neither the fundus of the patient's right eye nor the fundus of the left eye has been photographed, the information of the right eye, the image of the fundus of the right eye, the information of the left eye, and the image of the fundus of the left eye correspond to the patient ID. It is not stored in the storage device of 140. The management server 140 does not transmit the information of the right eye, the image of the fundus of the right eye, the information of the left eye, and the image of the fundus of the left eye to the ophthalmic apparatus 110. The ophthalmologic apparatus 110 that does not receive the information of the right eye, the fundus image of the right eye, the information of the left eye, and the fundus image of the left eye receives the fundus images of the right eye and the left eye as the acquisition state of the fundus image of the inspected eye. Can be confirmed that has not been acquired (not photographed).

The technique of the present disclosure is limited to storing the information of the right eye, the fundus image of the right eye, the information of the left eye, and the fundus image of the left eye in the storage device of the management server 140 corresponding to the patient ID. Instead, these information and images may be stored in the storage device of the ophthalmologic device 110. In this case, the alignment unit 52 does not make the above inquiry to the management server 140, but confirms the stored contents of the storage device of the ophthalmic device 110.

In step 263, the display unit 58 determines the display content of the acquisition status display sections 306 and 308 to be the content indicated by the acquisition status confirmed in step 261 and determines the display status of the acquisition status display sections 306 and 308. Make it content. For example, if the right eye is being photographed and neither the right eye nor the left eye is photographed as described above, "not photographed" is displayed in both the acquisition status display section 306 of the right eye and the acquisition status display section 308 of the left eye. do. When the right eye is being photographed and both the right eye and the left eye have been photographed, "photographed" is displayed in both the acquisition status display section 306 of the right eye and the acquisition status display section 308 of the left eye. If the right eye is being photographed and the right eye has not been photographed and the left eye has been photographed, "not photographed" is displayed in the acquisition status display section 306 of the right eye, and the acquisition status display section 308 of the left eye is displayed. , "Photographed" is displayed. FIG. 6 shows a case where the right eye is being photographed, the right eye is photographed, and the left eye is not photographed. Specifically, the acquisition status display section 306 of the right eye is “photographed”. Is displayed, and "not photographed" is displayed in the acquisition status display section 308 of the left eye.

The display process of the shooting preparation screen 300 in step 250 may be performed by the alignment unit 52 and the display unit 58. In this case, the display unit 58 generates the shooting preparation screen 300 based on the alignment information collected by the alignment unit 52, and outputs the image signal of the shooting preparation screen 300 to the display of the input / display unit 40. The display of the input / display unit 40 displays the shooting preparation screen 300 based on the image signal.

When the process of step 263 is completed, the display process (step 250) of the shooting preparation screen 600 of FIG. 5A is completed.

In the next step 252, the alignment unit 52 measures the position of the eye to be inspected, specifically, XYZ at the center of the pupil, by three-dimensional measurement using images from different directions based on the image signals from the stereo cameras 15A and 15B. Measure the position.

In step 254, the alignment unit 52 first determines the display position of the eye object 320 in the anterior segment imaging section 310 from the X, Y coordinate information of the three-dimensional position (X, Y, Z) at the center of the pupil. decide. Specifically, the alignment unit 52 calculates the amount of deviation between the X-direction and Y-direction positions of the center of the pupil and the reference position (X, Y), and the position of the eye to be inspected deviates from the reference position. The display position of the object 320 of the eye for showing the state of being is determined. If (X, Y) is the origin (0,0) indicating the center of the optical axis, the display position of the eye object 320 is the center of the anterior segment imaging section 310. When (X, Y) is different from the origin, the display position in the anterior segment imaging section 310 of the object 320 of the eye to be displayed is determined based on the position information of X and Y.

The eye object 320 is an illustration that imitates the characteristics of the anterior segment of the eye, and is an image object that includes a pupil, an iris, an eyeball conjunctiva (so-called white eye portion), and a circle 328 that highlights the pupil. .. It was

Since the stereo cameras 15A and 15B capture the pupil of the anterior segment of the eye to be inspected 12 from the upper and lower diagonals, when the image of the anterior segment based on the image signals from the stereo cameras 15A and 15B is displayed, they are oblique. Image is displayed. However, by using an eye object, it is possible to display the eye as an illustration image viewed from the front instead of a realistic front eye image that differs depending on the subject, which has the effect of facilitating the alignment work by the user. .. In addition, the realistic front eye part may reflect eyelids and eyelashes in the image, which may make the alignment work difficult. It becomes possible to perform alignment work.

In step 254, the alignment unit 52 secondly displays the eye object 320 displayed in the anterior segment imaging section 310 based on the Z coordinate information of the three-dimensional position (X, Y, Z) of the center of the pupil. Determine the size of. Specifically, the alignment unit 52 calculates the amount of deviation between the position in the Z direction of the center of the pupil and the reference position (Z), and indicates that the position of the eye to be inspected is deviated from the reference position. Determines the size of the object 320 of. If the center of the pupil is at the position where Z = 0 (the position where the luminous flux is the thinnest in the Z direction), the size of the pupil of the eye is the standard size (displayed by default).

However, when the position of the center of the pupil is closer to the position of Z = 0 (closer to the ophthalmic apparatus 110 side) (Z <0), the size of the anterior eye portion photographed by the stereo cameras 15A and 15B is Z = 0. It is taken larger than the one in particular. Therefore, the size of the object 320 of the eye is enlarged according to the value of Z.

When the position of the center of the pupil is farther than the position of Z = 0 (away from the ophthalmic apparatus 110) (Z> 0), the size of the anterior eye portion photographed by the stereo cameras 15A and 15B is Z = 0. It is taken smaller than in particular. Therefore, the size of the eye object 320 is reduced according to the value of Z.

That is, the alignment unit 52 sets the size of the eye object to be displayed according to the coordinate value of Z, which is the position in the optical axis direction of the measured position of the center of the pupil, from the position where the position of the center of the pupil is Z = 0. The size of the eye object is determined so that when it is close, it is larger than the standard size, and when the position of the center of the pupil is farther than the position of Z = 0, it is smaller than the standard size.

In step 256, the display unit 58 displays an eye object of a determined size at a determined position in the anterior ocular segment photographing section 310. Specifically, first, the image data of the eye animation object 320 is stored in advance in the storage device of the control unit 36 of the ophthalmic apparatus 110. The display unit 58 reads out the stored eye object and executes image processing for enlarging or reducing the size of the eye object 320 so as to have a size determined by the alignment unit 52. Then, the resized eye object 320 is displayed at a position in the anterior ocular segment photographing section 310 determined by the alignment portion 52.

Further, the display unit 58 superimposes and displays the cross index (322 and 324) and the circle index 326 at the center of the anterior eye portion photographing section 310. The cross index indicates the position of the optical axis of the ophthalmic apparatus 110. The cross index is composed of a vertical line 324 and a horizontal line 322, and the intersection of the vertical line 324 and the horizontal line 322 is the optical axis position of the ophthalmic apparatus 110. The circle index 326 is a circle indicating the standard pupil size, and the center of the circle index coincides with the intersection of the cross indexes.

In this way, the eye object 320 and the cross index (322 and 324) according to the alignment state between the eye to be inspected 12 and the ophthalmologic apparatus 110 as shown in the anterior ocular segment imaging section 310 of the imaging preparation screen 300 of FIG. And the circle index 326 are displayed.

When the display process of the eye object 320 in step 256 of FIG. 5A is executed as described above, then, in step 258, the alignment portion 52 is positioned in each direction of the measured pupil center in each direction. By controlling the position adjusting device 42 such as the XYZ stage and the chin rest, the position of the ophthalmic device 110 is adjusted so that the center of the pupil is positioned at the correct position (automatic alignment adjustment). Alternatively, while looking at the anterior ocular segment imaging section 310 of the imaging preparation screen 300, the user may operate the position adjusting device 42 to adjust the position of the ophthalmic apparatus 110 so that the center of the pupil is positioned at the correct position. Good (manual alignment adjustment).

In FIG. 7A, the position of the center of the pupil of the eye to be inspected 12 in each direction measured by the alignment unit 52 is deviated from the position of (X, Y, Z) = (0,0,0). It is shown that the eye object 320 is displayed in the anterior ocular segment photographing section 310 with a size corresponding to the case where the position in the Z direction is farther than the correct position (Z = 0).

As shown in FIG. 7A, the center of the anterior ocular segment imaging section 310 is the position of the optical axis of the ophthalmologic apparatus 110, and in order to make the position easier to understand, the display unit 58 is attached to the anterior ocular segment imaging section 310. A cross index determined by the horizontal line 322 and the vertical line 324 is displayed, and the intersection of the horizontal line 322 and the vertical line 324 is positioned at the center of the anterior ocular segment imaging section 310. A circular index 326 of the standard size of the pupil located at the correct position in the Z direction about the center of the anterior segment imaging section 310 is displayed in the anterior segment imaging section 310.

In the example shown in FIG. 7A, as described above, the measured position of the pupil center of the eye 12 in the Z direction is farther than the correct position (Z = 0), so that the size of the circle 328 is smaller than the circle index 326. It is displayed (the object 320 of the eye is enlarged from the default). The first state (the amount of deviation of the pupil of the eye to be inspected 12 from the optical axis in the XY plane) is expressed by the amount of deviation of the center of the circle 328 from the center of the anterior segment imaging section 310, and the size of the circle 328. The second state (whether or not the fundus of the eye to be inspected 12 is in focus or not, the amount of deviation in the Z-axis (optical axis) direction) is expressed by the above.

In FIG. 7B, the measured position of the center of the pupil in each direction of XY is deviated from the position of (X, Y, Z) = (0,0,0), but the position in the direction of Z is correct. It is shown that the object of the eye, when located in position, is displayed in the anterior segment imaging section 310. The size of the circle 328 is the same as the circle index 326 of the standard size of the pupil (the object 320 of the eye is displayed by default) because the position in the Z direction is in the correct position.

In FIG. 7C, the measured position of the center of the pupil in each direction of XY is located at the correct position, but the position in the direction of Z is deviated from the correct position, especially when the position is far away. It is shown that the object is displayed in the anterior segment imaging section 310. The center of the circle 328 coincides with the center of the circle index 326 of the standard size of the pupil, since the measured position of the center of the pupil in each direction of XY is located at the correct position. However, since the measured Z-direction position of the pupil center of the eye 12 is farther than the correct position (Z = 0), the size of the circle 328 is smaller than the circle index 326 of the standard size of the pupil (of the eye). Object 320 is displayed smaller than the default).

In FIG. 7D, the measured positions of the center of the pupil in each direction of XY are located at the correct positions, but the positions in the direction of Z are deviated from the correct positions, especially when they are located close to each other. It is shown that the object of the eye is displayed in the anterior segment imaging section 310. The center of the circle 328 coincides with the center of the circle index 326 of the standard size of the pupil, but the size of the circle 328 is larger than the circle index 326 of the standard size of the pupil.

FIG. 7E shows how the animation of the eye is displayed in the anterior segment imaging section 310 when the measured position of the center of the pupil in each direction of XYZ is located at the correct position. The center of the circle 328 coincides with the center of the circle index 326 of the standard size of the pupil, and the size of the circle 328 coincides with the circle index 326 of the standard size of the pupil.

As shown in FIGS. 7A to 7E, the intersection of the horizontal line 322 and the vertical line 324 in the cross index is located at the center of the anterior ocular segment imaging section 310. In this state, the eye object 320 is displayed in the anterior segment imaging section 310 corresponding to the position measured by the stereo cameras 15A and 15B, and in a size corresponding to the position of the center of the pupil.

Therefore, depending on the position and size of the object of the eye, the user intuitively knows whether or not the position of the eye to be inspected 12 is in the correct position, and if not, in which direction and how much. Can be understood.

The user operates the XYZ stage and the chin rest of the ophthalmologic apparatus 110 while looking at the eye object displayed in the anterior ocular segment imaging section 310 to obtain the positional relationship between the eye to be inspected 12 and the ophthalmologic apparatus 110. It can be adjusted to the optimum position for shooting. In addition, the display position and size of the eye object are displayed as a moving image such as an animation according to the operation of the XYZ stage and the chin rest. Therefore, the user can perform the alignment work while looking at the display of the ophthalmic apparatus 110 without looking at the XYZ stage, the operation buttons of the jaw rest, or the joystick.

The display unit 58 is in the case where the Z position information (coordinate value) measured by the alignment unit 52 is closer to the Z value of the predetermined safety reference value (closer to the ophthalmic apparatus 110 side). In addition, a warning display may be displayed on the imaging preparation screen 300 to inform that the device is in danger of colliding with the patient.

By observing the cross index 322, 324 and the circle index 326 at the center of the anterior optometry section 310 and the display position and size of the object of the eye, the user can see the amount of deviation in the X direction and the Y direction and the reference position. It is possible to intuitively grasp the amount of deviation of the eye to be inspected in the Z direction. Therefore, the control of the three-dimensional drive stage for moving the vertical, horizontal, and front-back positions of the ophthalmic apparatus 110 and the control of the position of the jaw stand are operated while looking at the eye object displayed in the anterior ocular segment imaging section 310. You can easily adjust the optimum position for shooting.

In the present embodiment, as described above, the size of the pupil in the object 320 of the eye is displayed in a size corresponding to the position of the center of the pupil in the Z direction. This allows the user to understand whether or not the position of the eye 12 to be inspected 12 in the Z direction is located at the correct position, and if not, how much it is.

Further, in the present embodiment, the color of the display unit 58 changes according to the amount of deviation of the Z direction position of the ophthalmic apparatus 110 from the correct position based on the Z position information measured by the alignment unit 52. The mark to be displayed is displayed in the deviation amount display section 314. For example, when the deviation amount is 0, the display unit 58 changes the color of the deviation amount display section 314 to blue. As the amount of deviation from the correct position in the Z direction toward the ophthalmologic device 110 increases, the display unit 58 increases the color of the deviation amount display section 314 from blue to red. As the amount of deviation from the correct position in the Z direction toward the ophthalmologic device 110 increases, the display unit 58 increases the color of the deviation amount display section 314 from blue to green.

In this way, the color of the deviation amount display section 314 is changed according to the deviation amount and the deviation direction in the Z direction from the correct position of the pupil center of the eye to be inspected 12. Therefore, the user can understand whether or not the position of the eye 12 to be inspected 12 in the Z direction is located at the correct position, and if not, how much it is. When there is a deviation amount display section 314 in the Z-axis direction, it is not necessary to change the display size of the object 320 of the eye.

In the example shown in FIGS. 7A to 7E, the alignment unit 52 displays the animation 320 of the eye, but the technique of the present disclosure is not limited to this, and computer graphics (CG) are used instead of the object. (Computer graphics)) images may be displayed with their positions and sizes adjusted in the same manner as in the examples shown in FIGS. 7A to 7E. Further, it may be a three-dimensional three-dimensional object image instead of a two-dimensional object image.
The images of the object 320 and CG are examples of simulated images imitating the eye to be inspected.

In the position adjustment process of the ophthalmic apparatus 110 shown in FIG. 5A described above, the realistic anterior ocular segment image of the eye 12 to be inspected obtained by the stereo cameras 15A and 15B is not used. Therefore, when the position adjustment process of the ophthalmic apparatus 110 shown in FIG. 5A is executed, the control unit 36 does not need to form the anterior eye portion image including the pupil. Therefore, a laser rangefinder can be used instead of the stereo cameras 15A and 15B.

However, the technique of the present disclosure is not limited to this, and the anterior eye portion image of the eye to be inspected 12 obtained by one of the stereo cameras 15A and 15B may be used instead of the object of the eye.

FIG. 8 shows a flowchart of the position adjustment process of the ophthalmologic apparatus 110 when the anterior eye portion image of the eye to be inspected 12 obtained by one of the stereo cameras 15A and 15B is used instead of the eye object. .. In the adjustment process shown in FIG. 8, between step 250 and step 258 of FIG. 5A, steps 251 to 258 are executed instead of steps 252 to 256.

In step 251 the alignment unit 52 photographs the pupil of the anterior eye portion of the eye to be inspected 12 (from an angle) with the stereo cameras 15A and 15B.

In step 253, the alignment unit 52 measures the position of the pupil in each direction of XYZ by three-dimensional measurement using images from different directions based on the image signals from the stereo cameras 15A and 15B.

In step 255, the alignment unit 52 uses the position information of the pupil in each direction of XYZ and one of the stereo cameras 15A and 15B held in advance, for example, the ophthalmology of the stereo camera 15B for photographing the anterior segment of the eye from below. Using the tilt information with respect to the optical axis of the device 110, the (oblique) image G0 (see FIG. 9A) of the pupil based on the image signal from the stereo camera 15B is converted into the front image G1 as shown in FIG. 9B.

In step 257, first, the alignment unit 52 displays the position of the front image G1 and the position information of Z based on the position information in the directions of X and Y, as in the process of step 254 of FIG. 5A. Based on, the size of the front image G1 to be displayed is determined.

In step 257, the alignment unit 52 secondly displays the front image G1 of the determined size and the cross index at the determined position in the anterior ocular segment photographing section 310.

In the example shown in FIG. 8, the front image G1 of the pupil of the eye to be inspected 12 is displayed, but since the size of the pupil is physiologically changed by the light from the environment, the center of the pupil of the eye to be inspected 12 is displayed. Even if is positioned at the correct position, the size of the pupil in the front image G1 may not match the above standard size. Therefore, in order to measure the position of the pupil in each direction of XYZ, the position of the pupil in each direction of XYZ may be measured by a laser range finder without using the image signals from the stereo cameras 15A and 15B.

When the adjustment of the position of the ophthalmic apparatus 110 is completed as described above, the process of step 202 in FIG. 4 is completed.

In step 204, the focus adjustment unit 54 adjusts the position of the focus lens 32 to adjust the focus so that the fundus of the eye to be inspected 12 forms an image on the image sensor 34.

In step 206, the photographing unit 56 waits for the photographing button 316 to be turned on, and when the photographing button 316 is turned on, the photographing light source 28 is turned on and the fundus of the eye to be inspected 12 is photographed.

Note that in step 206, by highlighting the imaging button 316, for example, changing the color or blinking the image, the alignment and focus adjustment of the ophthalmologic apparatus 110 are completed, and the operator is in a state where imaging is possible. It may be aroused. A message indicating that shooting may be possible may be displayed.

The photographing button 316 may be omitted, and when the process of step 204 is completed, the photographing light source 28 may be automatically turned on to photograph the fundus of the eye to be inspected 12.

Further, a mode selection button for setting either an automatic shooting mode or a manual shooting mode may be provided. The auto photographing mode is a mode in which the fundus of the eye to be inspected 12 is automatically photographed when the process of step 204 is completed. The manual shooting mode is a mode for shooting the fundus of the eye to be inspected 12 when the shooting button 316 is turned on. The photographing unit 56 photographs the fundus of the eye to be inspected 12 according to the setting by the mode selection button.

In step 208, the display unit 58 executes a display process of the shooting result confirmation screen (screen (graphic user interface)) 400 shown in FIG. 11 instead of the shooting preparation screen 300 on the display of the input / display unit 40.

FIG. 10 shows a flowchart of the display process of the shooting result confirmation screen 400. As shown in FIG. 10, in step 350, the display unit 58 displays the shooting result confirmation screen 400.

As shown in FIG. 11, the following display items are displayed on the shooting result confirmation screen 400. The display items include a patient ID display field 402, a patient name display field 404, a right eye fundus image display field 410 for displaying the right eye fundus image, and a left eye fundus image display field 412 for displaying the left eye fundus image. be.

The right eye fundus image display field 410 and the left eye fundus image display field 412 displaying the imaging result of the left eye are examples of the "first field" and the "second field" of the technique of the present disclosure, respectively.

On the left and right sides of the right eye fundus image display field 410, when the right eye is photographed multiple times and multiple fundus images of the right eye are stored, the plurality of fundus images of the right eye are sequentially displayed in the right eye fundus image display field. The right eye fundus image switching buttons 410A and 410B for instructing the display on the 410 are provided. When the right eye fundus image switching button 410A is turned on, it is instructed to switch to the fundus image obtained by taking a picture in the past from the time when the fundus image currently displayed in the right eye fundus image display field 410 is taken. When the right eye fundus image switching button 410B is turned on, it is instructed to switch to the fundus image obtained by being photographed in the future from the time when the fundus image currently displayed in the right fundus image display field 410 is taken. Left eye fundus image switching buttons 412A and 412B are also provided on the left and right sides of the left eye fundus image display field 412. Below the right eye fundus image display field 410, a right eye image quality determination result display field 420 for displaying the image quality determination result of the fundus image displayed in the right eye fundus image display field 410 is provided. A left eye image quality determination result display field 422 is also provided below the left eye fundus image display field 412.

The right eye image quality determination result display field 420 and the left eye image quality determination result display field 422 are examples of the "third field" and the "fourth field" of the technique of the present disclosure, respectively.

Further, the display items include a shooting continuation button 432, an AI analysis execution button 434, and an inspection end button 436 for ending the information display process.
The AI analysis execution button 434 is an example of the "button" of the technique of the present disclosure, and the portion where the AI analysis execution button 434 is displayed is an example of the "fifth field" of the technique of the present disclosure.

In step 352, the display unit 58 discriminates between the left and right of the eye to be inspected in the fundus image obtained by the imaging in step 206. It should be noted that the left and right discrimination of the eye to be inspected in the fundus image can be performed as follows. For example, at the time of shooting in step 206, the operator presses a right eye button or a left eye button (not shown) according to the eye to be imaged, and the control unit 36 stores the information in the storage device. .. In step 352, the display unit 58 determines the left and right of the eye to be inspected of the fundus image based on the information stored in the storage device of the control unit 36. Further, the display unit 58 may perform the above-mentioned left / right discrimination by image processing. More specifically, the display unit 58 performs image processing for extracting the optic disc and the blood vessel to the optic disc on the fundus image. Then, the display unit 58 compares the direction of the blood vessel to the optic disc of the fundus image with the direction of the blood vessel to the optic disc of the left and right fundus memorized in advance, so that the fundus image can be displayed on either the left or right eye. Judge whether it is a fundus image of. The display unit 58 may verify the left / right discrimination result of the eye to be inspected of the fundus image determined based on the information stored in the storage device of the control unit 36 by the above image processing.

In step 354, the display unit 58 displays the fundus image corresponding to the discrimination result of step 352 in the right fundus image display field 410 or the left fundus image display field 412.

In the example shown in FIG. 11, the fundus of the right eye is photographed, but the fundus of the left eye is not photographed. As described above, for example, when the fundus of the right eye is photographed, the obtained fundus image is displayed in the right eye fundus image display field 410. When the fundus of the left eye is not photographed, nothing is displayed in the fundus image display field 412 of the left eye, indicating that the fundus of the left eye is not photographed. In this way, from the display state of the right eye fundus image display field 410 and the left eye fundus image display field 412, the operator can determine whether or not the fundus of each eye is photographed.

If the fundus of the right eye has been photographed multiple times, the right eye fundus image switching buttons 410A and 410B are displayed. This allows the operator to know that the fundus of the right eye has been photographed multiple times. If the fundus of the left eye has not been photographed even once, the left eye fundus image switching buttons 412A and 412B are not displayed. This allows the operator to know that the fundus of the left eye has never been photographed.

In step 356, the display unit 58 determines whether or not the image quality of the fundus image obtained by the photographing in step 206 is acceptable. The position of the ophthalmic apparatus 110 is adjusted in step 202, and the focus is adjusted in step 204. There is.

The permissible image quality is an image quality of a certain level or higher that can execute a process of generating information regarding the symptom of the eye to be inspected using AI. Specifically, the fundus image having a certain image quality or higher satisfies at least one condition (both in the present embodiment) that the fundus image is not blurred and that the position of the optic nerve head is within a predetermined position. It is an image.

Specifically, the display unit 58 extracts blood vessels from the fundus image. The display unit 58 determines whether or not the image is out of focus by determining whether or not the contrast between the extracted blood vessel portion and the portion other than the blood vessel portion of the fundus image is equal to or greater than a predetermined value. The display unit 58 determines the position of the optic nerve head from the fundus image from the blood vessel pattern obtained in advance, and determines whether or not the determined position is within the range of the predetermined position of the fundus image.

Further, the image quality is determined by comparing the reference image and the photographed fundus image using algorithms such as PSNR (Peak signal-to-noise ratio) and SSIM (Structural Similarity), thereby taking a picture in step 206. It may be determined whether or not the image quality of the fundus image obtained by the above is acceptable.

In step 358, the display unit 58 displays the image quality determination result in the evaluation result display field 420/422 corresponding to the determination result of the eye to be inspected. In the example shown in FIG. 11, when it is determined that the image quality of the fundus image of the right eye is an acceptable image quality, a display indicating that the image quality is an acceptable image quality is displayed in the right eye image quality determination result display field 420. For example, "OK" is displayed. Since the fundus image of the left eye has not been obtained as described above, the image quality cannot be determined. Therefore, nothing is displayed in the left eye image quality determination result display field 422.

When the display process of the determination result in step 358 is completed, the display process of the shooting result confirmation screen 400 in step 208 of FIG. 4 is completed.

In step 210, the determination unit 60 determines whether or not the image quality of the fundus image obtained by the photographing in step 206 is acceptable. This judgment is made based on the judgment result of step 356. If it is determined that the image quality is not acceptable, that is, if the determination result is a negative acceptance result, the information processing returns to step 200 and executes steps 200 to 210. If it is determined that the image quality is acceptable, that is, if the determination result is an acceptable result, the information processing proceeds to step 212.

In step 212, the determination unit 60 determines whether or not steps 200 to 210 have been completed for both eyes. When it is not determined that steps 200 to 210 are completed for both eyes, for example, when it is determined that the fundus of the right eye is photographed but the fundus of the left eye is not photographed as described above. Information processing proceeds to step 214.

In step 214, the determination unit 60 determines whether or not the imaging continuation button 432 is turned on, so that the operator has instructed whether or not the fundus of the eye is photographed differently from the eye in which the fundus has already been imaged. to decide. In step 214, by highlighting the shooting continuation button 432, for example, changing the color or blinking the color, the image quality of the fundus image of the eye whose fundus has already been photographed is allowed, and the image quality of the fundus image of another eye is allowed. The operator may be alerted that shooting is necessary. It should be noted that a message may be displayed to the effect that the image quality of the fundus image of the eye whose fundus has already been photographed is acceptable and that the fundus of another eye needs to be imaged. When the shooting continuation button 432 is turned on, the information processing returns to step 200 and executes steps 200 to 212.

If it is determined in step 212 that step 200 to step 210 have been completed for both eyes, in step 216, the display unit 58 activates the AI analysis execution button 434, that is, the user can operate it. Change to state. In step 218, the determination unit 60 determines whether or not the AI analysis execution button 434 is turned on.

If it is determined in step 212 that steps 200 to 210 have been completed for both eyes, the fundus image of the right eye is displayed in the right eye fundus image display field 410, and the left eye is displayed in the left eye fundus image display field 412. The fundus image is displayed, and "OK" is displayed in the right eye image quality determination result display field 420 and the left eye image quality determination result display field 422. Therefore, it can be understood that the operator can understand that the permissible fundus image was obtained for the left and right eye to be inspected, and that it is possible to execute the process of generating the information regarding the symptom of the inspected eye using AI. Therefore, the operator turns on the AI analysis execution button 434.

In step 216, the display unit 58 may highlight the AI analysis execution button 434, for example, blinking display or changing the color. Further, the shooting result confirmation screen 400 is provided with an AI analysis execution button display field for displaying the AI analysis execution button 434. If it is determined in step 212 that step 210 has been completed for both eyes, the display unit 58 may display the AI analysis execution button 434 in the AI analysis execution button display field.
The AI analysis execution button display field is an example of the "fifth field" of the technique of the present disclosure.

When it is determined in step 218 that the AI analysis execution button 434 is turned on, in step 220, the transmission unit 62 controls the communication unit 38 to transmit the fundus image of both eyes to the AI analysis server 120. The fundus image to be transmitted is only a fundus image having an acceptable image quality. Further, instead of the information that can identify an individual such as the patient ID and the patient name, the image management ID uniquely assigned to the fundus image is transmitted as additional information together with the fundus image data.

The AI analysis server 120 executes a process of generating information on the symptoms of the eye to be inspected, such as the presence or absence of diabetic retinopathy or the estimation of the degree of progression (grade) of the transmitted fundus images of both eyes. The generated information is transmitted to the management server 140 together with the image management ID. The management server 140 identifies the patient ID and the patient name from the image management ID, and stores the generated information. When the patient ID and the patient name are specified from the viewer 150 and the transmission instruction of the generated information such as diabetic retinopathy is transmitted to the management server 140, the management server 140 transmits the generated information to the viewer 150. do. The viewer 150 displays the generated information on the display.

As described above, in order for the AI analysis server 120 to process the fundus image to generate information on the symptoms of the eye to be inspected such as the presence or absence of diabetic retinopathy and the degree of progression, the image quality of the fundus image to be processed is processed. However, it is necessary that the image quality is above a certain level so that the above processing can be executed using AI. Therefore, conventionally, it has been necessary to determine in advance a fundus image in which the fundus image obtained by photographing the eye to be inspected has the above-mentioned image quality of a certain level or higher. It is difficult to judge whether or not the image quality of the fundus image is above a certain level unless the person is familiar with the AI diagnosis of ophthalmology.

However, in the present embodiment, the ophthalmologic apparatus 110 photographs the fundus of each of the left and right eyes to be inspected until a fundus image having a certain image quality or higher is obtained.

Therefore, in the present embodiment, even a person who is not familiar with the AI diagnosis of the ophthalmology can execute a process of generating information on the symptom of the eye to be inspected on the ophthalmology apparatus 110 with an image quality of a certain level or higher. The fundus image can be output to the AI analysis server 120. Further, in the present embodiment, since a graphic user interface that can easily distinguish whether or not a fundus image having a certain image quality or higher on the left and right has been taken is used, the fundus images of the left and right eyes having the above image quality are analyzed by AI. It can be output to the server 120.

The ophthalmology system 100 of the present embodiment includes an ophthalmology apparatus 110, an AI analysis server 120, a management server 140, and a viewer 150. However, the techniques of the present disclosure are not limited to this. For example, the AI analysis server 120 may be omitted, and at least one of the ophthalmology device 110, the management server 140, and the viewer 150 may have the AI function (AI algorithm) of the AI analysis server 120.

The present embodiment includes an AI analysis server 120 that executes a process of generating information regarding the symptom of the eye to be inspected by using an AI algorithm trained by machine learning. However, the techniques of the present disclosure are not limited to this. For example, images of various diseases of the eye are stored in advance, and the fundus image obtained by photographing the fundus is matched with the images of various diseases of the eye stored in advance, and the eye to be inspected is examined. It may be provided with a device that performs processing for generating information on the symptom of. In addition, a process of extracting information specific to a disease from images of various diseases of the eye and comparing it with the transmitted characteristics of the fundus image to be analyzed to generate information on the symptoms of the eye to be inspected. It may be provided with an analyzer that executes a program that performs the above.

In the present disclosure, each component (device, etc.) may exist only once or two or more as long as there is no contradiction.

In the example described above, the case where the image processing is realized by the software configuration using the computer is illustrated, but the technique of the present disclosure is not limited to this. For example, instead of the software configuration using a computer, the image processing may be executed only by the hardware configuration such as FPGA (Field-Programmable Gate Array) or ASIC (Application Specific Integrated Circuit). Some of the image processing may be performed by the software configuration and the rest may be performed by the hardware configuration.

As described above, the technology of the present disclosure includes the following technology because it includes the case where the image processing is realized and the case where the image processing is not realized by the software configuration using the computer.

(First technology)
A judgment unit for determining that the first fundus image of the subject's right eye and the second fundus image of the subject's left eye have been acquired.
If the first fundus image has been acquired, the first fundus image is displayed, and if the first fundus image has not been acquired, the first field for indicating unacquired and the second fundus image have been acquired. In the case of, the second fundus image is displayed, and if the second fundus image has not been acquired, a second field for indicating that the second fundus image has not been acquired, and a generation unit for generating a display screen including.
Information display device.

(Second technology)
A step of determining that the first fundus image of the subject's right eye and the second fundus image of the subject's left eye have been acquired by the determination unit.
When the generation unit has acquired the first fundus image, the first fundus image is displayed, and when the first fundus image has not been acquired, the first field for indicating that the first fundus image has not been acquired, and the second fundus. If the image has been acquired, the second fundus image is displayed, and if the second fundus image has not been acquired, a second field for indicating that the second fundus image has not been acquired, and a step of generating a display screen including the step of generating the display screen.
Information display method including.

(Third technology)
A judgment unit for determining whether the first fundus image of the right eye of the subject and the second fundus image of the left eye of the subject have been acquired.
A detection unit that detects the position of the eye to be inspected,
A calculation unit that calculates the amount of deviation between the detected position and the reference position where the eye to be inspected should be located in order to photograph the fundus of the eye to be inspected.
The detection is based on the first section for indicating that the first fundus image has been acquired or not acquired, the second section for indicating that the second fundus image has been acquired or not acquired, and the amount of deviation. A generator that generates a display screen including a third section for displaying an image showing that the position of the eye to be inspected deviates from the reference position.
Information display device.

(Fourth technology)
The step of determining whether the first fundus image of the right eye of the subject and the second fundus image of the left eye of the subject have been acquired by the determination unit.
The step in which the detection unit detects the position of the eye to be inspected,
A step in which the calculation unit calculates the amount of deviation between the detected position and the reference position where the eye to be inspected should be located in order to photograph the fundus of the eye to be inspected.
The generation unit is based on the first section for indicating that the first fundus image has been acquired or not acquired, the second section for indicating that the second fundus image has been acquired or not acquired, and the amount of deviation. A step of generating a display screen including a third section for displaying an image showing how the detected position of the eye to be inspected deviates from the reference position.
Information display method including.

The alignment unit 52 is an example of the "judgment unit", "detection unit", and "calculation unit" of the technique of the present disclosure. The alignment unit 52 and the display unit 58 are examples of the “generation unit” of the technique of the present disclosure.

The following technologies are proposed based on the above disclosure contents.

(Fifth technology)
A computer program product for displaying information
The computer program product comprises a computer-readable storage medium that is not itself a temporary signal.
A program is stored in the computer-readable storage medium.
The program
On the computer
It is determined whether the first fundus image of the subject's right eye and the second fundus image of the subject's left eye have been acquired.
If the first fundus image has been acquired, the first fundus image is displayed, and if the first fundus image has not been acquired, the first field for indicating that the first fundus image has not been acquired and the second fundus image have been acquired. In the case of, the second fundus image is displayed, and if the second fundus image has not been acquired, a second field for indicating that the second fundus image has not been acquired is displayed, and a computer program product for executing the generation of a display screen including.

(Sixth technology)
A computer program product for displaying information
The computer program product comprises a computer-readable storage medium that is not itself a temporary signal.
A program is stored in the computer-readable storage medium.
The program
On the computer
It is determined whether the first fundus image of the right eye of the subject is acquired and the second fundus image of the left eye of the subject is acquired.
The position of the eye to be inspected is detected,
The amount of deviation between the detected position and the reference position where the eye to be inspected should be located in order to photograph the fundus of the eye to be inspected is calculated.
The detection is based on the first section for indicating that the first fundus image has been acquired or not acquired, the second section for indicating that the second fundus image has been acquired or not acquired, and the amount of deviation. Generates a display screen that includes a third section for displaying an image showing how the position of the eye to be inspected deviates from the reference position.
A computer program product that lets you do that.

The control unit 36 is an example of the "computer program product" of the technology of the present disclosure.

The information display process described above is just an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be changed within a range that does not deviate from the purpose.

All documents, patent applications, and technical standards described herein are as if the individual documents, patent applications, and technical standards were specifically and individually stated to be incorporated by reference. Incorporated by reference herein.

Claims (12)

1. It is an information display method performed by the processor.
   A step of determining that the first fundus image of the subject's right eye and the second fundus image of the subject's left eye have been acquired, and
   If the first fundus image has been acquired, the first fundus image is displayed, and if the first fundus image has not been acquired, the first field for indicating that the first fundus image has not been acquired and the second fundus image have been acquired. In the case of, the second fundus image is displayed, and if the second fundus image has not been acquired, a second field for indicating that the second fundus image has not been acquired, and a step of generating a display screen including the step of generating the display screen.
   Information display method including.
2. On the display screen,
   A third field for displaying the first evaluation result of the image quality of the first fundus image, and
   A fourth field for displaying the second evaluation result of the image quality of the second fundus image, and
   The information display method according to claim 1, further comprising.
3. The display screen includes a button for executing a process for generating information regarding the symptom of the eye to be inspected.
   If at least one of the first evaluation result and the second evaluation result is an acceptable result, the step of activating the button is further included.
   The information display method according to claim 2.
4. The display screen includes a fifth field displaying a button for executing a process for generating information regarding the symptom of the eye to be inspected.
   If at least one of the first evaluation result and the second evaluation result is an acceptable result, the step of displaying the button in the fifth field is further included.
   The information display method according to claim 2.
5. When a plurality of fundus images of the right eye are acquired, a step of displaying the plurality of acquired fundus images in the first field so as to be switchable.
   When a plurality of the fundus images of the left eye are acquired, the step of displaying the plurality of acquired fundus images in the second field so as to be switchable.
   The information display method according to any one of claims 1 to 4, further comprising.
6. A memory and a processor connected to the memory are provided.
   The processor
   It is determined whether the first fundus image of the subject's right eye and the second fundus image of the subject's left eye have been acquired.
   If the first fundus image has been acquired, the first fundus image is displayed, and if the first fundus image has not been acquired, the first field for indicating that the first fundus image has not been acquired and the second fundus image have been acquired. In the case of, the second fundus image is displayed, and if the second fundus image has not been acquired, a display screen including a second field for indicating that the second fundus image has not been acquired is generated.
   Information display device.
7. On the computer
   It is determined whether the first fundus image of the subject's right eye and the second fundus image of the subject's left eye have been acquired.
   If the first fundus image has been acquired, the first fundus image is displayed, and if the first fundus image has not been acquired, the first field for indicating that the first fundus image has not been acquired and the second fundus image have been acquired. In the case of, the second fundus image is displayed, and if the second fundus image has not been acquired, a display screen including a second field for indicating that the second fundus image has not been acquired is generated.
   A program that lets you do things.
8. It is an information display method performed by the processor.
   A step of determining whether the first fundus image of the right eye of the subject and the second fundus image of the left eye of the subject have been acquired, and
   The step of detecting the position of the eye to be inspected and
   A step of calculating the amount of deviation between the detected position and the reference position where the eye to be inspected should be located in order to photograph the fundus of the eye to be inspected.
   The detection is based on the first section for indicating that the first fundus image has been acquired or not acquired, the second section for indicating that the second fundus image has been acquired or not acquired, and the amount of deviation. A step of generating a display screen including a third section for displaying an image showing how the position of the eye to be inspected deviates from the reference position.
   Information display method including.
9. Assuming that the left-right direction is the X direction and the up-down direction is the Y direction with respect to the reference position,
   In the third section, the first state in which the position of the detected eye to be inspected is deviated from the reference position in the X direction and the Y direction is displayed.
   The information display method according to claim 8.
10. Assuming that the left-right direction is the X direction, the up-down direction is the Y direction, and the X-direction and the direction perpendicular to the Y-direction are the Z-directions with respect to the reference position.
    In the third section, the first state in which the detected position of the eye to be inspected is deviated from the reference position in the X and Y directions and the second state in which the position is deviated in the Z direction are displayed. ,
    The information display method according to claim 8.
11. A memory and a processor connected to the memory are provided.
    The processor
    It is determined whether the first fundus image of the right eye of the subject is acquired and the second fundus image of the left eye of the subject is acquired.
    The position of the eye to be inspected is detected,
    The amount of deviation between the detected position and the reference position where the eye to be inspected should be located in order to photograph the fundus of the eye to be inspected is calculated.
    The detection is based on the first section for indicating that the first fundus image has been acquired or not acquired, the second section for indicating that the second fundus image has been acquired or not acquired, and the amount of deviation. Generates a display screen that includes a third section for displaying an image showing how the position of the eye to be inspected deviates from the reference position.
    Information display device.
12. On the computer
    It is determined whether the first fundus image of the right eye of the subject is acquired and the second fundus image of the left eye of the subject is acquired.
    The position of the eye to be inspected is detected,
    The amount of deviation between the detected position and the reference position where the eye to be inspected should be located in order to photograph the fundus of the eye to be inspected is calculated.
    The detection is based on the first section for indicating that the first fundus image has been acquired or not acquired, the second section for indicating that the second fundus image has been acquired or not acquired, and the amount of deviation. Generates a display screen that includes a third section for displaying an image showing how the position of the eye to be inspected deviates from the reference position.
    A program that lets you do things.

Images