JP7560300B2 - Slit Lamp Microscope System - Google Patents

Description

The present invention relates to a slit lamp microscope system.

Diagnostic imaging plays an important role in the field of ophthalmology. A variety of ophthalmic imaging devices are used for diagnostic imaging. Types of ophthalmic imaging devices include slit lamp microscopes, fundus cameras, scanning laser ophthalmoscopes (SLO), and optical coherence tomography (OCT).

Of these various ophthalmic devices, the most widely and frequently used is the slit lamp microscope. Slit lamp microscopes are used to illuminate the subject's eye with a slit of light, and to observe and photograph the illuminated cross-section from an oblique or lateral angle using a microscope (see, for example, Patent Documents 1 and 2).

One of the main uses of slit lamp microscopes is for observing the anterior segment of the eye. In this case, doctors observe the entire anterior segment while moving the illumination field and focus position using slit light to determine whether there are any abnormalities. Slit lamp microscopes are also used when prescribing vision correction devices, such as checking the fitting of contact lenses. Furthermore, slit lamp microscopes are also used by people other than doctors, such as optometrists, paramedical staff, and eyeglass store clerks, for the purpose of screening for eye diseases and follow-up (progression observation).

Furthermore, with recent advances in information and communication technology, research and development into telemedicine is progressing. Telemedicine is the practice of providing medical treatment to patients in remote locations using a communication network such as the Internet. Patent documents 3 and 4 disclose technology for operating a slit lamp microscope from a remote location.

To obtain good images using a slit lamp microscope, delicate and complicated operations such as adjusting the illumination angle and the shooting angle are required. However, with the technology disclosed in Patent Documents 3 and 4, even when observing the eye of a subject who is right in front of the examiner, an examiner in a remote location must perform difficult operations, which results in problems such as longer examination times and failure to obtain good images.

As mentioned above, slit lamp microscopes are effective for screening, follow-up, and other examinations, but there is currently a shortage of people skilled in operating the equipment to provide high-quality slit lamp microscopy examinations to many people.

Furthermore, in remote medical care and screening, the doctor who interprets the images (image diagnosis) is often not present at the imaging location. In such cases, images that are not suitable for interpretation may be provided to the doctor. Considering the difficulty of operation and the length of time it takes to perform imaging, it is expected that this problem will be more likely to occur with slit lamp microscopes.

Recent research has revealed that there is a relationship between information about certain parts of the eye and diseases. For example, in the case of glaucoma, thinning of the retinal nerve fiber layer around the optic nerve progresses before the patient experiences any symptoms, and by the time symptoms appear, thinning has occurred by about 70% in the local area presenting with visual field defects and about 30% overall. Research is also being conducted on the fact that visual field defects due to geographic atrophy (GA) are caused by damage to the retinal pigment epithelium (RPE), and on the relationship between functional data such as blood flow dynamics and specific diseases. In general, the area where abnormalities occur varies depending on the type and progression of the disease.

Follow-up (observation of progress) is also used to understand the progression of a disease and the progress after treatment. Ophthalmic follow-up techniques are disclosed, for example, in Patent Document 5. Patent Document 5 discloses trend analysis for following up on an area specified by an examiner in an image of the eye.

In such conventional technology, even though the type and progression of a disease differs from patient to patient, follow-up is performed by uniformly calculating certain parameters, so it may not be possible to detect small local changes. In addition, in conventional technology, it is common to specify the area of interest each time an examination is performed, but this is time-consuming and there is a risk of errors in specifying the area.

JP 2016-159073 A JP 2016-179004 A JP 2000-116732 A JP 2008-284273 A JP 2014-83268 A

One objective of the present invention is to detect small local changes in the eye regardless of disease type, stage, or condition.

An exemplary embodiment of a slit lamp microscope system includes a reception unit, a position of interest information acquisition unit, an image acquisition unit, and an image analysis unit. The reception unit accepts patient identification information for identifying a patient. The position of interest information acquisition unit acquires position of interest information associated with the patient identification information accepted by the reception unit from a storage means in which position of interest information indicating the position of a region of interest is pre-associated with each of a plurality of pieces of patient identification information and stored. The image acquisition unit acquires an image by scanning the anterior segment of the subject eye of the patient corresponding to the patient identification information with slit light. The image analysis unit generates partial analysis data corresponding to a partial region of the image acquired by the image acquisition unit based on the position of interest information acquired by the position of interest information acquisition unit.

According to an exemplary embodiment, it becomes possible to capture small local changes in the eye regardless of the type, stage, or condition of the disease.

1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram for explaining the operation of a slit lamp microscope system according to an exemplary embodiment. FIG. 1 is a schematic diagram for explaining the operation of a slit lamp microscope system according to an exemplary embodiment. FIG. 1 is a schematic diagram for explaining the operation of a slit lamp microscope system according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. A schematic diagram for explaining the construction of an inference model of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a flowchart illustrating the operation of a slit lamp microscope system according to an exemplary embodiment. 1 is a flowchart illustrating the operation of a slit lamp microscope system according to an exemplary embodiment. 1 is a flowchart illustrating the operation of a slit lamp microscope system according to an exemplary embodiment. 1 is a flowchart illustrating the operation of a slit lamp microscope system according to an exemplary embodiment. 1 is a flowchart illustrating the operation of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram for explaining the operation of a slit lamp microscope system according to an exemplary embodiment. FIG. 1 is a schematic diagram for explaining the operation of a slit lamp microscope system according to an exemplary embodiment. FIG. 1 is a schematic diagram for explaining the operation of a slit lamp microscope system according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment.

Some exemplary aspects will be described in detail with reference to the drawings. Any known technology, such as those disclosed in the documents cited in this specification, may be combined with the exemplary aspects. The entire disclosure of other applications filed by the applicant may be incorporated into this disclosure. For example, at least a portion of the disclosure of an application filed by the applicant relating to a slit lamp microscope, at least a portion of the disclosure of an application filed by the applicant relating to a system including a slit lamp microscope, or at least a portion of the disclosure of an application filed by the applicant relating to a system connectable to a slit lamp microscope may be incorporated into this disclosure.

The slit lamp microscope system according to the exemplary embodiment may be either stationary or portable. The slit lamp microscope system according to the exemplary embodiment may be used in a situation or environment where there is no expert (expert) with expertise in the device or the examination, or where an expert is nearby, or where an expert can monitor, give instructions, and operate from a remote location. Examples of facilities in which the slit lamp microscope system according to the exemplary embodiment may be installed include eyeglass stores, optometrists, medical institutions, health checkup locations, medical examination locations, patients' homes, welfare facilities, public facilities, medical examination vehicles, etc.

When the slit lamp microscope system according to the exemplary embodiment includes two or more devices, these devices may be located in the same location or in different locations. For example, when the slit lamp microscope system includes a slit lamp microscope and an information processing device, the slit lamp microscope and the information processing device may be located remotely.

The slit lamp microscope system according to the exemplary embodiment has at least the imaging function of a slit lamp microscope and a predetermined information processing function, and may optionally further have imaging functions (modalities) other than the slit lamp microscope, an eye characteristic measurement function, other information processing functions, etc.

The system according to the exemplary embodiment (first system) may include one or more slit lamp microscopes, one or more information processing devices, and one or more image reading terminals, and may be usable for remote medical care, for example. One or more of the slit lamp microscope, the information processing device, and the image reading terminal may have an information processing function according to any of the exemplary embodiments. The slit lamp microscope may be a slit lamp microscope according to any of the exemplary embodiments, or may be a slit lamp microscope equipped with at least a part of the slit lamp microscope. The information processing device receives an image acquired by the slit lamp microscope and transmits it to the image reading terminal. The information processing device may also have a function of managing the image acquired by the slit lamp microscope. Note that an image may be transmitted from the slit lamp microscope to the image reading terminal without providing an information processing device. For example, the information processing device may be incorporated in the slit lamp microscope. The image reading terminal is a computer used by a doctor (typically a specialist such as an ophthalmologist or image reading doctor) to read an image acquired by the slit lamp microscope (observe the image to obtain a clinical finding). The information input by the image reader to the image reading terminal may be converted into an image reading report or electronic medical record information by the image reading terminal or another computer, and transmitted to an information processing device. Another exemplary embodiment may be configured such that the image reading terminal accepts the information input by the image reader, the image reading terminal transmits the accepted information to the information processing device, the information processing device receives the information transmitted from the image reading terminal, and the information processing device or another computer generates an image reading report or electronic medical record information from the received information. The information processing device may have a function of managing the image reading report or electronic medical record information. The information processing device may also have a function of transferring the image reading report or electronic medical record information to another device or another medical system (e.g., an electronic medical record system).

A system according to another exemplary embodiment (a second system) may include one or more slit lamp microscopes, one or more information processing devices, and one or more image reading devices. One or more of the slit lamp microscope, the information processing device, and the image reading device may have an information processing function according to any of the exemplary embodiments. The slit lamp microscope may be the same as the slit lamp microscope of the first system, and the information processing device may be the same as the information processing device of the first system. In addition, an image may be transmitted from the slit lamp microscope to the image reading device without providing an information processing device. For example, the information processing device may be incorporated in the slit lamp microscope. The image reading device is, for example, an image processing processor that operates according to a predetermined program, and/or a computer that performs image reading processing of an image acquired by the slit lamp microscope using an artificial intelligence engine (such as an inference engine, an inference model, or a learned model). The image reading device or another computer can generate an image reading report or electronic medical record information from information derived from the image by the image reading device. The generated image reading report or electronic medical record information may be transmitted to the information processing device. In another example, the image reading device can transmit information derived from the image to the information processing device. In this case, the information processing device or another computer can generate an image reading report or electronic medical record information from the information derived from the image by the image reading device. In yet another example, the information derived from the image by the image reading device and/or information generated based on this information (image reading report, electronic medical record information, etc.) can be provided to a doctor as diagnostic support information.

A system according to yet another exemplary embodiment (a third system) may include one or more slit lamp microscopes and one or more information processing devices. The slit lamp microscope and/or the information processing device may have an information processing function according to any of the exemplary embodiments. The slit lamp microscope may be similar to the slit lamp microscope of the first system, and the information processing device may be similar to the information processing device of the first system and/or the information processing device of the second system. In other words, the information processing device may be capable of data communication with an image reading terminal or a similar computer of the first system, and/or may be capable of data communication with an image reading device or a similar computer of the second system. The information processing device may be incorporated in the slit lamp microscope.

The slit lamp microscope system according to the exemplary embodiment can be used for remote medical care, but as mentioned above, it is not easy to obtain good images with a slit lamp microscope, and in order to effectively interpret and diagnose images, it is necessary to obtain images of a wide range of the anterior segment "in advance." For these reasons, it can be said that effective remote medical care using a slit lamp microscope has not yet been realized. The exemplary embodiment can provide technology that contributes to the realization of this. However, the use of the exemplary embodiment is not limited to remote medical care, and the exemplary embodiment can also be applied to other uses.

The slit lamp microscope system according to the exemplary embodiment can be used to obtain a series of images (image group, image set) that represent a wide area of the anterior segment with good image quality.

The exemplary embodiment also focuses on the following problem. That is, in the application form (such as remote medical care) assumed by the exemplary embodiment, a series of images (a group of images, a set of images) from a sufficiently wide range of the anterior segment is directly or indirectly provided to the reader using a slit lamp microscope. Even if the quality of the series of images provided is low, it is difficult to retake them, and as a result, there is a risk of problems such as no interpretation being possible or only insufficient interpretation being possible. Therefore, it is necessary to obtain images of "good" quality "in advance." In other words, it is necessary to obtain a series of images with a quality that allows effective diagnosis (such as interpretation) all at once before providing them to the reader. However, considering the difficulty of operating a slit lamp microscope and the occurrence of blinking and eye movement during imaging, it is extremely difficult to obtain a series of images that represent the entire target area of observation or interpretation with good image quality. The present disclosure also takes into consideration how to deal with such problems.

Below, several exemplary aspects are described. Any two or more of these aspects can be at least partially combined. In addition, any known technology can be combined with such combinations, and modifications (substitutions, omissions, etc.) can be made based on them.

At least a portion of the functionality of the elements disclosed in this specification is implemented using circuitry or processing circuitry. The circuitry or processing circuitry may be a general purpose processor, a special purpose processor, an integrated circuit, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), a Field Programmable Gate Array (FPGA), or a combination of a number of different devices configured and/or programmed to perform at least some of the disclosed functions. Array), conventional circuitry, and any combination thereof. A processor is considered to be a processing circuitry or circuitry, including transistors and/or other circuitry. In this disclosure, a circuitry, unit, means, or similar term is hardware that performs at least some of the disclosed functions or hardware that is programmed to perform at least some of the disclosed functions. The hardware may be hardware disclosed herein, or may be known hardware that is programmed and/or configured to perform at least some of the described functions. If the hardware is a processor that can be considered as a type of circuitry, the circuitry, unit, means, or similar term is a combination of hardware and software, and the software is used to configure the hardware and/or the processor.

<Configuration of the slit lamp microscope system>
The slit lamp microscope system according to the exemplary subject matter may be, for example, a slit lamp microscope equipped with an information processing function, or a combination of a slit lamp microscope and an information processing device. The configuration of a slit lamp microscope system according to one exemplary embodiment is shown in Figures 1A and 1B. Figure 1A shows an example of the overall configuration of a slit lamp microscope system 1, and Figure 1B shows an example of the configuration of a data processing unit 8 and a storage unit 10 of the slit lamp microscope system 1. In Figure 1A, the symbol C indicates the cornea, the symbol CL indicates the crystalline lens, the symbol IR indicates the iris, and the symbol CA indicates the angle of the eye.

The slit lamp microscope system 1 is equipped with a configuration for automatically generating analysis data for the region of interest (ROI) for each patient (examined eye), making it possible to capture small local changes in the examined eye regardless of the disease type, stage, or pathology, and also optimizing (automating, simplifying, facilitating, streamlining, etc.) at least some of the steps in medical treatment related to the region of interest (observation, analysis, follow-up, etc.).

The region of interest is a location of interest in medical treatment, and may be, for example, a lesion, an abnormality, a site to be diagnosed, a site to be treated, or the like. In some exemplary embodiments, the region of interest may include a region corresponding to any of the cornea, iris, pupil, lens, artificial object, ciliary body, angle, trabecular meshwork, and Schlemm's canal. The artificial object is, for example, an object implanted in the subject's eye, and examples of such an object include an intraocular lens (IOL), a phakic intraocular lens (phakic IOL), a posterior chamber phakic intraocular lens (ICL), a minimally invasive glaucoma surgery (MIGS) device, and a stent.

As shown in FIG. 1A, the slit lamp microscope system 1 includes an illumination system 2, an imaging system 3, a moving mechanism 6, a control unit 7, a data processing unit 8, a communication unit 9, and a memory unit 10. As shown in FIG. 1B, the slit lamp microscope system 1 also includes a reception unit 11. Details will be described later, but the reception unit 11 may be composed of at least a part of any of the elements shown in FIG. 1A, or may be composed of at least a part of other elements, or may be composed of a combination thereof.

The slit lamp microscope system 1 may be a single device, or may be a system including two or more devices. As an example of the latter, the slit lamp microscope system 1 includes a main device including an illumination system 2, an imaging system 3, and a moving mechanism 6, a computer including a control unit 7, a data processing unit 8, a communication unit 9, a storage unit 10, and a reception unit 11, and a communication interface that handles data communication between the main device and the computer. This computer may be installed, for example, together with the main device, or may be installed on a network. Examples of this computer include the above-mentioned information processing device, image reading terminal, and image reading device.

<Reception Unit 11>
The accepting unit 11 accepts an identifier of a patient (subject). The accepting unit 11 may have any configuration or aspect.

A patient identifier is called patient identification information, subject identification information, patient ID, subject ID, etc. A patient ID is an at least locally unique code (character string information, image information, etc.) that is assigned to each patient. For example, a patient ID may be a code that is uniquely assigned to each patient at a certain medical institution, or a public code that is uniquely assigned to each patient in a certain value range (country, prefecture, etc.).

The reception unit 11 in some exemplary embodiments includes a device that receives input of a patient ID and converts it into a signal, and a control unit 7 that receives the signal from this device. This device may be, for example, a user interface (man-machine interface) such as a keyboard or a touch screen, or a data reader that reads the patient ID from a recording medium. The slit lamp microscope system 1 (control unit 7, data processing unit 8, etc.) can perform patient authentication.

<Storage unit 10>
The storage unit 10 stores various types of data. For example, the storage unit 10 stores an image group collected by scanning the anterior segment of the subject's eye E with slit light, an image set created from two or more image groups, analysis data generated by image analysis, a processed image constructed from an image group or an image set, and the like. The control unit 7 executes a process of storing data in the storage unit 10 and a process of reading data from the storage unit 10. The storage process and read process are executed as appropriate in various operations of the slit lamp microscope system 1.

Furthermore, the storage unit 10 prestores related information 100. The related information 100 at least associates multiple patient IDs with multiple pieces of location information of interest. For example, the related information 100 associates each of the multiple patient IDs with location information of interest.

The interest position information indicates the position of the region of interest. The position of the region of interest may be, for example, any of the following: the position of the cornea, the position of the iris, the position of the pupil, the position of the lens, the position of an artificial object, the position of the ciliary body, the position of the angle, the position of the trabecular meshwork, and the position of the Schlemm's canal. In some exemplary aspects, the position of the region of interest may be any of the following: the position of anterior/posterior corneal features (apex, inflammation, tumor, etc.), the position of internal corneal features (changes due to surgery, changes due to treatment, position of the corneal circumference, changes due to medication, etc.), the position of anterior/posterior lens capsule features (apex, etc.), the position of internal lens features (center, center of gravity, opacity, etc.), the position of anterior/posterior IOL features (apex, stain, scratch, etc.), and the position of the circumference of the angle. In some exemplary embodiments, the region of interest may be a sub-tissue of the cornea (such as the anterior corneal surface, the posterior corneal surface, the corneal epithelium, Bowman's membrane, the lamina propria, Duar's layer, Descemet's membrane, or the corneal endothelium) or a sub-tissue of the lens (such as the anterior lens surface, the posterior lens surface, the lens nucleus, the epithelium, or the lens capsule).

An example of related information 100 is shown in FIG. 2. In this example, related information 101 associates target eye information (right eye/left eye, R/L), position of interest information (ROI(1R), ROI(1L), ROI(2R), ...), and parameter type information (P(1R), P(1L), P(2R), ...) with each of multiple patient IDs (PID(1), PID(2), ...). For example, related information 101 associates target eye information "R", position of interest information "ROI(1R)", and parameter type information "P(1R)" with patient ID "PID(1)", and also associates target eye information "L", position of interest information "ROI(1L)", and parameter type information "P(1L)". In other words, the right eye "R" of the patient with patient ID "PID(1)" is associated with the position of interest information "ROI(1R)" and the parameter type information "P(1R)," and the left eye "L" is associated with the position of interest information "ROI(1L)" and the parameter type information "P(1L)." The same applies to the other patient IDs.

The parameter type information is information indicating the type of the parameter to be calculated. The parameter type information is information that is adopted arbitrarily, and may be changeable after the fact. For example, a doctor or the like can observe the part of the subject's eye (anterior segment) corresponding to the interest position information, and can specify the type of the parameter that is determined to be used for follow-up of the part using a user interface. In response to this specification, the control unit 7 (or another information processing device) registers the parameter type information in association with the patient ID of this patient. If the doctor or the like edits (adds, deletes, changes, etc.) the parameters using the user interface during subsequent follow-up, the control unit 7 (or another information processing device) can update the parameter type information based on the content of the edits made by the doctor.

In addition, the type of parameters to be applied generally varies depending on the type of disease and the type of region of interest. For example, parameters related to the angle are applied to eyes with glaucoma, and parameters related to lens opacity are applied to eyes with cataracts. For other diseases, parameters corresponding to the diseases are also applied. Therefore, the slit lamp microscope system 1 can be provided with the following functions: a function for pre-storing information in which disease types and/or region of interest types are associated with parameter types; a function for accepting input of disease types and/or region of interest types; a function for identifying parameter types associated with the input information from the above information; a function for associating the identified parameter types and the input region of interest types with a patient ID. As described above, the region of interest type may be determined according to the disease type. In that case, it is possible to further provide a function for pre-storing information in which one or more region of interest types are associated with the disease type, and a function for identifying region of interest types associated with the input disease type from the information. Note that even when a configuration for automatically setting parameter types (and region of interest types) is applied in this way, a function for allowing the user to change the automatically set content after the fact may be provided.

In some exemplary aspects of the related information 100, while allowing "both eyes" to be specified as the target eye information, common location of interest information and/or parameter type information may be associated with both the right and left eyes. Such related information 100 is applied, for example, to a disease that progresses (almost) the same in both the left and right eyes. In addition, some exemplary aspects of the related information 100 may be configured to associate location of interest information and parameter type information with a patient ID without including target eye information. Such related information 100 is applied, for example, to a disease that progresses (almost) the same in both the left and right eyes, or is applied when an examination is performed on only one eye.

<Lighting system 2>
The illumination system 2 irradiates slit light onto the anterior segment of the subject's eye E. Reference symbol 2a indicates the optical axis (illumination optical axis) of the illumination system 2. The illumination system 2 may have a configuration similar to that of an illumination system of a conventional slit lamp microscope. Although not shown in the figure, the exemplary illumination system 2 includes, in order from the side farthest from the subject's eye E, an illumination light source, a positive lens, a slit forming unit, and an objective lens.

The illumination light source outputs illumination light. The illumination system 2 may include multiple illumination light sources. For example, the illumination system 2 may include an illumination light source that outputs continuous light and an illumination light source that outputs flash light. The illumination system 2 may also include an illumination light source for the anterior segment and an illumination light source for the posterior segment. The illumination system 2 may also include two or more illumination light sources with different output wavelengths. A typical illumination system 2 includes a visible light source as an illumination light source. The illumination system 2 may also include an infrared light source. The illumination light output from the illumination light source passes through a positive lens and is projected onto the slit forming section.

The slit forming unit generates slit light by passing a portion of the illumination light. A typical slit forming unit has a pair of slit blades. The width of the area (slit) through which the illumination light passes is changed by changing the distance between the slit blades (slit width), and thus the width of the slit light is changed. The slit forming unit may also be configured to change the length of the slit light. The length of the slit light is the cross-sectional dimension of the slit light in a direction perpendicular to the cross-sectional width direction of the slit light corresponding to the slit width. The width and length of the slit light are typically expressed as the dimensions of the image of the slit light projected onto the anterior segment of the eye, but are not limited to this. For example, they can be expressed as the dimensions of the cross section of the slit light at any position, or as the dimensions of the slit formed by the slit forming unit.

The slit light generated by the slit forming unit is refracted by the objective lens and irradiated onto the anterior segment of the subject's eye E.

The illumination system 2 may further include a focusing mechanism for changing the focus position of the slit light. The focusing mechanism, for example, moves the objective lens along the illumination optical axis 2a. The movement of the objective lens can be performed automatically and/or manually. Note that a focusing lens may be disposed at a position on the illumination optical axis 2a between the objective lens and the slit forming section, and the focus position of the slit light may be changed by moving this focusing lens along the illumination optical axis 2a.

Note that FIG. 1A is a top view, and as shown in the figure, in this embodiment, the direction along the axis of the subject's eye E is the Z direction, the direction perpendicular to this that is the left-right direction for the subject is the X direction, and the direction perpendicular to both the X and Z directions is the Y direction. Typically, the X direction is the alignment direction of the left and right eyes, and the Y direction is the direction along the subject's body axis (body axis direction).

<Photography Series 3>
The imaging system 3 images the anterior segment of the eye illuminated with the slit light from the illumination system 2. Reference numeral 3a denotes an optical axis (imaging optical axis) of the imaging system 3. The imaging system 3 in this embodiment includes an optical system 4 and an image sensor 5.

The optical system 4 guides light from the anterior segment of the subject's eye E, which is irradiated with the slit light, to the image sensor 5. The image sensor 5 receives the light guided by the optical system 4 on the imaging surface.

The light guided by the optical system 4 (light from the anterior segment of the subject's eye E) includes the return light of the slit light irradiated on the anterior segment, and may further include other light. Examples of the return light include reflected light, scattered light, and fluorescence. Examples of other light include light from the environment (room light, sunlight, etc.). When an anterior segment illumination system for illuminating the entire anterior segment is provided separately from the illumination system 2, the return light of this anterior segment illumination light is included in the light guided by the optical system 4.

The optical system 4 may have a configuration similar to that of the imaging system of a conventional slit lamp microscope. For example, the optical system 4 includes, in order from the side closest to the subject's eye E, an objective lens, a variable magnification optical system, and an imaging lens. Light from the anterior segment of the subject's eye E irradiated with the slit light passes through the objective lens and the variable magnification optical system, and is imaged on the imaging surface of the image sensor 5 by the imaging lens.

The imaging element 5 is an area sensor having a two-dimensional imaging area, and may be, for example, a charge-coupled device (CCD) image sensor or a complementary metal-oxide semiconductor (CMOS) image sensor.

The imaging system 3 may further include a focusing mechanism for changing the focus position. The focusing mechanism, for example, moves the objective lens along the imaging optical axis 3a. The movement of the objective lens can be performed automatically and/or manually. Note that the focus position may be changed by disposing a focusing lens at a position on the imaging optical axis 3a between the objective lens and the imaging lens, and moving this focusing lens along the imaging optical axis 3a.

The illumination system 2 and the imaging system 3 function as a Scheimpflug camera. That is, the illumination system 2 and the imaging system 3 are configured so that the object plane along the illumination optical axis 2a, the optical system 4, and the imaging surface of the imaging element 5 satisfy the so-called Scheimpflug condition. More specifically, the YZ plane (including the object plane) passing through the illumination optical axis 2a, the principal plane of the optical system 4, and the imaging surface of the imaging element 5 intersect on the same straight line. This makes it possible to focus on all positions within the object plane (all positions in the direction along the illumination optical axis 2a) and perform imaging.

In this embodiment, for example, imaging is performed with the imaging system 3 focused on a three-dimensional region including at least a portion of a specific area of the anterior eye. With regard to the Z direction, for example, imaging can be performed with the imaging system 3 focused on the entire range from the apex of the anterior surface of the cornea C (Z = Z1) to the apex of the posterior surface of the crystalline lens CL (Z = Z2). Note that Z = Z0 indicates the Z coordinate of the intersection of the illumination optical axis 2a and the imaging optical axis 3a.

Such conditions are typically realized by the configuration and arrangement of elements included in the illumination system 2, the configuration and arrangement of elements included in the imaging system 3, and the relative positions of the illumination system 2 and the imaging system 3. Parameters indicating the relative positions of the illumination system 2 and the imaging system 3 include, for example, the angle θ between the illumination optical axis 2a and the imaging optical axis 3a. The angle θ is set to, for example, 17.5 degrees, 30 degrees, or 45 degrees. Note that the angle θ may be variable.

<Moving mechanism 6>
The moving mechanism 6 moves the illumination system 2 and the imaging system 3. The moving mechanism 6 includes, for example, a movable stage on which the illumination system 2 and the imaging system 3 are mounted, an actuator that operates according to a control signal input from the control unit 7, and a mechanism that moves the movable stage based on a driving force generated by the actuator. In another example, the moving mechanism 6 includes a movable stage on which the illumination system 2 and the imaging system 3 are mounted, and a mechanism that moves the movable stage based on a force applied to an operation device (not shown). The operation device is, for example, a lever. The movable stage is movable at least in the X direction, and may further be movable in the Y direction and/or the Z direction.

In this embodiment, the moving mechanism 6 moves the illumination system 2 and the imaging system 3 together in the X direction, for example. That is, the moving mechanism 6 moves the illumination system 2 and the imaging system 3 in the X direction while maintaining a state in which the above-mentioned Scheimpflug condition is satisfied. In parallel with this movement, the imaging system 3 captures video at a predetermined time interval (imaging rate), for example. As a result, a three-dimensional region of the anterior segment of the subject's eye E is scanned with slit light, and multiple images (image group) corresponding to multiple cross sections within this three-dimensional region are collected.

In addition, in a typical example, the longitudinal direction (length direction) of the slit light irradiated to the anterior segment by the illumination system 2 is aligned with the Y direction, and the transverse direction (width direction) is aligned with the X direction. Furthermore, the movement mechanism 6 moves the illumination system 2 and the imaging system 3 integrally in the X direction. In other words, the movement mechanism 6 moves the illumination system 2 and the imaging system 3 so that the slit light irradiated to the anterior segment by the illumination system 2 moves in the width direction of the slit light. This makes it possible to simply and efficiently scan the three-dimensional region of the anterior segment. Note that any combination of the orientation of the slit light and the movement direction may be used.

<Control Unit 7>
The control unit 7 controls each unit of the slit lamp microscope system 1. For example, the control unit 7 controls elements of the illumination system 2 (illumination light source, slit forming unit, focusing mechanism, etc.), elements of the imaging system 3 (focusing mechanism, image sensor, etc.), the moving mechanism 6, the data processing unit 8, the communication unit 9, etc. The control unit 7 may also be capable of executing control for changing the relative position between the illumination system 2 and the imaging system 3. Furthermore, the control unit 7 may also be capable of controlling a device (e.g., a display device) included in the slit lamp microscope system 1 or an apparatus (e.g., a display device) connected to the slit lamp microscope system 1.

The control unit 7 includes a processor, a main memory device, an auxiliary memory device, etc. The auxiliary memory device stores a control program, etc. The control program, etc. may be stored in a computer or a memory device that is accessible to the slit lamp microscope system 1. The functions of the control unit 7 are realized by the cooperation of software such as the control program and hardware such as a processor.

The control unit 7 can apply the following control to the illumination system 2, the imaging system 3, and the movement mechanism 6 in order to scan the three-dimensional area of the anterior segment of the subject's eye E with slit light.

First, the control unit 7 controls the movement mechanism 6 to place the illumination system 2 and the imaging system 3 at a predetermined scan start position (alignment control). The scan start position is, for example, a position corresponding to the end (first end) of the cornea C in the X direction, or a position further away from the axis of the subject's eye E.

The symbol X0 in FIG. 4A shows an example of a scan start position that corresponds to the first end of the cornea C in the X direction. Also, the symbol X0' in FIG. 4B shows an example of a scan start position that is farther away from the axis EA of the test eye E than the position that corresponds to the first end of the cornea C in the X direction.

The control unit 7 controls the illumination system 2 to start irradiating the anterior segment of the subject's eye E with slit light (slit light irradiation control). Note that the slit light irradiation control may be performed before or during the execution of the alignment control. The slit light is typically continuous light, but may also be intermittent light (pulsed light). The control of the pulsed light is synchronized with the imaging rate of the imaging system 3. Furthermore, the slit light is typically visible light, but may also be infrared light, or a mixture of visible light and infrared light.

The control unit 7 controls the imaging system 3 to start imaging of the anterior segment of the subject's eye E (imaging control). Note that imaging control may be performed before or during the execution of alignment control. Typically, imaging control is performed simultaneously with or after slit light irradiation control.

After performing the alignment control, the slit light irradiation control, and the photographing control, the control unit 7 controls the moving mechanism 6 to start moving the illumination system 2 and the photographing system 3 (movement control). The movement control moves the illumination system 2 and the photographing system 3 as a unit. That is, the illumination system 2 and the photographing system 3 are moved while maintaining the relative position (angle θ, etc.) between the illumination system 2 and the photographing system 3. Typically, the illumination system 2 and the photographing system 3 are moved while maintaining a state in which the above-mentioned Scheimpflug condition is satisfied. The illumination system 2 and the photographing system 3 are moved from the above-mentioned scan start position to a predetermined scan end position. The scan end position is, for example, a position corresponding to the end (second end) of the cornea C opposite the first end in the X direction, similar to the scan start position, or a position further away from the axis of the subject's eye E. In such a case, the range from the scan start position to the scan end position is the scan range.

Typically, a slit of light with the X direction as the width direction and the Y direction as the length direction is irradiated onto the anterior segment, and the illumination system 2 and the imaging system 3 are moved in the X direction while the imaging system 3 captures a moving image.

Here, the length of the slit light (i.e., the dimension of the slit light in the Y direction) is set, for example, to be greater than or equal to the diameter of the cornea C on the surface of the subject's eye E. That is, the length of the slit light is set to be greater than or equal to the corneal diameter in the Y direction. Also, as described above, the movement distance of the illumination system 2 and the imaging system 3 by the movement mechanism 6 (i.e., the scanning range) is set to be greater than or equal to the corneal diameter in the X direction. This makes it possible to scan at least the entire cornea C with the slit light.

By performing such a scan, multiple anterior eye images with different irradiation positions of the slit light are obtained. In other words, a moving image is obtained that depicts the irradiation position of the slit light moving in the X direction. An example of multiple such anterior eye images (i.e., a group of frames that make up a moving image) is shown in Figure 5.

Figure 5 shows multiple anterior eye images (frame groups, image groups) F1, F2, F3, ..., FN. The subscript n of these anterior eye images Fn (n = 1, 2, ..., N) represents the chronological order. That is, the anterior eye image acquired nth is represented by the symbol Fn. The anterior eye image Fn includes a slit light irradiation area An. As shown in Figure 5, the slit light irradiation areas A1, A2, A3, ..., AN move to the right in the chronological order.

In the example shown in FIG. 5, the longitudinal direction (length direction) of the slit light coincides with the Y direction (up-down direction, body axis direction), the transverse direction (width direction) coincides with the X direction (left-right direction, horizontal direction), and the moving direction of the slit light coincides with the X direction. In addition, the scan start position is a position outside (left side) of the first end (left end) of the cornea C, and the scan end position is a position outside (right side) of the second end (right end) of the cornea C. According to such a scanning mode, a three-dimensional region including the entire cornea C can be photographed. Note that the scan start position and/or scan end position are not limited to this example, and for example, the scan start position and/or scan end position may be the end of the cornea C in the X direction. In addition, various scanning parameters such as the direction of the slit light, the moving direction of the slit light, and the number of scans can also be set arbitrarily.

<Data Processing Unit 8>
The data processing unit 8 executes various types of data processing. The data to be processed may be either data acquired by the slit lamp microscope system 1 or data input from the outside. The data processing unit 8 can process images acquired by the imaging system 3.

The data processing unit 8 includes a processor, a main memory device, an auxiliary memory device, etc. The auxiliary memory device stores a data processing program, etc. The data processing program, etc. may include a model constructed by machine learning (a trained model, an inference model, etc.). The data processing program, etc. may be stored in a computer or a memory device accessible to the slit lamp microscope system 1. The functions of the data processing unit 8 are realized by cooperation between software such as the data processing program and hardware such as a processor.

The exemplary data processing unit 8 shown in FIG. 1B includes an image group processing unit 80, a processed image construction unit 85, a position of interest information acquisition unit 86, an identification unit 87, an analysis unit 88, and an update unit 89. The identification unit 87 and the analysis unit 88 function as an image analysis unit.

<Image Group Processing Unit 80>
The image group processor 80 processes at least some of the images collected by the anterior eye scan using slit light. Several examples of the image group processor 80 will be described. Figures 6A, 6B, 6C, and 6D show image group processors 80A, 80B, 80C, and 80D, which are first, second, third, and fourth examples of the image group processor 80, respectively. Note that the configuration of the image group processor 80 is not limited to these. For example, the image group processor 80 may include a combination of any two or more of the four image group processors 80A, 80B, 80C, and 80D. In addition, it is possible to provide the image group processor 80 with any element for obtaining the same type of result or a similar type of result.

The slit lamp microscope system 1 of this embodiment applies one or more scans to the anterior segment of the subject's eye E. Each scan results in an image group such as the multiple anterior segment images F1 to FN shown in FIG. 5. The image group processing unit 80 can process one or more image groups thus obtained.

The slit lamp microscope system 1 may be capable of applying two or more scans to the anterior segment. For example, if a good set of images is not obtained in the first scan, a second scan is performed. More generally, if a good set of images is not obtained in the first through v scans, a v+1 scan is performed (v is an integer equal to or greater than 1). In some exemplary aspects, if a good set of images (image set) is not obtained from the v image sets obtained in the first through v scans, a v+1 scan is performed. In some other exemplary aspects, if none of the v image sets obtained in the first through v scans is a good image set, a v+1 scan is performed. Note that the manner in which two or more scans are performed is not limited to these.

When performing two or more scans, the start and end positions of these scans (i.e., the scan range) may be the same, or one or both of the start and end positions may be different. Typically, two or more scans are applied to the same scan range. As a result, a group of images such as the multiple anterior eye images F1 to FN shown in FIG. 5 is obtained in the same number as the number of scans.

Taking into consideration the eye movement of the subject's eye E, the scan range (scan start position, scan end position) is typically defined by the operation of the slit lamp microscope system 1, not by the position or range in the anterior segment. For example, the start point of the movement of the illumination system 2 and the imaging system 3 during scanning is set as the scan start position, and the end point is set as the scan end position.

On the other hand, it is also possible to define the scan range based on the position and range in the anterior segment. In this case, an operation (tracking) is applied in which the illumination system 2 and the imaging system 3 are moved so as to follow the movement of the subject's eye E. The slit lamp microscope system 1 of this example has, for example, a tracking function similar to that of a conventional system. The tracking function includes, for example, video recording of the anterior segment, extraction of landmarks from each frame, calculation of deviation between frames using the landmarks, and movement control of the illumination system 2 and the imaging system 3 to cancel out the deviation.

As another example of defining (setting) the scan range, it is possible to set the scan range for an image that has already been collected. In other words, it is possible to configure the scan range to be set retroactively (after scanning). Note that the scan range in this example represents the range of the image that will be provided for subsequent processing.

A first example of such a post-event scan range setting will be described. The slit lamp microscope of this example is capable of anterior segment photography, and is capable of recognizing the positional relationship between the anterior segment photography optical system and the scanning optical system (illumination system 2 and photography system 3). In this example, first, an image group (wide-area image group) is collected by scanning a sufficiently wide range (particularly in the X direction and Y direction) of the anterior segment, while performing anterior segment photography. Next, a scan range is set for the anterior segment image obtained by anterior segment photography. This setting may be performed either manually or automatically. The automatic setting includes, for example, a process of analyzing the anterior segment image to detect a landmark, and a process of setting the scan range by referring to this landmark. This landmark may be, for example, the corneal limbus, the angle, or the pupil margin. When the scan range is set, a partial area of each wide-area image corresponding to this scan range is specified based on the positional relationship of the optical system described above. Finally, an image corresponding to the scan range is formed by cropping this specified partial area from the wide-area image. As a result, a group of images corresponding to the set scan range is obtained. In this example, when the fixation of the subject's eye is stable (or when it is assumed or can be assumed that the fixation is stable), the anterior segment may be captured as still images. When the anterior segment is captured as video, the wide-area image group and the frame group in the video are associated with each other based on, for example, control of the scan and video capture (synchronization information, etc.), and the same processing as described above is performed for each associated pair of wide-area image and frame.

A second example of the post-scan range setting will be described. In this example, the anterior eye does not need to be photographed in parallel, and a sufficiently wide range of the anterior eye (particularly in the X and Y directions) is scanned to collect a group of wide-area images. Next, a partial area corresponding to the desired scan range is specified for each wide-area image. This specification may be performed either manually or automatically. The automatic specification includes, for example, a process of analyzing the wide-area image to detect a landmark (for example, the corneal limbus or the iridocorneal angle) and a process of setting the scan range by referring to the landmark. In addition, a landmark may be manually specified for one of the wide-area images (reference wide-area image), and the reference wide-area image and an adjacent wide-area image may be analyzed to specify a landmark for the adjacent wide-area image. By sequentially applying such processing, it is possible to specify landmarks for all wide-area images. Furthermore, a partial area of the wide-area image corresponding to the scan range can be specified based on the landmark, and an image corresponding to the scan range can be formed by cropping the specified partial area from the wide-area image. This allows a group of images corresponding to the set scan range to be obtained.

A first example of the image group processing unit 80 will be described. The image group processing unit 80A shown in FIG. 6A includes an image group evaluation unit 81. The image group evaluation unit 81 evaluates the quality of an image group collected in one scan of the subject's eye E. In some exemplary aspects, the image group evaluation unit 81 is configured to determine whether the image group satisfies a predetermined condition, and is typically configured to determine whether each image included in the image group satisfies the predetermined condition.

The specified conditions (image group evaluation conditions) are, for example, conditions related to the image quality required for effective interpretation and diagnosis, such as conditions related to the subject's eye E, conditions related to the slit lamp microscope system 1, and conditions related to the environment. The image group evaluation conditions may include, for example, either or both of "the image to be evaluated is not affected by blinking (blink condition)" and "the image to be evaluated is not affected by eye movement (eye movement condition)." The image group evaluation conditions may be any index related to image quality evaluation, and may include, for example, any of brightness, contrast, noise, signal-to-noise ratio (SNR), tone reproduction, dynamic range, sharpness, vignetting, aberration (spherical aberration, astigmatism, coma aberration, field curvature, distortion aberration, chromatic aberration, etc.), color accuracy, exposure accuracy, lens flare, moire, and artifacts. The image selection conditions are not limited to these examples and may be selected arbitrarily.

The blinking condition will now be described. For an image included in the image group, the image group evaluation unit 81 judges, for example, whether the image contains a reflected image of slit light irradiated onto the anterior segment of the subject's eye E. This judgment is a process that utilizes the fact that the reflected image of the slit light is not depicted in an image captured during a blink, and that the reflected image of the slit light is depicted significantly brighter than other regions, and is performed, for example, based on the luminance distribution in the image.

As an example, the image group evaluation unit 81 creates a luminance histogram from the image and determines whether there are any pixels with a luminance equal to or greater than a predetermined threshold. If it is determined that there is a luminance equal to or greater than the predetermined threshold, it is determined that the image contains a reflected image of slit light. This example has the advantage that the processing is extremely simple, but there is a risk of false detection of high-luminance noise or reflections of external light.

As another example, the image group evaluation unit 81 creates a brightness histogram from the image and determines whether there are a predetermined number of pixels with a brightness equal to or greater than a predetermined threshold. If it is determined that there are a predetermined number of pixels with a brightness equal to or greater than a predetermined threshold, it is determined that the image contains a reflected image of slit light. This example has the advantage that the above-mentioned erroneous detection can be prevented by simple processing.

Explains eye movement conditions. For an image included in an image group, the image group evaluation unit 81 determines whether or not the image is affected by eye movement by, for example, comparing the image with adjacent images. This determination is a process that utilizes the fact that "image jumps" occur when eye movement occurs during video capture.

As an example, the image group evaluation unit 81 detects landmarks from each of the image in question and the adjacent images, calculates the amount of deviation of these landmarks, and determines whether this amount of deviation is equal to or greater than a predetermined threshold. If it is determined that the amount of deviation is equal to or greater than the predetermined threshold, it is determined that eye movement has occurred. Here, the landmarks may be, for example, the cornea, iris, pupil, or angle of the eye. In addition, the threshold is calculated based on predetermined scanning conditions, such as the shooting rate of the shooting system 3 and the moving speed of the moving mechanism 6.

In another example, the image group evaluation unit 81 may be configured to determine the presence or absence of eye movement from a single image. For example, when the shooting rate of the shooting system 3 is slow, rapid eye movement may cause "blur" in the image. The image group evaluation unit 81 can determine the presence or absence of eye movement by using blur detection. Blur detection is typically performed using known techniques such as edge detection.

The image group evaluation unit 81 may include an artificial intelligence engine for determining whether the input image satisfies the above image selection conditions. This artificial intelligence engine typically includes a convolutional neural network (CNN), which is trained in advance using training data including a large number of images acquired by a slit lamp microscope and a determination result of whether each image satisfies the image selection conditions. The images included in the training data are not limited to images acquired by a slit lamp microscope, and may be, for example, images acquired by other ophthalmic modalities (fundus cameras, OCT devices, SLOs, surgical microscopes, etc.), images acquired by image diagnostic modalities of other medical departments (ultrasound diagnostic devices, X-ray diagnostic devices, X-ray CT devices, magnetic resonance imaging (MRI) devices, etc.), images generated by processing actual images, pseudo images, etc. In addition, the methods and technologies (for example, hardware, software, machine learning methods, types of neural networks, etc.) used in the artificial intelligence engine are arbitrary.

A series of images constituting an image group collected in one scan may be associated with multiple positions in the scan range. This association is performed, for example, by the image group processing unit 80. A specific example will be described. The scan range along the X direction is divided into (N-1) sections, the scan start position is set to the first position, and the scan end position is set to the Nth position (N is an integer equal to or greater than 2). This sets N positions in the scan range. The N positions are represented by B1, B2, B3, ..., BN (not shown). The intervals between the N positions B1 to BN may be equal or different. The number (N) of positions B1 to BN is set based on, for example, the scan start position, the scan end position, the moving speed (scanning speed) of the slit light, the frame rate of the imaging system 3, and the like. The number (N) of positions B1 to BN set in this example is equal to the number of images collected in one scan. Note that the number (N) of positions B1 to BN is not limited to this example, and the setting method is also not limited to this example. Assume that one scan is applied to the anterior segment of the subject's eye E, and a group of images F1, F2, F3, ..., FN shown in FIG. 5 is obtained. The image group processing unit 80 can assign an image Fn to a position Bn. This results in N images Fn (n = 1, 2, ..., N) corresponding to the N positions Bn (n = 1, 2, ..., N).

The image group may include only a series of images, or may further include other information. Examples of information included in the image group along with the series of images include various types of additional information, such as subject information, information about the examined eye, shooting date and time, and shooting conditions. It is also possible to include images obtained with other modalities and examination data acquired by an examination device in the image group. Some examples of the configuration and operation of the image group evaluation unit 81 will be described later with reference to Figures 7A to 7D.

A second example of the image group processing unit 80 will be described. The effectiveness of this example is particularly evident when two or more scans are applied to the subject's eye E. The image group processing unit 80B shown in FIG. 6B includes an image set creation unit 82 in addition to the image group evaluation unit 81 similar to that in FIG. 6A. The image group processing unit 80B is provided with two or more image groups collected in two or more scans applied to the subject's eye E. The image group evaluation unit 81 in this example may perform quality evaluation on each of these image groups, or may perform quality evaluation only on the image group collected in the initial scan (first scan). The image set creation unit 82 selects a series of images corresponding to the scan range from the two or more image groups input to the image group processing unit 80B to create an image set.

The range represented by the series of images constituting the image set may be, for example, a three-dimensional area to which any one of the two or more scans is applied, or may be a three-dimensional area set based on the scan ranges in at least two of the two or more scans. As an example of the former, the maximum or minimum scan range among the two or more scan ranges to which the two or more scans are respectively applied may be adopted. As an example of the latter, the union or intersection of at least two scan ranges may be adopted.

The image set may include only the series of images, or may include other information as well. Examples of information included in the image set along with the series of images include various types of additional information, such as subject information, information about the examined eye, shooting date and time, and shooting conditions. Images obtained with other modalities and examination data acquired by an examination device can also be included in the image set.

A third example of the image group processing unit 80 will be described. As with the second example, the effectiveness of this example is particularly evident when two or more scans are applied to the subject eye E. The image group processing unit 80C shown in FIG. 6C includes an image group evaluation unit 81 similar to that in FIG. 6A, as well as an image set creation unit 82 similar to that in FIG. 6B, but the image set creation unit 82 in this example includes a selection unit 821.

The image group processing unit 80C is provided with two or more image groups collected in two or more scans applied to the subject's eye E. The image group evaluation unit 81 in this example may perform a quality evaluation on each of these image groups, or may perform a quality evaluation only on the image group collected in the initial scan (first scan). The image set creation unit 82 selects a series of images corresponding to the scan range from the two or more image groups input to the image group processing unit 80B to create an image set. In creating this image set, the selection unit 821 selects images that satisfy predetermined conditions from the two or more image groups.

The specified conditions (image selection conditions) may be the same as the image group evaluation conditions described above, or may be different. For example, if the image group evaluation unit 81 is configured to provide two or more image groups to the selection unit 821 after the quality evaluation of each image is performed by the image group evaluation unit 81, the selection unit 821 may be configured to select images taking into consideration conditions related to the image arrangement (e.g., blinking conditions, eye movement conditions, etc.). Note that the image selection conditions are not limited to these examples, and the relationship between the image group evaluation conditions and the image selection conditions is not limited to these examples.

The following describes the cases where the selection unit 821 takes blink conditions into account and the cases where it takes eye movement conditions into account. Note that when other conditions are taken into account, the selection unit 821 can execute processing similar to the specific examples (see Figures 7A to 7D) related to the image group evaluation unit 81 described below.

Image selection that takes blinking conditions into consideration will now be described. Note that blinking conditions are conditions for confirming that "the image to be evaluated is not affected by blinking." For an image included in two or more image groups, the selection unit 821 determines, for example, whether the image contains a reflected image of slit light irradiated onto the anterior segment of the subject's eye E. This determination is a process that utilizes the fact that the reflected image of the slit light is not depicted in an image captured during a blink, and that the reflected image of the slit light is depicted significantly brighter than other regions, and is performed, for example, based on the luminance distribution in the image.

As an example, the selection unit 821 creates a luminance histogram from the image and determines whether there are pixels with a luminance equal to or greater than a predetermined threshold. If it is determined that there is a luminance equal to or greater than the predetermined threshold, it is determined that the image contains a reflected image of slit light. This example has the advantage that the processing is extremely simple, but there is a risk of false detection of high-luminance noise or reflections of external light.

As another example, the selection unit 821 creates a luminance histogram from the image and determines whether there are a predetermined number of pixels with a luminance equal to or greater than a predetermined threshold. If it is determined that there are a predetermined number of pixels with a luminance equal to or greater than a predetermined threshold, it is determined that the image contains a reflected image of slit light. This example has the advantage that the above-mentioned erroneous detection can be prevented by simple processing.

Image selection that takes into account eye movement conditions is described below. The eye movement conditions are conditions for confirming that "the image to be evaluated is not affected by eye movement." For an image included in two or more image groups, the selection unit 821 determines whether or not the image is affected by eye movement, for example, by comparing the image with adjacent images. This determination is a process that utilizes the fact that "image jumps" occur when eye movement occurs during video capture.

As an example, the selection unit 821 detects landmarks from each of the image in question and the adjacent image, calculates the amount of deviation of these landmarks, and determines whether the amount of deviation is equal to or greater than a predetermined threshold. If it is determined that the amount of deviation is equal to or greater than the predetermined threshold, it is determined that eye movement has occurred. Here, the landmarks may be, for example, the cornea, iris, pupil, or angle of the eye. Furthermore, the threshold is calculated based on predetermined scanning conditions, such as the imaging rate of the imaging system 3 and the moving speed of the moving mechanism 6.

In another example, the selection unit 821 may be configured to determine the presence or absence of eye movement from one image. For example, when the shooting rate of the shooting system 3 is slow, the occurrence of rapid eye movement may cause "blur" in the image. The selection unit 821 can determine the presence or absence of eye movement by using blur detection. Blur detection is typically performed using a known technique such as edge detection.

As with the image group evaluation unit 81, the selection unit 821 may include an artificial intelligence engine for determining whether the input images satisfy the above image selection conditions.

A series of images included in the image set created by the image set creation unit 82 may be associated with multiple positions in the scan range. For example, the selection unit 821 may be configured to select images so as to assign one or more images to each of multiple positions in the scan range.

A specific example will be described. As described above, the scan range along the X direction is divided into (N-1) sections, the scan start position is set to the first position, and the scan end position is set to the Nth position (N is an integer equal to or greater than 2). In this way, N positions are set in the scan range. The N positions are represented as B1, B2, B3, ..., BN.

Assume that two or more scans are applied to the anterior segment of the subject's eye E, resulting in the acquisition of a group of images F1, F2, F3, ..., FN shown in FIG. 5. Note that in the above description, the N images F1 to FN shown in FIG. 5 are considered to be a group of images obtained from a single scan, but in this specification, for the sake of simplicity, the N images F1 to FN shown in FIG. 5 may be referred to as an arbitrary group of images (multiple images). For example, in the description of this example, the N images F1 to FN are referred to as a series of images included in an image set.

For such N positions B1 to BN and N images F1 to FN, the selection unit 821 can assign an image Fn to a position Bn. This provides N images Fn (n = 1, 2, ..., N) corresponding to the N positions Bn (n = 1, 2, ..., N), and creates an image set in which the image group Fn is a "series of images."

In some exemplary aspects, if the image group evaluation unit 81 evaluates that the quality of the image group obtained in the first scan is not good, a second scan is performed automatically or in response to a user's instruction. If the number of scans applied to the anterior segment is two or more, the slit lamp microscope system 1 performs an operation of applying two or more scans to the anterior segment and an operation of selecting a series of images from two or more image groups collected thereby, but the manner of performing these operations may be arbitrary. As a first example, the slit lamp microscope system 1 may be configured to alternately repeat a scan of the anterior segment and a selection of images from the image group acquired by this scan in response to the evaluation that the quality of the image group obtained in the first scan is not good. As a second example, the slit lamp microscope system 1 may be configured to perform two or more scans together and then select a series of images from two or more image groups collected thereby in response to the evaluation that the quality of the image group obtained in the first scan is not good. These two examples will be described below. Note that instead of or in addition to the selection of images from the image group, the image group evaluation unit 81 may evaluate the image group. As previously discussed, image group evaluation and image selection may be similar, analogous, or interchangeable processes. In some exemplary aspects, matters described as image group evaluation may be applied to image selection, and conversely, matters described as image selection may be applied to image group evaluation.

A first example of the mode of scanning and image selection executed after the image group evaluation unit 81 evaluates that the quality of the image group obtained in the first scan is not good is alternating repetition of scanning and image selection. More specifically, in the first example, for example, a set of application of a scan to the anterior eye and selection of an image from the image group obtained by this scan is repeatedly executed a predetermined number of times. That is, in the first example, U sets of operations are executed in the order of the first set of operations (scanning and image selection), the second set of operations (scanning and image selection), ..., the Uth set of operations (scanning and image selection) (U is an integer equal to or greater than 1). In addition, the set of the first scan performed before this alternating repetition and the quality evaluation (image group evaluation unit 81) of the image group obtained in the first scan will be called the 0th set of operations.

Here, the number of scans in the u-th set may be any number equal to or greater than 1 (u = 0, 1, ..., U). Also, the number of scans in the _u1- th set and the number of scans in the _u2 -th set may be equal to or different from each other ( _u1 = 0, 1, ..., U; _u2 = 0, 1, ..., U; _u1 ≠ _u2 ).

In a first example, the selection unit 821 may be configured to select images from two or more image groups collected in two or more scans that have already been performed to create a provisional image set. That is, the slit lamp microscope system 1 may be configured to create a provisional image set from two or more image groups obtained in two or more scans performed up to that point at any point during the alternating repetition of scanning and image selection. For example, after the scan in the u-th set is performed, the selection unit 821 is configured to create a provisional image set from all images obtained in the 0-th set to the u-th set. With this configuration, a provisional image set can be created from the images obtained up to the current point in time to build a final image set.

When the above configuration for creating a provisional image set is applied, the following configuration can be combined. When a new scan is applied to the anterior segment of the subject's eye E, the selection unit 821 first selects an image from a new image group collected in the new scan. Next, the selection unit 821 creates a new provisional image set by adding an image selected from the new image group to a provisional image set based on one or more scans performed before the new scan. For example, after a scan in the (u+1)th set is performed, the selection unit 821 can first select an image from the image group obtained in the (u+1)th set. Furthermore, the selection unit 821 can create a new provisional image set by adding an image selected from the image group obtained in the (u+1)th set to a provisional image set based on the image group obtained in the 0th set to the uth set. With this configuration, each time a scan is applied to the anterior segment, the provisional image set can be sequentially updated based on the image group obtained in this scan. This makes it possible to reliably and efficiently construct a final image set.

When the above configuration for creating (and updating) a provisional image set is applied, the following configuration can be combined. The control unit 7 (or the image set creation unit 82 (selection unit 821)) includes an image number counter that counts the number of images included in the provisional image set. When the number of images included in the provisional image set reaches a predetermined number, the control unit 7 controls the scanning unit (illumination system 2, shooting system 3, movement mechanism 6) and the selection unit 821 to end the alternating repetition of applying the scan and selecting the images. Here, the predetermined number is the number of a series of images included in the final image set, and is set in advance or from the processing status. In addition, the control unit 7 determines whether the number of images included in the provisional image set has reached the predetermined number. This determination may be made only by comparing the numbers. Alternatively, when multiple positions in the scan range are associated with a series of images (as described above), it may be determined whether corresponding images have been assigned to all of the multiple positions. With this configuration, when the number required for the final image set is obtained, the alternating repetition of scanning and image selection can be automatically ended.

When the above configuration for creating (and updating) a provisional image set is applied, the following configuration can be further combined. The control unit 7 includes a repetition counter that counts the number of alternating repetitions of scanning and image selection. The counted number may be defined in units of pairs of scanning and image selection (the first pair to the Uth pair), or may be defined in units of the number of scans. When the number of repetitions reaches a predetermined number, the control unit 7 controls the scanning unit (illumination system 2, shooting system 3, moving mechanism 6) and the selection unit 821 to end the alternating repetition of scanning application and image selection. When the number of repetitions is defined in units of pairs of scanning and image selection, the predetermined number is equal to the total number of pairs (U+1) set in advance. When the number of repetitions is defined in units of the number of scans, the predetermined number is equal to the total number of scans set in advance. In addition, the control unit 7 determines whether the number of repetitions has reached the predetermined number. According to this configuration, it is possible to automatically end the repetitions of scanning and image selection when they have been repeated a predetermined number of times. Without this configuration, scanning and image selection are repeated until the number of images required to construct the final image set is selected, tiring the subject and reducing imaging efficiency. In particular, when imaging multiple subjects in succession, imaging throughput is significantly impaired.

As described above, this embodiment may be configured to automatically end the alternating repetition of scanning and image selection. The condition for this automatic end is not limited to the above two examples, and may be, for example, an instruction input from the user. Alternatively, the elapsed time from the start of the alternating repetition of scanning and image selection may be measured, and the alternating repetition may be ended when a predetermined time is reached. In addition, when the repetition rate of scanning and image selection is constant, the automatic end control based on the elapsed time is equivalent to the automatic end control based on the number of repetitions described above. The image set creation unit 82 can create an image set based on a provisional image set when the alternating repetition of scanning and image selection is ended. The provisional image set is included in the image set as, for example, a series of images corresponding to the scan range. Predetermined subject information, such as a patient ID, a public ID, a name, an age, and a sex, is separately input to the slit lamp microscope system 1. The image set creation unit 82 can create an image set by configuring such subject information, subject eye information (such as information indicating the left eye/right eye), shooting date and time, shooting conditions, etc. as incidental information of a series of images. The image set may also include other images obtained by the slit lamp microscope system 1, images obtained by other modalities, inspection data acquired by an inspection device, etc. This concludes the explanation of the first example of the execution mode of scanning and image selection.

Next, a second example of the scanning and image selection mode executed after the image group evaluation unit 81 evaluates that the quality of the image group obtained in the first scan is not good will be described. This example is configured to perform two or more scans together and then select a series of images from the two or more image groups collected thereby. Images are then selected from the two or more image groups collected in these two or more scans and the image group obtained in the first scan performed before that, and an image set including the selected series of images is created.

A specific example of such processing will be described. The selection unit 821 first associates a group of images corresponding to each scan with multiple positions (described above) in the scan range. As a result, two or more images corresponding to different scans are assigned to each of the multiple positions in the scan range.

Next, for each of the multiple positions in the scan range, the selection unit 821 selects one image from the two or more images assigned to that position. The image selection conditions applied in this example may be, for example, the blink condition and eye movement condition described above. As a result, one image is assigned to each of the multiple positions in the scan range. The multiple images associated with the multiple positions in this way are adopted as a series of images included in the image set. This concludes the explanation of the second example of the execution mode of scanning and image selection.

A fourth example of the image group processing unit 80 will be described. As with the second and third examples, the effectiveness of this example is particularly evident when two or more scans are applied to the subject eye E. The image group processing unit 80D shown in FIG. 6D includes an image group evaluation unit 81 similar to that in FIG. 6A, an image set creation unit 82 similar to that in FIG. 6B or FIG. 6C, and an image set evaluation unit 83.

The image set evaluation unit 83 evaluates the quality of the image set created by the image set creation unit 82. This evaluation is to determine whether the image set has sufficient quality to perform effective diagnosis (image interpretation), and the evaluation items and evaluation criteria are determined from this perspective. The evaluation items and evaluation criteria may be common to at least a portion of the image group evaluation conditions and/or at least a portion of the image selection conditions, but are not limited to them.

The image set evaluation unit 83 may be configured to perform different evaluations depending on the aspect of the image set. For example, the evaluation applied to an image set created in response to the number of images included in the provisional image set reaching a predetermined number may be different from the evaluation applied to an image set created in response to the number of alternating repetitions of scanning and image selection reaching a predetermined number. Note that the same evaluation may be applied regardless of the aspect of the image set.

Examples of quality assessment of an image set include assessment of the quality of each image (which may be similar to the assessment of a group of images), as well as assessment of the "arrangement order" of a series of images, assessment of "image skips (missing images)", and assessment of "misalignment". Defects in an image set, such as rearrangement of the arrangement order, skipping of images, and misalignment, arise due to eye movement, fixation misalignment, etc.

The evaluation of the arrangement order of a series of images will now be described. In some examples, the aforementioned correspondence (one-to-one correspondence) is set between the series of images and multiple positions in the scan range. The image set evaluation unit 83 can use this correspondence to evaluate the arrangement order.

Here, the multiple positions in the scan range are ordered according to their positional relationships in real space. An example will be described. As described above, the scan range along the X direction is divided into (N-1) sections, and N positions B1, B2, B3, ..., BN are set in order from the scan start position toward the scan end position. In other words, the N positions B1 to BN are ordered according to their positional relationships in real space. Also, N images F1 to FN (a series of images) are associated with the N positions B1 to BN in a one-to-one relationship.

Under these assumptions, the image set evaluation unit 83 arranges the N images F1 to FN, for example, according to the arrangement order (relative positional relationship) of the N positions B1 to BN. This process is realized, for example, by setting the coordinates of the N positions B1 to BN in a certain three-dimensional coordinate system, and arranging (embedding) the N images F1 to FN according to the set N coordinates. More specifically, the image set evaluation unit 83 can, for example, extract slit light irradiation areas A1 to AN (two-dimensional cross-sectional images) from each of the N images F1 to FN, set the coordinates of the N positions B1 to BN in the three-dimensional coordinate system, and embed the N two-dimensional cross-sectional images A1 to AN according to the set N coordinates.

The image set evaluation unit 83 can evaluate whether the arrangement order is appropriate by analyzing the images F1-FN (two-dimensional cross-sectional images A1-AN) embedded in the three-dimensional coordinate system. For example, the image set evaluation unit 83 can detect areas of interest (image areas corresponding to areas of interest such as the anterior cornea, posterior cornea, iris, pupil, anterior crystalline lens, posterior crystalline lens, and angle) from the images F1-FN (two-dimensional cross-sectional images A1-AN), and perform an evaluation based on the shape (connectivity, continuity, etc.) of the areas of interest in the arrangement direction (X direction in this example) of the images F1-FN (two-dimensional cross-sectional images A1-AN). For example, if a gap of a predetermined dimension or more exists in the area of interest, it is determined that the arrangement order is inappropriate (there is a change in the arrangement order).

In another example, the image set evaluation unit 83 constructs a cross-sectional image along the X direction from images F1 to FN (2D cross-sectional images A1 to AN) embedded in a 3D coordinate system. Furthermore, the image set evaluation unit 83 can perform evaluation based on the form of this cross-sectional image (connectivity, continuity, etc.).

Evaluation of image skipping and misalignment can also be performed in the same way as evaluation of the array order.

As with the image group evaluation unit 81 and the selection unit 821, the image set evaluation unit 83 may include an artificial intelligence engine to evaluate whether the input image set has sufficient quality for effective diagnosis.

When the image set evaluation unit 83 evaluates the quality of the image set to be good, the control unit 7 may be configured to perform control to cause the communication unit 9 to transmit this image set. For example, the control unit 7 prepares transmission information including this image set, and controls the communication unit 9 to transmit this transmission information to a specified external device.

The manner in which an image set or the like is output from the slit lamp microscope system 1 is not limited to transmission. Examples of output manners other than transmission include saving in a storage device (such as a database), recording on a recording medium, and printing on a print medium.

If the image set evaluation unit 83 evaluates the quality of the image set to be poor, the control unit 7 may be configured to perform control to perform a new scan of the subject's eye E, that is, control to obtain a new image group (new image set). For example, the control unit 7 may be configured to display and/or output predetermined output information as sound. The predetermined output information has content to prompt the user to retake the image, such as, for example, that the image capture has failed or that retaking the image is necessary.

Alternatively, the control unit 7 may be configured to send a command to at least the scanning unit (illumination system 2, photographing system 3, movement mechanism 6) and the image set creation unit 82 to automatically start re-photographing (re-executing scanning and image set creation) when the image set evaluation unit 83 evaluates that the quality of the image set is not good.

Furthermore, several examples of the image group evaluation unit 81 will be described with reference to Figures 7A to 7D. Note that the image group evaluation unit 81 is not limited to these examples, and any modification (addition, substitution, omission, etc.) is possible. Also, two or more of these examples and modifications can be at least partially combined.

A first example of the image group evaluation unit 81 will be described with reference to Figures 7A and 7B. This example uses artificial intelligence technology to evaluate the quality of an image group. The image group evaluation unit 81A shown in Figure 7A includes an inference unit 811A that uses an inference model 812A to evaluate the quality of an image group.

The inference model 812A is constructed in advance by machine learning using training data including multiple anterior segment images. The device that constructs the inference model 812A (inference model construction device) may be provided in the slit lamp microscope system 1 (such as the data processing unit 8) or its peripheral device (such as a computer), or may be another computer.

The model construction unit 90 shown in FIG. 7B is an example of such an inference model construction device, and is provided in the slit lamp microscope system 1 or its peripheral devices. The model construction unit 90 includes a learning processing unit 91 and a neural network 92.

The neural network 92 typically includes a convolutional neural network (CNN). An example of the structure of a convolutional neural network is shown in FIG. 7B.

An image is input to the input layer. Behind the input layer, multiple pairs of convolutional layers and pooling layers are arranged. In the example shown in Figure 7B, three pairs of convolutional layers and pooling layers are provided, but the number of pairs is arbitrary.

In the convolution layer, a convolution operation is performed to grasp features (such as contours) from an image. The convolution operation is a multiplication and accumulation operation of a filter function (weighting factor, filter kernel) of the same dimension as the input image for the input image. In the convolution layer, the convolution operation is applied to each of multiple parts of the input image. More specifically, in the convolution layer, the value of each pixel of the partial image to which the filter function is applied is multiplied by the value (weight) of the filter function corresponding to that pixel to calculate the product, and the sum of the products is calculated over the multiple pixels of this partial image. The product-accumulation value obtained in this way is substituted for the corresponding pixel in the output image. By performing the product-accumulation operation while moving the part (partial image) to which the filter function is applied, the convolution operation result for the entire input image is obtained. According to this convolution operation, many images in which various features are extracted using many weighting factors are obtained. In other words, many filtered images such as smoothed images and edge images are obtained. The many images generated by the convolution layer are called feature maps.

In the pooling layer, the feature map generated by the immediately preceding convolutional layer is compressed (data thinning, etc.). More specifically, in the pooling layer, statistics of a specified number of neighboring pixels of a pixel of interest in the feature map are calculated at a specified pixel interval, and an image with dimensions smaller than the input feature map is output. The statistics applied to the pooling operation are, for example, the maximum value (max pooling) or the average value (average pooling). The pixel interval applied to the pooling operation is called the stride.

A convolutional neural network can extract many features from an input image by processing it using multiple pairs of convolutional layers and pooling layers.

After the last pair of convolutional and pooling layers, a fully connected layer is provided. In the example shown in FIG. 7B, two fully connected layers are provided, but the number of fully connected layers is arbitrary. In the fully connected layer, image classification, image segmentation, regression, and other processes are performed using features compressed by a combination of convolution and pooling. After the last fully connected layer, an output layer is provided to provide the output results.

In some exemplary embodiments, the convolutional neural network may not include a fully connected layer (e.g., a fully-layered convolutional network (FCN)), and may include a support vector machine, a recurrent neural network (RNN), or the like. The machine learning for the neural network 92 may be transfer learning. That is, the neural network 92 may include a neural network that has already been trained using other training data (training images) and has had its parameters adjusted. The model construction unit 90 (learning processing unit 91) may be configured to be capable of applying fine tuning to the trained neural network (92). The neural network 92 may be constructed using, for example, a known open source neural network architecture.

The learning processing unit 91 applies machine learning using training data to the neural network 92. When the neural network 92 includes a convolutional neural network, the parameters adjusted by the learning processing unit 91 include, for example, the filter coefficients of the convolutional layer and the connection weights and offsets of the fully connected layer.

As described above, the training data includes at least a plurality of anterior segment images. The plurality of anterior segment images are typically images acquired by a slit lamp microscope, but are not limited thereto, and may include, for example, images acquired by other ophthalmic modalities (fundus cameras, OCT devices, SLOs, surgical microscopes, etc.), images acquired by imaging diagnostic modalities of other medical departments (ultrasound diagnostic devices, X-ray diagnostic devices, X-ray CT devices, magnetic resonance imaging (MRI) devices, etc.), images generated by processing actual images, pseudo images, etc. Furthermore, the number of training data may be increased using techniques such as data expansion and data augmentation.

The training method for constructing an inference model may be any method, but may be, for example, supervised learning, unsupervised learning, or reinforcement learning, or a combination of two or more of these.

In some exemplary aspects, supervised learning is performed using training data in which input images are labeled with a final output label. For example, each of the multiple anterior segment images included in the training data is pre-labeled to indicate whether it is readable or unreadable. The labels are generated, for example, by a doctor or another inference model. The learning processing unit 91 can construct the inference model 812A by applying supervised learning using such training data to the neural network 92.

The inference model 812A of this example constructed in this manner is a trained model that takes as input the image obtained by scanning the anterior segment with slit light, and outputs the possibility of interpretation. Note that the possibility of interpretation, which is the output of the inference model 812A, may be any parameter indicating whether the image group to be evaluated is suitable for interpretation, and may be, for example, a determination of whether interpretation is possible or not, the probability that interpretation can be performed, the validity of the results when interpretation is performed (e.g., accuracy, precision), etc.

To prevent processing from concentrating on specific units of the neural network 92, the learning processing unit 91 may randomly select and disable some units and perform learning using the remaining units (dropout).

The methods used to build the inference model are not limited to the examples shown here. For example, any method such as a support vector machine, a Bayesian classifier, boosting, k-means, kernel density estimation, principal component analysis, independent component analysis, self-organizing map, random forest, or generative adversarial network (GAN) can be used to build the inference model.

The inference unit 811A shown in FIG. 7A uses such an inference model 812A to evaluate the quality of a group of images collected in one scan of the anterior segment of the subject's eye E. More specifically, the inference unit 811A first inputs the image group or each image contained therein to the inference model 812A. The inference model 812A derives information indicating the possibility of interpretation from the input image group or images.

The inference unit 811A may use the reading possibility information output from the inference model 812A as the inference result as is, or may generate an inference result based on the reading possibility information. As an example of the latter, the inference unit 811A may generate information for display from the reading possibility information, or may execute a predetermined statistical processing.

A second example of the image group evaluation unit 81 will be described with reference to FIG. 7C. In this example, an image group is converted into a three-dimensional image and quality evaluation is performed. The image group evaluation unit 81B shown in FIG. 7C includes a three-dimensional image construction unit 811B, a comparison unit 812B, and an evaluation processing unit 813B.

The three-dimensional image construction unit 811B constructs a three-dimensional image from a group of images acquired in a single scan of the anterior segment of the subject's eye E. For example, the three-dimensional image construction unit 811B may be configured to construct stack data by embedding the group of images in a single three-dimensional coordinate system. This stack data is constructed, for example, by embedding the N two-dimensional cross-sectional images A1 to AN shown in FIG. 5 in the three-dimensional coordinate system according to the arrangement (relative positional relationship) of the N positions B1 to BN described above.

The three-dimensional image construction unit 811B may be configured to construct volume data by applying a voxelization process to stack data constructed from the image group. Furthermore, the three-dimensional image construction unit 811B can apply a predetermined rendering to the stack data or the volume data. Examples of rendering include volume rendering and surface rendering.

The comparison unit 812B compares the 3D image (stack data, volume data, rendering image, etc.) constructed by the 3D image construction unit 811B with a predetermined reference 3D image. Any number of reference 3D images, one or more, are prepared.

The reference three-dimensional image may include one or more reference three-dimensional images corresponding to a normal eye (an eye in which no disease has been found (a healthy eye)). The reference three-dimensional image corresponding to a normal eye may be, for example, an image acquired by photographing a normal eye. The imaging modality for acquiring this reference three-dimensional image may be any, but is typically the slit lamp microscope system 1 or a similar slit lamp microscope. The reference three-dimensional image corresponding to a normal eye may also be an image acquired by photographing a model of a normal eye (such as a model eye), or an image created by computer graphics from a model of a normal eye or a clinical case.

The reference three-dimensional image may include one or more reference three-dimensional images corresponding to the diseased eye. The reference three-dimensional image corresponding to the diseased eye may be, for example, an image obtained by photographing an eye that has been definitively diagnosed with a specific disease. The imaging modality for obtaining this reference three-dimensional image may be any, but is typically the slit lamp microscope system 1 or a similar slit lamp microscope. The reference three-dimensional image corresponding to the diseased eye may also be an image obtained by photographing a model of the diseased eye (such as a model eye), or an image created by computer graphics from a model or clinical case of the diseased eye.

The comparison unit 812B performs matching between the 3D image constructed by the 3D image construction unit 811B and a reference 3D image to calculate the value of a predetermined parameter. This image matching may use any method, such as image correlation, feature-based matching, region-based matching, or machine learning (trained model). The calculated parameters may be any parameters, such as correlation values, matching parameters (angle, scale, similarity, compatibility, etc.), and output parameters of a trained model.

Such image matching is typically a process that quantitatively (i.e., numerically) expresses the degree to which the shape and structure of tissues and areas (cornea, iris, pupil, angle, etc.) depicted in a 3D image resembles the shape and structure of a standard normal eye and/or resembles the shape and structure of a standard diseased eye.

The evaluation processing unit 813B evaluates the quality of the image group based on the parameter values calculated by the comparison unit 812B. For example, the evaluation processing unit 813B may be configured to evaluate the quality of the image group by comparing the parameter values calculated by the comparison unit 812B with a predetermined threshold value. Alternatively, the evaluation processing unit 813B may be configured to evaluate the quality of the image group by determining whether the parameter values calculated by the comparison unit 812B are within a predetermined range. Note that the processing methods performed by the evaluation processing unit 813B are not limited to these, and may be any method that can be used to derive an evaluation result from the value of a certain parameter.

The quality evaluation of an image group using three-dimensional image construction is not limited to this example. For example, the image group evaluation unit 81 may be configured to be capable of performing one or more evaluation processes, in the same manner as the image set evaluation unit 83, of evaluating the arrangement order of a series of images constituting an image group, evaluating image skipping, and evaluating misalignment.

A third example of the image group evaluation unit 81 will be described with reference to FIG. 7D. In this example, the quality of an image group is evaluated by quantitatively evaluating the image quality of the image group. The image group evaluation unit 81C shown in FIG. 7D includes an evaluation data generation unit 811C and an evaluation processing unit 812C.

The evaluation data generating unit 811C obtains image quality evaluation data from images included in an image group collected in a single scan of the anterior segment of the subject's eye E. The image quality evaluation data quantitatively represents the image quality of the image group.

Several examples of image quality evaluation processing performed by the evaluation data generation unit 811C will be described. In some exemplary aspects, the image quality evaluation processing performed by the evaluation data generation unit 811C may be any processing, such as signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), root-mean-square (RMS) granularity, Wiener spectrum, modulation transfer function (MTF), quality index (QI), or other processing using any known technology.

For example, the evaluation data generation unit 811C calculates a value of a predetermined image quality evaluation index (image quality evaluation value) as image quality evaluation data for the image group. The image quality evaluation value may be any parameter that quantitatively expresses the quality of the image, and typically, the higher the image quality, the higher the image quality evaluation value.

As an example of a method for calculating an image quality evaluation value, a method for calculating an Image Quality value (IQ value) used in evaluating the image quality of an OCT image will be described below. First, the evaluation data generating unit 811C detects an image area (tissue image area) corresponding to the tissue (site) of the anterior eye and other image areas (background area, non-tissue image area) by applying a predetermined analysis process (e.g., segmentation) to the evaluation area set in the image to be evaluated. Next, the evaluation data generating unit 811C generates a histogram of brightness in the tissue image area and generates a histogram of brightness in the background area. Next, the evaluation data generating unit 811C calculates an image quality evaluation value (IQ value) from the degree of overlap of these two histograms. For example, the IQ value is defined in the range [0, 100] so that when both histograms completely overlap, the IQ value is 0, and when both histograms are completely separated, the IQ value is 100. This image quality evaluation calculation may include, for example, normalizing two histograms, generating a probability distribution function, and calculating an IQ value using a specified formula.

In this way, the evaluation data generating unit 811C may be configured to perform a process of identifying a tissue image region corresponding to the tissue of the anterior eye and a background region in the image to be evaluated, a process of creating a first histogram showing the frequency distribution of luminance in the tissue image region, a process of creating a second histogram showing the frequency distribution of luminance in the background region, and a process of calculating an image quality evaluation value (IQ value) as image quality evaluation data based on the first histogram and the second histogram.

The evaluation processing unit 812C evaluates the quality of the image group based on the image quality evaluation data generated by the evaluation data generation unit 811C. Several methods for quality evaluation based on image quality evaluation data are described below, but the quality evaluation method is not limited to these and may be any method.

A first example of quality evaluation based on image quality evaluation data will be described. When an IQ value is obtained for each image constituting an image group, the evaluation processing unit 812C compares each of the multiple IQ values obtained for this image group with a predetermined threshold. If all of the multiple IQ values are equal to or greater than the threshold, the evaluation processing unit 812C determines that the quality of this image group is good. On the other hand, if any of the multiple IQ values is less than the threshold, the evaluation processing unit 812C determines that the quality of this image group is not good.

A second example of quality evaluation based on image quality evaluation data will be described. When an IQ value is obtained for each image constituting an image group, the evaluation processing unit 812C selects the lowest IQ value from the multiple IQ values obtained for this image group, and compares this minimum IQ value with a predetermined threshold. If the minimum IQ value is equal to or greater than the threshold, the evaluation processing unit 812C determines that the quality of this image group is good. On the other hand, if the minimum IQ value is less than the threshold, the evaluation processing unit 812C determines that the quality of this image group is not good.

A third example of quality evaluation based on image quality evaluation data will be described. When an IQ value is obtained for each image constituting an image group, the evaluation processing unit 812C calculates a statistical value by applying a predetermined statistical operation to the multiple IQ values obtained for this image group. The type of statistical value may be any, and may be, for example, an average value, a minimum value, a maximum value, a mode value, a median value, etc. Note that the case where the statistical value is a minimum value corresponds to the second example above. The evaluation processing unit 812C compares the calculated statistical value with a predetermined threshold value. If the statistical value is equal to or greater than the threshold value, the evaluation processing unit 812C determines that the quality of this image group is good. On the other hand, if the statistical value is less than the threshold value, the evaluation processing unit 812C determines that the quality of this image group is not good.

The processing performed by the evaluation processing unit 812C is not limited to processing based on the image quality evaluation data generated by the evaluation data generation unit 811C. For example, the evaluation processing unit 812C may be configured to be capable of performing one or more of the following evaluation processes: evaluation of the arrangement order of a series of images constituting an image group, evaluation of image skipping, and evaluation of misalignment (all of which have been described above).

<Processed Image Construction Unit 85>
The processed image constructing unit 85 constructs a processed image from a group of images collected by scanning the anterior segment using a slit light. At least one image included in the group of images is used for constructing the processed image. The processed image construction may include any image processing. Some examples are described below, but the processing included in the processed image construction is not limited to these.

In some exemplary embodiments, the processed image construction unit 85 may be capable of performing three-dimensional image construction to construct a three-dimensional image from a group of images. The three-dimensional image constructed from the group of images may be, for example, stack data or volume data. The processed image construction unit 85 in this example may have a configuration similar to that of the above-mentioned three-dimensional image construction unit 811B. For example, the combination of hardware and software constituting the processed image construction unit 85 may be at least partially common to the combination of hardware and software constituting the three-dimensional image construction unit 811B.

In some exemplary embodiments, the processed image construction unit 85 may be capable of executing a rendering process that constructs a rendering image of a three-dimensional image. The processed image construction unit 85 may apply a predetermined rendering process to, for example, stack data or volume data obtained by constructing a three-dimensional image. Examples of the rendering process include volume rendering and surface rendering. The processed image construction unit 85 in this example may also have a similar configuration to the three-dimensional image construction unit 811B.

In some exemplary embodiments, the processed image construction unit 85 may be capable of performing processes such as identifying a partial region of an image (segmentation) and extracting the identified partial region (cropping, trimming). For example, the processed image construction unit 85 may be capable of extracting a region of interest corresponding to a specific part of the subject's eye E from an image included in an image group, a three-dimensional image, or a rendering image.

<Interest location information acquisition unit 86>
The position of interest information acquiring unit 86 acquires the position of interest information associated with the patient ID accepted by the accepting unit 11 from the related information 100 stored in the storage unit 10. Furthermore, the position of interest information acquiring unit 86 in this embodiment acquires parameter type information associated with this patient ID from the related information 100.

For example, when the related information 101 shown in FIG. 2 is used, if the patient ID accepted by the accepting unit 11 is "PID(1)", the position of interest information acquiring unit 86 acquires the position of interest information "ROI(1R)" and parameter type information "P(1R)" for the right eye, and the position of interest information "ROI(1L)" and parameter type information "P(1L)" for the left eye from the related information 101.

(Image analysis section)
As described above, the combination of the identification unit 87 and the analysis unit 88 functions as an example of an image analysis unit (not shown). The image analysis unit generates data (partial analysis data) corresponding to a partial region of the image of the anterior segment of the subject's eye E, based on the position of interest information acquired by the position of interest information acquisition unit 86 from the related information 100.

In some exemplary aspects, the image analysis unit may be configured to perform a process of identifying a partial image corresponding to the patient's region of interest from the anterior segment image of the subject's eye E, and a process of generating partial analysis data by analyzing the partial image. In such aspects, the partial analysis data is data generated by analyzing a part of the anterior segment image (partial image).

In some exemplary aspects, the image analysis unit may be configured to perform a process of analyzing an anterior segment image of the subject's eye E to generate analysis data, and a process of generating partial analysis data by identifying partial analysis data from the analysis data that corresponds to the patient's region of interest. In such aspects, the partial analysis data is a part of the analysis data of the anterior segment image.

The image analysis unit of some exemplary embodiments can generate partial analysis data from at least one image included in an image group acquired by scanning the anterior segment of the subject's eye E. The image analysis unit of some exemplary embodiments can generate partial analysis data from at least one image included in an image set generated from two or more image groups. The image analysis unit of some exemplary embodiments can generate partial analysis data from a processed image of an image included in an image group. The image analysis unit of some exemplary embodiments can generate partial analysis data from a processed image of an image included in an image set. Note that the type of image processed by the image analysis unit is not limited to these examples and may be any anterior segment image.

Below, we will explain the identification unit 87 and analysis unit 88, which are exemplary aspects of an image analysis unit having such functions.

(Specifying unit 87)
The identification unit 87 identifies a partial region of the anterior eye image of the patient corresponding to the patient ID based on the position of interest information acquired from the related information 100 by the position of interest information acquisition unit 86. The partial region may be, for example, a region in the anterior eye image that corresponds to the region of interest. The partial region does not have to be a region that coincides with the region of interest. For example, the identification unit 87 can identify the partial region so as to include at least a part of the region of the anterior eye image that corresponds to the region of interest indicated by the position of interest information acquired from the related information 100 by the position of interest information acquisition unit 86. The identification of the partial region may be performed automatically and/or manually.

Several exemplary aspects of automatic partial region identification will be described. The identification unit 87 may be configured to identify, as a partial region, an area that is larger than the area in the anterior segment image that corresponds to the region of interest. Several exemplary aspects of such a configuration (first to sixth examples) will be described below. Note that two or more of these exemplary aspects can be at least partially combined.

The identification unit 87 in the first example may be configured to perform a process of identifying a region in the anterior eye image corresponding to the region of interest by referring to the location of interest information acquired from the related information 100 by the location of interest information acquisition unit 86, a process of determining a region larger than the identified region (a region that truly includes the identified region), and a process of setting the region determined in this way as a partial region.

The identification unit 87 in the second example may be configured to perform the following processes: dividing the anterior eye image into a plurality of regions; using the location of interest information acquired from the related information 100 by the location of interest information acquisition unit 86, identifying a region that at least partially overlaps with the region of interest from among the plurality of regions obtained by the image division; determining the union of one or more regions thus identified; and setting the region consisting of this union as a partial region.

The identification unit 87 in the third example may be configured to use the location of interest information acquired from the related information 100 by the location of interest information acquisition unit 86 to perform the following processes: identifying a region in the anterior eye image corresponding to the region of interest; determining a region surrounded by a periphery obtained by expanding the periphery of the identified region outward by a predetermined dimension (distance, length, area, etc.); and setting the region thus obtained as a partial region. The parameters (direction, dimension, etc.) related to the periphery expansion in the third example may be set according to factors that affect and/or have the potential to affect the expansion of the lesion (such as the region of interest). Examples of such factors include disease type, pathological condition, disease stage, and events (medication, treatment, etc.).

The identification unit 87 in the fourth example may be configured to perform the following processes: identifying a position in the anterior eye image using the position of interest information acquired from the related information 100 by the position of interest information acquisition unit 86; gradually or continuously expanding a region of a predetermined shape based on the identified position; detecting that the region of the predetermined shape includes the region of interest and stopping the region expansion; and setting the region of the predetermined shape where the region expansion has been stopped as a partial region.

The identification unit 87 in the fifth example may be configured to perform the following processes: identifying a position in the anterior eye image using the position of interest information acquired from the related information 100 by the position of interest information acquisition unit 86; gradually or continuously enlarging a region of a predetermined shape based on the identified position; comparing the dimensions of the region of the predetermined shape with the dimensions of the region of interest; determining whether the relationship between these two dimensions satisfies a predetermined condition; and setting the region of the predetermined shape as a partial region when it is determined that the relationship between the two dimensions satisfies the predetermined condition.

The identification unit 87 in the sixth example may be configured to perform the following processes: analyzing the anterior eye image and evaluating a specific tissue (shape evaluation, thickness evaluation, thickness distribution evaluation, etc.) to generate evaluation information; identifying an area in the anterior eye image that corresponds to a lesion (lesion area) based on the evaluation information; comparing the lesion area with a region of interest; determining an area that includes an area that is included in the lesion area but not included in the region of interest and the region of interest (i.e., an area that includes the union of the lesion area and the region of interest); and setting the area determined in this way as a partial region.

As in the exemplary embodiment described above, by setting an area larger than the area of interest as a partial area, the time required to specify the area of interest each time is eliminated when following up on a disease that progresses over time (e.g., the lesion expands over time), is at risk of progressing, or has a tendency to progress. Another advantage is that changes in the extent of the lesion (expansion, contraction) can be easily grasped. Note that even when such automatic setting processing is applied, the slit lamp microscope system 1 can also be configured to allow manual adjustments, as described below, to be performed at will.

Several exemplary aspects will be described for the case where the partial region is manually specified. The specifying unit 87 of this aspect can specify the partial region according to instructions from the user. For example, the specifying unit 87 is configured to receive information (operation information) indicating the instruction content input by the user by operating the user interface, and specify the partial region based on this operation information. An exemplary aspect will be described. First, the control unit 7 causes the user interface (display device) to display the anterior segment image of the subject eye E. The user specifies a desired range in the anterior segment image displayed in the user interface using the user interface. The specifying unit 87 specifies the partial region based on the range in the anterior segment image specified by the user. At this time, the partial region may be an image area equivalent to the range specified by the user, or may be an image area wider than the specified range. When the latter is applied, the specifying unit 87 may be configured to execute, for example, a process similar to the automatic process (any of the aspects) described above.

The identification unit 87 may be configured to identify a partial region without receiving instructions from a user. For example, the identification unit 87 may be configured to switch between different partial region identification processes depending on whether a region of interest has been set in a previously acquired image (reference image) of the subject's eye E. The identification unit 87 may be configured to be able to handle either case, or to be able to handle only one of the cases. In the former case, for example, the processing mode is switched depending on the setting manner of the region of interest. In the latter case, for example, information in a form (format) that can be processed by the identification unit 87 is generated and input to the identification unit 87.

When a region of interest is set with respect to a reference image, the position information of interest may include position information indicating the position of the region in this reference image (pixel position, coordinates). In this example, the identification unit 87 may be configured as shown in FIG. 3. That is, the identification unit 87 in this example may include a deviation calculation unit 871.

The deviation calculation unit 871 calculates the deviation between the anterior eye image acquired in the current examination of the subject's eye E and the reference image. For example, the deviation calculation unit 871 may be configured to execute a process of identifying feature points in the reference image, a process of identifying feature points in the anterior eye image, and a process of calculating the deviation between the two identified feature points. In this example, the deviation between feature points is defined as the deviation between the reference image and the anterior eye image. Examples of feature points include the cornea (anterior surface, posterior surface, apex, outer edge, etc.), pupil (center, center of gravity, outer edge, etc.), iris (inner edge, pattern, etc.), angle, crystalline lens, artificial object, lesion, treatment scar, etc.

The identification unit 87 can identify a partial region based on the deviation calculated by the deviation calculation unit 871 and the position information included in the position information of interest. In some exemplary aspects, the identification unit 87 may be configured to perform registration (image matching) between the reference image and the anterior eye image so that the deviation calculated by the deviation calculation unit 871 is cancelled. This identifies a region in the anterior eye image that corresponds to the region of interest in the reference image, and the identified region in the anterior eye image is set as the partial region.

In some exemplary embodiments, the identification unit 87 may be configured to identify a region in the anterior eye image that corresponds to the region of interest in the reference image, and to identify the partial region by moving the range of coordinate values that correspond to the identified image region so as to cancel the deviation.

In some exemplary aspects, the identification unit 87 may be configured to identify a region in the anterior eye image that corresponds to the region of interest in the reference image by referring to the position information included in the position of interest information. The identification unit 87 in such an aspect can identify a partial region in the anterior eye image from the deviation calculated by the deviation calculation unit 871.

Next, some exemplary aspects in which the region of interest is not set with respect to the reference image will be described. The position information of interest may include position information indicating the position of a certain part of the anterior segment of the subject's eye E. For example, this position information may include information indicating the relative position (deviation: deviation direction, deviation amount) of that part with respect to a feature point in the anterior segment of the subject's eye E. The feature points in this example may be, for example, the cornea (anterior surface, posterior surface, apex, outer edge, etc.), pupil (center, center of gravity, outer edge, etc.), iris (inner edge, pattern, etc.), angle, crystalline lens, artificial object, lesion, treatment scar, etc.

If the region of interest is not set with respect to the reference image, the identification unit 87 may not include the deviation calculation unit 871. The identification unit 87 of this embodiment may be configured to identify a partial region by identifying a region in the anterior eye image that corresponds to the position of the part indicated in the position information included in the position information of interest. Some exemplary embodiments of such an identification unit 87 are described below.

The identification unit 87 according to the first example can identify the position (coordinate value) of the feature point by analyzing the anterior eye image. The identification unit 87 can identify a position shifted from the identified position of the feature point by the deviation in the deviation direction indicated in the position information, and set a partial region based on the identified position.

A second example will be described. As described above, the position information included in the interest position information may include information indicating the relative position (deviation: deviation direction, deviation amount) of a certain part with respect to a feature point in the anterior part of the subject's eye E. The relative position in this example includes, for example, the deviation (deviation direction and deviation amount) of a representative point (center of gravity, center, etc.) of a certain part with respect to the feature point, and information (range information) indicating the range (shape, size, direction, etc.) of the part based on the representative point. The identification unit 87 in this example may be configured to execute a process of analyzing the anterior part image to identify the position (coordinate value) of the feature point, and a process of obtaining a position (coordinate value) shifted from the coordinate value by the deviation indicated in the position information. The position (coordinate value) obtained in this way is set as the position of the representative point. Furthermore, the identification unit 87 can identify a partial area based on the position of the set representative point and the above-mentioned range information.

(Analysis unit 88)
The analysis unit 88 analyzes the partial region identified by the identification unit 87. Data generated in this way is called partial analysis data.

An example of processing executed by the analysis unit 88 will be described. The analysis unit 88 calculates parameters for the partial region identified by the identification unit 87 based on parameter type information corresponding to the patient (examined eye E) acquired by the position of interest information acquisition unit 86. An example of a parameter is a value representing the size of the partial region. Examples of the size of the partial region include the width, area, volume, thickness, and mass (estimated value) of the partial region. Examples of size values include the distribution of values within the partial region and values at representative points (center, center of gravity position, etc.) within the partial region.

The position information of interest may be information indicating the positions of two or more regions of interest. That is, two or more regions of interest may be set in the anterior eye image. In this case, the identification unit 87 identifies partial regions corresponding to each of the two or more regions of interest. As a result, two or more pieces of partial analysis data corresponding to the two or more regions of interest are obtained. The analysis unit 88 analyzes each of the two or more identified partial regions. As a result, two or more pieces of partial analysis data corresponding to the two or more regions of interest are obtained.

(Other Aspects of Identification Unit 87 and Analysis Unit 88)
The above-mentioned embodiment is configured to identify a partial region corresponding to a region of interest set for each patient from an anterior eye image, analyze the identified partial region, and acquire partial analysis data. In contrast, the present embodiment is configured to analyze an anterior eye image of a patient to generate analysis data, and acquire partial analysis data (a part of the analysis data) from the analysis data based on the region of interest of the patient. Both embodiments are the same in that partial analysis data corresponding to the region of interest of the patient is generated, but the order of processing is reversed. That is, in the above-mentioned embodiment, the analysis is performed after the part corresponding to the region of interest is identified, whereas in the present embodiment, the part is identified after the analysis. In other words, in the above-mentioned embodiment, the identification unit 87 operates before the analysis unit 88 operates, whereas in the present embodiment, the identification unit 87 operates after the analysis unit 88 operates.

The analysis unit 88 of this embodiment analyzes the anterior segment image of the subject's eye E to generate analysis data. The configuration of the analysis unit 88 of this embodiment may be similar to that of the analysis unit 88 of the above embodiment, and the process (analysis) performed by the analysis unit 88 of this embodiment may be similar to that performed by the analysis unit 88 of the above embodiment, except that the analysis is applied to the entire anterior segment image (or a specified portion regardless of the region of interest).

The identification unit 87 of this embodiment identifies partial analysis data from analysis data obtained by the analysis unit 88 of this embodiment analyzing the entire anterior eye image (or a portion specified regardless of the region of interest) based on the location of interest information acquired from the related information 100 by the location of interest information acquisition unit 86. The configuration of the identification unit 87 of this embodiment may be similar to that of the identification unit 87 of the above embodiment, and the process performed by the identification unit 87 of this embodiment (processing to identify a portion of data) may be similar to the process performed by the identification unit 87 of the above embodiment, except for the type of data to be processed (an anterior eye image in the above embodiment, and analysis data in this embodiment).

As in the above embodiment, the identification unit 87 of this embodiment may include a deviation calculation unit 871. When the position information of interest includes position information indicating the position of a region in a previously acquired reference image of the anterior eye, the deviation calculation unit 871 of this embodiment calculates the deviation between the anterior eye image of the subject's eye E and the reference image. Furthermore, the identification unit 87 of this embodiment specifies partial analysis data obtained by the analysis unit 88 of this embodiment analyzing the entire anterior eye image (or a portion specified regardless of the region of interest) based on the position information included in the position information of interest and the deviation calculated by the deviation calculation unit 871 of this embodiment. The configuration of the deviation calculation unit 871 of this embodiment may be the same as that of the deviation calculation unit 871 of the above embodiment, and the process performed by the deviation calculation unit 871 of this embodiment may be the same as the process performed by the deviation calculation unit 871 of the above embodiment, except for the type of data to be processed.

The position information of interest may include position information indicating the position of a specific site in the anterior eye. In this case, the identification unit 87 of this embodiment can identify an area of the analysis data that corresponds to the position of the specific site indicated by the position information included in the position information of interest, and set the identified area as partial analysis data.

The identification unit 87 of this embodiment can identify partial analysis data to include an area of analysis data corresponding to the area of interest set for the patient. Furthermore, the identification unit 87 of this embodiment can identify an area wider than the area of analysis data corresponding to the area of interest set for the patient as partial analysis data.

Some of the configurations and functions described above are similar to the above embodiment, and any of the configurations and functions in the above embodiment can be combined with this embodiment.

(Update unit 89)
In a mode in which the analysis unit 88 operates after the identification unit 87 operates, the update unit 89 operates when a new partial region (a partial region different from the one previously applied) is set in the current examination. When a new partial region is identified by the identification unit 87, the update unit 89 updates the position of interest information (recorded in the related information 100) associated with the patient ID of this patient based on this new partial region.

In a mode in which the identification unit 87 operates after the analysis unit 88 operates, the update unit 89 operates when a new range of partial analysis data (partial analysis data with a different range from that previously applied) is generated in the current examination. When a new range of partial analysis data is generated by the identification unit 87, the update unit 89 updates the position of interest information (recorded in the related information 100) associated with the patient ID of this patient based on this new range.

<Communication Unit 9>
The communication unit 9 performs data communication between the slit lamp microscope system 1 and other devices (external devices). That is, the communication unit 9 transmits data to other devices and receives data transmitted from other devices.

For example, the communication unit 9 can transmit at least a part of the image group collected by scanning the anterior segment of the subject's eye E with slit light to another device (first external device). The communication unit 9 can also transmit data generated from the image group by the data processing unit 8 to another device (second external device). The first external device and the second external device may be the same device or different devices. When the first external device and the second external device are the same device, the external device may be, for example, any of an image reading terminal, an image reading device, and a computer that executes a process different from that of the data processing unit 8. On the other hand, when the first external device and the second external device are different devices, the first external device may be, for example, any of an image reading terminal, an image reading device, a computer that executes a process similar to that of the data processing unit 8, and a computer that executes a process different from that of the data processing unit 8, and the second external device may be any of an image reading terminal, an image reading device, and a computer that executes a process different from that of the data processing unit 8. When the first external device and the second external device are different devices, the control unit 7 may be configured to select the external device to be the destination according to the type of data to be transmitted.

The data communication method implemented by the communication unit 9 is arbitrary. For example, the communication unit 9 includes one or more of various communication interfaces, such as a communication interface compliant with the Internet, a communication interface compliant with a dedicated line, a communication interface compliant with a LAN, and a communication interface compliant with short-range communication. The data communication may be wired communication or wireless communication.

The data transmitted by the communication unit 9 and/or the data received may be encrypted. When encryption is used, for example, the control unit 7 and/or the data processing unit 8 includes at least one of an encryption processing unit that encrypts the data transmitted by the communication unit 9 and a decryption processing unit that decrypts the data received by the communication unit 9.

<Other elements>
1A and 1B, the slit lamp microscope system 1 may include a user interface. The user interface includes, for example, a display device and an operation device. The display device operates under the control of the control unit 7. The display device may be a flat panel display such as an LCD. The operation device includes a device for operating the slit lamp microscope system 1 and a device for inputting information. The operation device includes, for example, a button, a switch, a lever, a dial, a handle, a knob, a mouse, a keyboard, a trackball, an operation panel, and the like. The user interface may include a device having both a display function and an operation function, such as a touch screen.

The user interface can be used, for example, to register, create, and edit related information 100 (location of interest information). In addition, the control unit 7 can display images and information acquired by the slit lamp microscope system 1 on a display device.

<Alignment>
The following describes alignment of the slit lamp microscope system 1 with respect to the subject's eye E. Generally, alignment is an operation for disposing the device optical system at a suitable position for photographing and measuring the subject's eye E. Alignment in this embodiment is an operation for disposing the illumination system 2 and the photographing system 3 at suitable positions for acquiring a plurality of anterior segment images (a series of images, a video image, an image group, an image set, etc.) as shown in FIG.

There are various methods for aligning ophthalmic devices. Below, some alignment methods are given as examples, but the methods that can be applied to this embodiment are not limited to these.

One alignment method that can be applied to this embodiment is stereo alignment. Stereo alignment can be applied to an ophthalmic device capable of photographing the anterior eye from two or more different directions, and a specific method thereof is disclosed in, for example, Japanese Patent Application Laid-Open Publication No. 2013-248376 filed by the present applicant. Stereo alignment includes, for example, the following steps: a step in which two or more anterior eye cameras photograph the anterior eye from different directions to obtain two or more photographed images; a step in which a processor analyzes these photographed images to determine the three-dimensional position of the subject's eye; and a step in which the processor controls the movement of the optical system based on the determined three-dimensional position. This allows the optical system (illumination system 2 and photographing system 3 in this example) to be positioned in a suitable position relative to the subject's eye. In a typical stereo alignment, the position of the pupil (center or center of gravity of the pupil) of the subject's eye is used as the reference.

In addition to this type of stereo alignment, any known alignment method can be used, such as a method using a Purkinje image obtained by alignment light, a method using an optical lever, or a method using an alignment index. In the method using a Purkinje image, or the method using an optical lever or alignment index, the position of the corneal apex of the test eye is used as the reference.

Note that while typical conventional alignment methods, including the above examples, are performed with the aim of aligning the axis of the subject's eye with the optical axis of the optical system, in this embodiment, it is possible to perform alignment so that the illumination system 2 and the imaging system 3 are positioned at a position corresponding to the scan start position.

As a first example of alignment in this embodiment, after performing alignment based on the pupil or corneal apex of the test eye E by applying any of the alignment methods described above, the illumination system 2 and the imaging system 3 can be moved (in the X direction) by a distance equivalent to a preset standard value of the corneal radius, or a distance equivalent to a value obtained by adding a predetermined value to the preset standard value of the corneal radius. Note that instead of using the standard value, a previously obtained measured value of the corneal radius of the test eye E may be used.

As a second example, after performing alignment based on the pupil or corneal apex of the subject's eye E by applying any of the alignment methods described above, an image of the anterior segment of the subject's eye E is analyzed to measure the corneal radius, and the illumination system 2 and the imaging system 3 can be moved (in the X direction) by a distance equivalent to this measurement value or a distance equivalent to the measurement value plus a predetermined value. The image of the anterior segment analyzed in this example is, for example, an anterior segment image obtained by the imaging system 3, or another image. The other image may be any image, such as an image obtained by an anterior segment camera or an anterior segment OCT.

As a third example, an image of the anterior eye obtained by an anterior eye camera or imaging system 3 for stereo alignment can be analyzed to determine the first end of the cornea, and by applying stereo alignment, the illumination system 2 and imaging system 3 can be moved to a position corresponding to this first end or to a position shifted outward a predetermined distance from this first end.

It is also possible to apply any of the above-mentioned alignment methods to perform alignment based on the pupil or corneal apex of the subject's eye E, and start scanning with the slit light from the position determined thereby. Even in this case, the scan sequence can be set to scan a three-dimensional area including the cornea C (e.g., a three-dimensional area including the entire cornea C). For example, the scan sequence is set to scan to the left from the position determined by alignment based on the pupil or corneal apex, and then scan to the right.

<Other matters>
The slit lamp microscope system 1 may include a fixation system that outputs light (fixation light) for fixating the subject's eye E. The fixation system typically includes at least one visible light source (fixation light source) or a display device that displays an image such as a landscape chart or a fixation target. The fixation system is arranged, for example, coaxially or non-coaxially with the illumination system 2 or the imaging system 3. The fixation system may include an internal fixation system that presents a fixation target to the subject through an optical path of the device optical system, and/or an external fixation system that presents a fixation target to the subject from outside the optical path.

The types of images that can be acquired by the slit lamp microscope system 1 are not limited to multiple anterior segment images as shown in FIG. 5. For example, the slit lamp microscope system 1 can acquire three-dimensional images based on multiple anterior segment images, rendering images based on these three-dimensional images, transillumination images, moving images showing the movement of a contact lens worn on the subject's eye, and images showing the gap between the contact lens and the corneal surface due to the application of a fluorescent agent. Rendering images will be described in other aspects. Transillumination images are images obtained by a transillumination method that uses the retinal reflection of illumination light to depict opacities or foreign bodies within the eye. Fundus photography, corneal endothelial cell photography, meibomian gland photography, etc. may also be possible.

The slit lamp microscope system 1 (data processing unit 8) may be capable of performing analysis processing. For example, the slit lamp microscope system 1 may be capable of calculating any anterior segment parameter. The anterior segment parameter is a value that represents the shape of the anterior segment. Examples of the anterior segment parameter include the radius of curvature of the anterior cornea, the radius of curvature of the posterior cornea, the radius of curvature of the anterior lens, the radius of curvature of the posterior lens, the corneal diameter (vertical diameter, horizontal diameter (white-to-white)), the corneal thickness (central thickness, peripheral thickness), the lens thickness, the anterior chamber depth, the anterior chamber volume, the pupil diameter, and the pupil center (eccentricity). The anterior segment parameter may be shape distribution data, and may be, for example, various corneal shape maps such as an axial curvature map, a tangential curvature map, an elevation map, a refractive power map, a thickness map (pachymetry map), and a wavefront aberration map. The anterior segment parameter may also include an angle parameter. Known examples of angle parameters include AOD (angle opening distance), ACA (anterior chamber angle), TISA (trabecular iris space area), ARA (angle recess area), and AtA (angle-to-angle distance). Methods for calculating anterior segment parameters are known. Typically, anterior segment parameter calculation includes a process for identifying the area and/or position to be measured (e.g., segmentation, feature point detection, etc.) and a measurement process for determining the value of the anterior segment parameter based on the identified area and/or position (e.g., distance measurement, ratio calculation, angle calculation, etc.).

<Other Aspects of the Optical System>
A slit lamp microscope system according to some exemplary aspects may have the configuration shown in Fig. 8. The configuration in Fig. 8 is one example of the optical system in Fig. 1A. Note that the slit lamp microscope system according to this aspect includes, in addition to the group of elements shown in Fig. 8, for example, the control unit 7, data processing unit 8, communication unit 9, storage unit 10, and the like described above. Unless otherwise specified, the matters related to the slit lamp microscope system 1 described above can be applied to this aspect.

The illumination system 20 shown in FIG. 8 is an example of the illumination system 2 of the slit lamp microscope system 1, and the left imaging system 30L and the right imaging system 30R are examples of the imaging system 3. Hereinafter, the illumination system 20 may be described as the illumination system 2, and the left imaging system 30L and/or the right imaging system 30R may be described as the imaging system 3. In some embodiments, only one of the left imaging system 30L and the right imaging system 30R may be provided and configured as an example of the imaging system 3. Symbol 20a indicates the optical axis (illumination optical axis) of the illumination system 20, symbol 30La indicates the optical axis (left imaging optical axis) of the left imaging system 30L, and symbol 30Ra indicates the optical axis (right imaging optical axis) of the right imaging system 30R. The left imaging optical axis 30La and the right imaging optical axis 30Ra are arranged in different directions. The angle between the illumination optical axis 20a and the left photographing optical axis 30La is indicated as θL, and the angle between the illumination optical axis 20a and the right photographing optical axis 30Ra is indicated as θR. The angles θL and θR may be equal to each other or may be different. The illumination optical axis 20a, the left photographing optical axis 30La, and the right photographing optical axis 30Ra intersect at one point. As in FIG. 1A, the Z coordinate of this intersection is indicated as Z0.

The moving mechanism 6 may be the same element as the moving mechanism 6 shown in FIG. 1A, and can move the illumination system 20, the left imaging system 30L, and the right imaging system 30R in the direction indicated by the arrow 49 (X direction). Typically, the illumination system 20, the left imaging system 30L, and the right imaging system 30R are placed on a stage that is movable at least in the X direction, and the moving mechanism 6 moves this movable stage in accordance with a control signal from the control unit 7.

The illumination system 20 irradiates slit light onto the anterior segment of the subject's eye E. Similar to the illumination system of a conventional slit lamp microscope, the illumination system 20 includes, in order from the side furthest from the subject's eye E, an illumination light source 21, a positive lens 22, a slit forming section 23, and objective lens groups 24 and 25.

The illumination light (typically visible light) output from the illumination light source 21 is refracted by the positive lens 22 and projected onto the slit forming section 23. A portion of the projected illumination light passes through the slit formed by the slit forming section 23 to become slit light. The generated slit light is refracted by the objective lens groups 24 and 25, and then reflected by the beam splitter 47 and irradiated onto the anterior segment of the subject's eye E.

The left imaging system 30L includes a reflector 31L, an imaging lens 32L, and an image sensor 33L. The reflector 31L and the imaging lens 32L guide light from the anterior segment irradiated with slit light by the illumination system 20 (light traveling in the direction of the left imaging system 30L) to the image sensor 33L.

The light traveling from the anterior segment toward the left imaging system 30L is light from the anterior segment irradiated with the slit light, and is light traveling in a direction away from the illumination optical axis 20a. The reflector 31L reflects the light in a direction approaching the illumination optical axis 20a. The imaging lens 32L refracts the light reflected by the reflector 31L and forms an image on the imaging surface 34L of the imaging element 33L. The imaging element 33L receives the light on the imaging surface 34L.

As with the slit lamp microscope system 1 described above, the left imaging system 30L repeatedly captures images in parallel with the movement of the illumination system 20, the left imaging system 30L, and the right imaging system 30R by the movement mechanism 6. This allows multiple anterior segment images (image group) to be obtained.

Also, similar to the slit lamp microscope system 1 described above, the object plane along the illumination optical axis 20a, the optical system including the reflector 31L and the imaging lens 32L, and the imaging plane 34L satisfy the Scheimpflug condition. More specifically, when considering the deflection of the optical path of the imaging system 30L by the reflector 31L, the YZ plane (including the object plane) passing through the illumination optical axis 20a, the principal surface of the imaging lens 32L, and the imaging plane 34L intersect on the same straight line. This allows the left imaging system 30L to focus on all positions within the object plane (for example, in the range from the anterior surface of the cornea to the posterior surface of the crystalline lens) and perform imaging.

The right photographing system 30R includes a reflector 31R, an imaging lens 32R, and an image sensor 33R. Similar to the left photographing system 30L, the right photographing system 30R guides light from the anterior eye, which is irradiated with slit light by the illumination system 20, to the imaging surface 34R of the image sensor 33R by the reflector 31R and the imaging lens 32R. Furthermore, similar to the left photographing system 30L, the right photographing system 30R obtains multiple anterior eye images (image group) by repeatedly photographing in parallel with the movement of the illumination system 20, the left photographing system 30L, and the right photographing system 30R by the movement mechanism 6. Similar to the left photographing system 30L, the object surface along the illumination optical axis 20a, the optical system including the reflector 31R and the imaging lens 32R, and the imaging surface 34R satisfy the Scheimpflug condition.

The control unit 7 can synchronize the repeated photographing by the left photographing system 30L and the repeated photographing by the right photographing system 30R. This allows a correspondence relationship to be obtained between the multiple anterior eye images obtained by the left photographing system 30L and the multiple anterior eye images obtained by the right photographing system 30R. This correspondence relationship is a temporal correspondence relationship, and more specifically, images acquired substantially simultaneously are paired together.

Alternatively, the control unit 7 or the data processing unit 8 can execute a process to determine the correspondence between multiple anterior segment images obtained by the left imaging system 30L and multiple anterior segment images obtained by the right imaging system 30R. For example, the control unit 7 or the data processing unit 8 can pair the anterior segment images sequentially input from the left imaging system 30L and the anterior segment images sequentially input from the right imaging system 30R based on the input timing of the images.

This embodiment further includes a video capture unit 40. The video capture unit 40 captures video of the anterior segment of the subject's eye E from a fixed position in parallel with the capture by the left and right imaging systems 30L and 30R. "Capturing video from a fixed position" means that the video capture unit 40 does not move when the illumination system 20, the left and right imaging systems 30L and 30R are moved to scan the anterior segment. The video capture unit 40 is also capable of capturing still images.

In this embodiment, the video capture unit 40 is arranged coaxially with the illumination system 20, but the arrangement is not limited to this. For example, the video capture unit can be arranged non-coaxially with the illumination system 20. In addition, an optical system may be provided that illuminates the anterior segment with illumination light in a band to which the video capture unit 40 is sensitive.

The light that passes through the beam splitter 47 is reflected by the reflector 48 and enters the video capture unit 40. The light that enters the video capture unit 40 is refracted by the objective lens 41, and then an image is formed on the imaging surface of the image sensor 43 by the imaging lens 42. The image sensor 43 is an area sensor. The image sensor 43 has sensitivity to, for example, either one or both of the visible light and infrared light bands.

If a video capture unit 40 is provided, it is possible to monitor and track the movement of the subject's eye E. It is also possible to perform alignment using the video capture unit 40.

Depending on the output wavelength of the illumination system 20 and the detection wavelength of the video capture unit 40, the beam splitter 47 is, for example, a dichroic mirror or a half mirror.

Some features, functions, and effects of this slit lamp microscope are described below.

As an example of the imaging system 3 of the slit lamp microscope system 1 described above, this embodiment provides a left imaging system 30L and a right imaging system 30R. The left imaging system 30L includes a reflector 31L and an imaging lens 32L (first optical system) that guide light from the anterior segment of the eye irradiated with slit light, and an imaging element 33L (first imaging element) that receives the guided light on an imaging surface 34L (first imaging surface). Similarly, the right imaging system 30R includes a reflector 31R and an imaging lens 32R (second optical system) that guide light from the anterior segment of the eye irradiated with slit light, and an imaging element 33R (second imaging element) that receives the guided light on an imaging surface 34R (second imaging surface).

The optical axis of the left imaging system 30L (left imaging optical axis 30La) and the optical axis of the right imaging system 30R (right imaging optical axis 30Ra) are arranged in different directions. Furthermore, the object surface along the optical axis of the illumination system 20 (illumination optical axis 20a), the reflector 31L, the imaging lens 32L, and the imaging surface 34L satisfy the Scheimpflug condition. Similarly, the object surface, the reflector 31L, the imaging lens 32L, and the imaging surface 34L satisfy the Scheimpflug condition.

The left imaging system 30L acquires a first group of images by repeatedly capturing images in parallel with the movement of the illumination system 20, the left imaging system 30L, and the right imaging system 30R by the movement mechanism 6. Similarly, the right imaging system 30R acquires a second group of images by repeatedly capturing images in parallel with the movement of the illumination system 20, the left imaging system 30L, and the right imaging system 30R by the movement mechanism 6.

The control unit 7 can control the illumination system 20, the left imaging system 30L, the right imaging system 30R, and the moving mechanism 6, for example, so that the left imaging system 30L and the right imaging system 30R apply one scan to the anterior segment of the subject's eye E in parallel. The data processing unit 8 can select a series of images corresponding to the scan range from the pair of left and right image groups collected in this scan to create one image group. The image group evaluation unit 81 can evaluate the quality of the created image group. If the quality of this image group is evaluated to be good, the control unit 7 can control the communication unit 9 to transmit this image group to a specified external device.

If the quality of this image group is evaluated as not good, the control unit 7 can control the illumination system 20, the left imaging system 30L, the right imaging system 30R, and the moving mechanism 6 to apply a new scan (one or more scans) to the anterior segment of the subject's eye E. By this additional scan, the left imaging system 30L collects one or more first image groups corresponding to one or more scans, and the right imaging system 30R collects one or more second image groups corresponding to one or more scans. The image set creation unit 82 can, for example, select a first series of images corresponding to the scan range from two or more first image groups obtained by the first scan and the additional scan to create a first image set, and select a second series of images corresponding to the scan range from two or more second image groups obtained by the first scan and the additional scan to create a second image set. Alternatively, the image set creation unit 82 may be configured to select a series of images corresponding to the scan range from two or more first image groups and two or more second image groups obtained by the first scan and the additional scan to create an image set. Any aspect of the slit lamp microscope system 1 described above (configuration, control, processing, action, function, etc.) can be combined with this embodiment.

According to this aspect, the anterior segment of the eye irradiated with slit light can be captured in video from different directions. Even if an image acquired by one imaging system contains an artifact, an image acquired by the other imaging system substantially simultaneously with the image may not contain the artifact. Furthermore, even if both of a pair of images acquired substantially simultaneously by both imaging systems contain artifacts and an artifact in one image overlaps with a region of interest (e.g., a region irradiated with slit light), an artifact in the other image may not overlap with the region of interest. This increases the likelihood of acquiring a suitable image. This further increases the probability of acquiring a group and/or set of images of good quality.

The imaging system 3 may include a third imaging system, ..., a Kth imaging system (K is an integer equal to or greater than 3) of a similar configuration in addition to the left imaging system 30L (first imaging system) and the right imaging system 30R (second imaging system). This complicates the structure of the optical system, but it is possible to further improve the probability of obtaining a good quality image group and/or image set. Those who wish to implement this embodiment can design a slit lamp microscope taking into account these trade-off factors (complexity of the optical system and probability of obtaining a high quality image set).

The left imaging system 30L in this embodiment includes a reflector 31L and an imaging lens 32L. The reflector 31L reflects light from the anterior segment irradiated with the slit light, traveling in a direction away from the illumination optical axis 20a, in a direction toward the illumination optical axis 20a. Furthermore, the imaging lens 32L forms an image of the light reflected by the reflector 31L on the imaging surface 34L. Here, the imaging lens 32L includes one or more lenses.

Similarly, the right imaging system 30R includes a reflector 31R and an imaging lens 32R. The reflector 31R reflects light from the anterior eye segment irradiated with the slit light, traveling in a direction away from the illumination optical axis 20a, in a direction toward the illumination optical axis 20a. Furthermore, the imaging lens 32R forms an image of the light reflected by the reflector 31R on the imaging surface 34R. Here, the imaging lens 32R includes one or more lenses.

This configuration allows the device to be made more compact. That is, the image acquired by the image sensor 33L (33R) is output through a cable extending from the surface opposite the image sensor 34L (34R), but with this configuration, the cable can be arranged from the back surface of the image sensor 33L (33R), which is located relatively close to the illumination optical axis 20a, toward the opposite direction to the subject's eye E. This allows the cable to be routed in an optimal manner, making it possible to make the device more compact.

In addition, with this configuration, since it is possible to set the angles θL and θR large, when an image acquired by one imaging system contains an artifact, it is possible to increase the likelihood that the image acquired by the other imaging system at substantially the same time as the image contains no artifact. Also, when a pair of images acquired substantially simultaneously by both imaging systems contain artifacts, and an artifact in one image overlaps with the region of interest (e.g., the slit light irradiation region), it is possible to reduce the likelihood that an artifact in the other image overlaps with the region of interest.

This embodiment includes a video capture unit 40. The left and right capture systems 30L and 30R repeatedly capture images of the anterior segment in parallel with the movement of the illumination system 20, the left and right capture systems 30L and 30R by the movement mechanism 6. In parallel with this repeated capture, the video capture unit 40 captures video of the anterior segment from a fixed position.

With this configuration, video recording is performed from a fixed position (e.g., the front) in parallel with scanning the anterior segment with slit light, making it possible to grasp the condition of the subject's eye E during scanning and to perform control according to the condition of the subject's eye E.

For example, when the slit lamp microscope of this embodiment includes the image group evaluation unit 81 (image set evaluation unit 83) of the slit lamp microscope system 1 described above, the slit lamp microscope of this embodiment can evaluate whether the image group (image set) acquired by the left imaging system 30L and/or the right imaging system 30R is of a quality suitable for interpretation. The image group evaluation unit 81 (image set evaluation unit 83) can perform the following operations in combination with the video capture unit 40. Note that similar operations can also be performed using two or more anterior eye cameras for stereo alignment or similar capture means.

The video capture unit 40 captures video of the anterior segment of the subject's eye E from a fixed position in parallel with the application of a scan to the anterior segment of the subject's eye E. This video capture is performed, for example, under the control of the control unit 7. In other words, the control unit 7 can coordinately control the illumination system 2 (illumination system 20), the capture system 3 (left capture system 30L and/or right capture system 30R), the moving mechanism 6, and the video capture unit 40 to scan the anterior segment of the subject's eye E.

In this linked control, the control unit 7 can synchronize the shooting rate of the imaging system 3 with the shooting rate of the video shooting unit 40. For example, the shooting rate of the imaging system 3 and the shooting rate of the video shooting unit 40 are set to be equal, and the shooting timing of the imaging system 3 and the shooting timing of the video shooting unit 40 are matched. This makes it possible to temporally correspond the frame group acquired by the imaging system 3 during scanning with the frame group acquired by the video shooting unit 40.

Even if one or both of the imaging rate and the imaging timing are different, it is possible to temporally correspond the frame group acquired by the imaging system 3 during scanning with the frame group acquired by the video imaging unit 40, for example, by allowing a time difference within a predetermined range.

A pair of frames that are temporally associated (a pair of a frame obtained by the imaging system 3 and a frame obtained by the video imaging unit 40) can be considered to have been acquired substantially simultaneously. Therefore, when considering a pair of frames that are temporally associated, positional deviations due to eye movement can be ignored.

Under such a premise, the image group evaluation unit 81 (image set evaluation unit 83) may be configured to evaluate the quality of a series of images included in the image group (image set) based on the video images (frame group) acquired by the video shooting unit 40.

At this time, the image group evaluation unit 81 (image set evaluation unit 83) may be configured to evaluate the quality of the image group (image set) based on the correspondence between a series of images included in the image group (image set) and a series of frames included in the video captured by the video capture unit 40. In other words, the image group evaluation unit 81 (image set evaluation unit 83) may be configured to evaluate the quality of the image group (image set) based on the temporal correspondence between the frame group obtained by the imaging system 3 and the frame group acquired by the video capture unit 40. Furthermore, the image group evaluation unit 81 (image set evaluation unit 83) may be configured to evaluate the quality of the image group (image set) based on the landmarks in the series of frames and the correspondence.

One specific example will be described. In this example, for a group of frames F1 to FN (described above) obtained by the imaging system 3 and a group of frames D1 to DN obtained in parallel by the video imaging unit 40, frame Fn is associated with frame Dn (n=1, 2, ..., N).

The image group evaluation unit 81 (image set evaluation unit 83) identifies landmarks in each of the frame groups D1 to DN. The landmarks may be, for example, image regions corresponding to the iris (iris regions).

Next, the image group evaluation unit 81 (image set evaluation unit 83) determines the arrangement order (spatial arrangement order) of the frame groups D1 to DN based on the positions (e.g., changes in spatial position) of the N landmarks identified in each of the frame groups D1 to DN.

As mentioned above, frame groups D1 to DN are arranged in this order over time (temporal arrangement order). If the temporal arrangement order differs from the spatial arrangement order, the order may have been changed or the positions may have shifted due to the influence of eye movement. Also, a skipped frame may have occurred due to the influence of a blink.

If such a defect is detected, that is, if the temporal arrangement order and the spatial arrangement order differ, the image group evaluation unit 81 (image set evaluation unit 83) determines that the quality of this image group (image set) is not good.

With this configuration, a moving image that depicts a wide area of the anterior segment (at least in the X and Y directions) can be acquired in parallel with scanning the anterior segment with slit light, and defects in a series of images included in an image group (image set) can be detected using the moving image.

Each of the series of images included in the image group (image set) is an image that extends in the depth direction (Z direction). Therefore, in order to recognize the arrangement and alignment of the series of images in the X and Y directions perpendicular to the Z direction from the series of images themselves, it is necessary to use image processing (image analysis) similar to that of the slit lamp microscope system 1 described above.

In this example, the quality of a series of images is evaluated using a video image acquired separately in parallel with the scan, without applying image analysis to the series of images. Note that the evaluation items in this example may be any of the arrangement order of the series of images, missing images, and positional deviation, as in the slit lamp microscope system 1 described above. It is also possible to combine the evaluation process of this example with any of the evaluation processes described for the slit lamp microscope system 1 described above.

Another application of the video capture unit 40 will now be described. In this example, when two or more scans are performed after the quality of the image group collected in the first scan is evaluated as not good, the purpose is to prevent the position of the subject's eye E from shifting between scans by adjusting the start timing of these two or more additional scans.

In this example, the video capture unit 40 also captures an image of the anterior segment of the subject's eye E from a fixed position. The control unit 7 is configured to cause the scanning unit to start a second scan in response to the video capture unit 40 acquiring an image that is substantially the same as the reference image acquired by the video capture unit 40 in response to the start of a first scan of two or more additional scans.

To be more specific, any of the two or more scans applied to the anterior segment to create an image set is called the first scan. In this example, the first scan of the two or more scans is the first scan.

First, the control unit 7 stores an image (reference image) acquired by the video capture unit 40 at the start timing of the first scan. The reference image may be, for example, a frame acquired closest to the start of the first scan among a group of video frames whose capture began before the start of the first scan. Alternatively, the video capture unit 40 may acquire the reference image by capturing a still image of the anterior eye segment at the start of the first scan (immediately before, simultaneously with, or immediately after) under the control of the control unit 7.

Any scan (second scan) performed after the first scan is started while the video capture unit 40 is capturing video of the anterior segment. The control unit 7 (or the data processing unit 8) compares the frames sequentially acquired by the video capture unit 40 with a reference image. This comparison may include any image processing, such as segmentation to identify landmarks, image matching, image correlation, etc.

When it is determined that a frame that is substantially the same as the reference image has been obtained, the control unit 7 controls the illumination system 2, the imaging system 3, and the movement mechanism 6 to start a second scan.

The slit lamp microscope of this example repeatedly executes the above-mentioned series of processes until the number of scans applied to the anterior segment reaches the number of scan repetitions required to create an image set. This reduces the positional shift of the subject's eye E between scans, and prevents degradation of the image set quality due to eye movement. It also makes it possible to make image selection more efficient and easier.

<Operation>
Several examples of the operation of a slit lamp microscope according to exemplary embodiments will now be described.

Although not shown in the figures, in preparation for imaging, the table on which the slit lamp microscope system 1 is placed, the chair on which the subject (patient) sits, and the chin rest of the slit lamp microscope system 1 are adjusted (all not shown in the figures). For example, the heights of the table, chair, and chin rest are adjusted. The chin rest is provided with a chin rest and a forehead rest to stably position the subject's face.

Once preparation is complete, the subject sits on a chair, rests his or her chin on the chin rest, and places his or her forehead on the forehead rest. Before or after these actions, the user performs an instruction operation to start photographing the subject's eye. This operation may be, for example, pressing an imaging start trigger button (not shown) or inputting an instruction voice. Alternatively, the control unit 7 may detect the completion of the preparation phase and automatically transition to the photographing phase. In addition, a fixation target (not shown) may be presented to the subject (subject's eye E or its fellow eye).

<First Operation Example>
A first operation example that is executed after the above preparation will be described with reference to FIG.

(S1: Accept patient ID)
First, the slit lamp microscope system 1 receives an identifier (patient ID) of a subject via the reception unit 11 .

(S2: Obtaining location of interest information)
The position of interest information acquiring unit 86 acquires the position of interest information associated with the patient ID accepted in step S1 from the associated information 100 stored in the storage unit 10.

(S3: Alignment)
The slit lamp microscope system 1 aligns the illumination system 2 and the imaging system 3 with respect to the subject's eye E. Unlike general alignment for aligning the optical axis of the optical system with the corneal apex or pupil center of the subject's eye E, the alignment in step S3 is performed to position the illumination system 2 and the imaging system 3 at the start position of the scan performed in step S4. The alignment in step S3 may be performed in any manner, and may be any of the examples described above. Before the start of, during, and/or after the alignment, operations such as adjustment of the image sensor 5 and focus adjustment may be performed.

(S4: Acquire an image of the anterior segment)
After the alignment in step S3 is completed, the slit lamp microscope system 1 photographs the anterior segment of the subject's eye E to obtain an image. For example, the slit lamp microscope system 1 scans the anterior segment of the subject's eye E by performing a combination of irradiation of slit light by the illumination system 2, video shooting by the imaging system 3, and movement of the illumination system 2 and the imaging system 3 by the moving mechanism 6. This scan is applied to a three-dimensional area of the anterior segment. By performing one scan (scanning from the scan start position to the scan end position), for example, a group of images (multiple anterior segment images) F1 to FN shown in FIG. 5 can be obtained. Note that the imaging performed in step S2 is not limited to such a scan, and may be imaging for obtaining one image, for example.

The data processing unit 8 may perform a predetermined process on the acquired anterior eye image. For example, any signal processing or image processing such as noise removal, contrast adjustment, brightness adjustment, and color correction can be applied.

(S5A: Identify a partial area of an image)
The identification unit 87 identifies a partial region of the image acquired in step S4, based on the position of interest information acquired in step S2.

(S6A: Analyze partial regions)
The analysis unit 88 analyzes the partial region identified in step S5A and generates partial analysis data.

(S7: Display partial analysis data)
The control unit 7 causes the display device to display the partial analysis data generated in step S6 A. The control unit 7 may also cause the display device to display the anterior eye image acquired in step S4, the partial region specified in step S5 A, and the like together with the partial analysis data.

(S8: Save partial analysis data)
The control unit 7 stores the partial analysis data generated in step S6A in a predetermined storage device (END). The control unit 7 may store the anterior eye image acquired in step S4, the partial area specified in step S5A, and the like in a predetermined storage device together with the partial analysis data. These pieces of information are stored, for example, in association with the patient ID accepted in step S1.

<Second Operation Example>
The second operation example will be described with reference to Fig. 10. In this operation example, steps S5A and S6B in the first operation example are replaced with steps S5B and S6B, respectively. Steps S1 to S4, S7, and S8 in this operation example are executed in the same manner as in the first operation example. Steps S5B and S6B will be described below.

(S5B: Analyze the image)
The analysis unit 88 analyzes the image acquired in step S4 to generate analysis data. In this operation example, for example, the entire image acquired in step S4 is analyzed.

(S6B: Identify the part of the analysis data)
The identifying unit 87 identifies partial analysis data from the analysis data generated in step S5B based on the position of interest information acquired in step S2.

<Third Operation Example>
A third operation example will be described with reference to Figures 11A to 11C. This operation example shows an example of reading an anterior eye image acquired by the slit lamp microscope system 1. Those skilled in the art will understand that the same processing as this operation example can be applied to medical procedures other than image reading. In this operation example, the configuration of Figure 8 is adopted.

(S11: Alignment)
First, alignment is performed in the same manner as in step S3 of the first operation example. As a result, the illumination system 2 (illumination system 20) and the imaging system 3 (either the left imaging system 30L or the right imaging system 30R, or both) are disposed at a position corresponding to the scan start position, and the video imaging unit 40 is disposed, for example, in front of the subject's eye E.

(S12: Start video recording)
After the alignment is completed, video capture of the anterior segment is started by the video capture unit 40. The capture rate of this video capture may be equal to the capture rate of the imaging system 3 in the anterior segment scan described below.

The illumination system 2 and the imaging system 3 are moved to scan the anterior segment, but the video capture unit 40 captures video of the anterior segment from a fixed position. When an anterior segment scan is performed in parallel with video capture, the aforementioned correspondence is assigned between the image group collected in the anterior segment scan and the frame group collected by the video capture unit 40.

The timing for starting video recording can be arbitrary, for example, before or during alignment.

(S13: Anterior eye scan)
The slit lamp microscope system 1 scans the anterior segment of the subject's eye E by combining the irradiation of slit light by the illumination system 2, video capture by the imaging system 3, and movement of the illumination system 2 and the imaging system 3 by the movement mechanism 6.

In this example, 256 images are acquired in one scan (scan from the scan start position to the scan end position). The 256 images (image group) acquired in the first scan (the first scan performed in step S13; the same applies below) are indicated as H1(1) to H1(256). The 256 images H1(1) to H1(256) are assigned to 256 ordered positions in the scan range (the range in the X direction with the scan start position and scan end position at both ends).

(S14: Select a good quality image)
The selection unit 821 selects images that satisfy a predetermined image selection condition from the 256 images H1(1) to H1(256) acquired in step S13.

In this example, as shown in FIG. 12A, the 71st to 170th 100 images H1(71) to H1(170) do not satisfy the image selection condition, and the 1st to 70th and 171st to 256th 156 images H1(1) to H1(70) and H1(171) to H1(256) do satisfy the image selection condition. In this case, the selection unit 821 selects the 1st to 70th and 171st to 256th 156 images H1(1) to H1(70) and H1(171) to H1(256) from among the 256 images H1(1) to H1(256).

(S15: Create a provisional image set)
The selector 821 creates a provisional image set based on the image(s) acquired up to the current point in time. At this point in the example, a provisional image set is created that includes 156 images H1(1)-H1(70) and H1(171)-H1(256).

(S16: Number of selected images = predetermined number?)
The selection unit 821 compares the number of images included in the temporary image set created in the immediately preceding step S15 (the number of selected images) with a predetermined number. As described above, in this example, the predetermined number (the number of images included in the image set) is set to 256.

If the number of images included in the provisional image set reaches 256 (S16: Yes), the operation proceeds to step S18. If the number of images included in the provisional image set does not reach 256 (S16: No), the operation proceeds to step S17.

(S17: Number of scans = predetermined number?)
The selection unit 821 compares the number of scans performed up to the current stage (the number of times that scanning and image selection have been alternately repeated) with a predetermined number of times. As described above, in this example, the predetermined number of times is set to five times.

If the number of scans performed up to the current stage reaches five (S17: Yes), the operation proceeds to step S18. If the number of scans performed up to the current stage does not reach five (S17: No), the operation returns to step S13.

Steps S13 to S37 are repeated until step S16 or S17 returns "Yes." A "Yes" determination in step S16 corresponds to the number of images (256) required to create an image set being prepared. A "Yes" determination in step S17 corresponds to the scanning and image selection being repeated the maximum number of times.

(S18: Create image set)
If the determination in step S16 or S17 is "Yes", the image set creation unit 82 creates an image set based on the provisional image set created in the immediately preceding step S15.

Here, a specific example of steps S13 to S18 will be described. As described above, in the example of FIG. 12A, a provisional image set including 156 images H1(1) to H1(70) and H1(171) to H1(256) was created (step S15) based on the first scan (step S13) and the first image selection (step S14).

At this stage, the number of images included in the provisional image set has not reached 256 (S16: No), so the operation proceeds to step S17. At this stage, only one scan has been performed (S17: No), so the operation returns to step S13.

When the operation returns to step S13, a second anterior eye scan is performed. The 256 images obtained in the second scan are denoted by H2(1) to H2(256). As with the set of images obtained in the first scan, the 256 images H2(1) to H2(256) are assigned to 256 ordered positions in the scan range. Furthermore, the image H1(n) obtained in the first scan and the image H2(n) obtained in the second scan are associated with each other via the 256 positions (n = 1, 2, ..., 256). The same applies to the sets of images obtained in subsequent scans.

In step S14, the selection unit 821 selects images that satisfy a predetermined image selection condition from among the 256 images H2(1) to H2(256) acquired in step S13. In this example, as shown in FIG. 12B, it is assumed that the 100 images H2(1) to H2(100) from the 1st to 100th do not satisfy the image selection condition, and the 156 images H2(101) to H2(256) from the 101st to 256th do satisfy the image selection condition. In this case, the selection unit 821 selects the 156 images H2(101) to H2(256) from the 256 images H2(1) to H2(256).

Here, if two or more images corresponding to a certain position in the scan range are selected, the selection unit 821 can select one of those images. In the example shown in Figures 12A and 12B, for example, image H1 (256) and image H2 (256) are selected as images corresponding to the 256th position.

The selection unit 821 may be configured to execute a process of selecting one of the image H1 (256) and the image H2 (256). For example, the selection unit 821 may be configured to select the image H1 (256) acquired first, or to select the image H2 (256) acquired later. In this example, it is assumed that the selection unit 821 is configured to select the image acquired first.

As another example, the selection unit 821 may be configured to select one of the images H1 (256) and H2 (256) by comparing them. For example, the selection unit 821 may be configured to calculate an image quality evaluation value for the image H1 (256), calculate an image quality evaluation value for the image H2 (256), compare these image quality evaluation values, and select one of the images H1 (256) and H2 (256). Typically, the image with the highest image quality evaluation value is selected.

This process assigns up to one image to each of the 256 positions in the scan range. This means that the provisional image set contains a maximum of 256 images.

The selection unit 821 creates a provisional image set based on the two image groups obtained up to this point (image group H1(1) to H1(256) and image group H2(1) to H2(256)). At this stage in this example, a provisional image set is created that includes 156 images H1(1) to H1(70) and H1(171) to H1(256) based on the first scan, and 70 images H2(101) to H2(170) based on the second scan.

In other words, the provisional image set obtained at this stage includes 226 images H1(1) to H1(70), H2(101) to H2(170), and H1(171) to H1(256), which correspond to the 1st to 70th and 101st to 256th positions (226 positions) of the 256 positions in the scan range.

At this stage, the number of images included in the provisional image set has not reached 256 (S16: No), so the operation proceeds to step S17. At this stage, only two scans have been performed (S17: No), so the operation returns to step S13 again.

When operation returns to step S13, a third anterior segment scan is performed. The 256 images obtained in the third scan are shown as H3(1) to H3(256).

In step S14, the selection unit 821 selects images that satisfy a predetermined image selection condition from among the 256 images H3(1) to H3(256) acquired in step S13. In this example, as shown in FIG. 12C, it is assumed that the 227th to 256th 30 images H3(227) to H3(256) do not satisfy the image selection condition, and the 1st to 226th 226 images H3(1) to H3(226) do satisfy the image selection condition. In this case, the selection unit 821 selects the 1st to 226th 226 images H3(1) to H3(226) from among the 256 images H3(1) to H3(256).

Furthermore, the selection unit 821 selects 30 images H3(71) to H3(100) from among the 226 images H3(1) to H3(226), corresponding to the 71st to 100th positions, excluding the 1st to 70th and 101st to 256th positions of the 256 positions in the scan range to which images have already been assigned.

The selection unit 821 creates a provisional image set based on the three image groups obtained up to this point (image group H1(1) to H1(256), image group H2(1) to H2(256), and image group H3(1) to H3(256)). At this stage in this example, a provisional image set is created that includes 156 images H1(1) to H1(70) and H1(171) to H1(256) based on the first scan, 70 images H2(101) to H2(170) based on the second scan, and 30 images H3(71) to H3(100) based on the third scan.

In other words, the provisional image set obtained at this stage includes 256 images H1(1) to H1(70), H3(71) to H3(100), H2(101) to H2(170), and H1(171) to H1(256) corresponding to all 256 positions in the scan range. Therefore, the answer in step S16 is "Yes," and the operation proceeds to step S18.

In step S18, the image set creation unit 82 creates an image set based on the 256 images H1(1) to H1(70), H3(71) to H3(100), H2(101) to H2(170), and H1(171) to H1(256) included in the provisional image set created in the immediately preceding step 35.

The operations shown in FIG. 11A provide an example process for creating an image set using alternating iterations of scanning and image selection, followed by the operations shown in FIG. 11B.

(S19: Evaluate the quality of the image set)
The image set evaluation unit 83 evaluates the quality of the image set created in step S18.

Alternatively, in this embodiment, the quality of the image set may be evaluated using the video captured by the video capture unit 40 in the manner described above. For example, the image set based on the example shown in Figures 12A to 12C includes a series of 256 images H1(1) to H1(70), H3(71) to H3(100), H2(101) to H2(170), and H1(171) to H1(256). Three scans were performed to collect these images, and video capture was performed by the video capture unit 40 in parallel with each scan.

Of the frames obtained by video capture in parallel with the first scan, the frame associated with image H1(n) is denoted by J1(n) (n = 1, 2, ..., 256). Of the frames obtained by video capture in parallel with the second scan, the frame associated with image H2(n) is denoted by J2(n) (n = 1, 2, ..., 256). Of the frames obtained by video capture in parallel with the third scan, the frame associated with image H3(n) is denoted by J3(n) (n = 1, 2, ..., 256).

Thus, the 256 images included in the image set in this example, H1(1) to H1(70), H3(71) to H3(100), H2(101) to H2(170), and H1(171) to H1(256), correspond to frames J1(1) to J1(70), J3(71) to J3(100), J2(101) to J2(170), and J1(171) to J1(256), respectively.

The image set evaluation unit 83 can evaluate this image set by applying the same processing as that performed on the frame groups D1 to DN described above to the 256 frames corresponding to the 256 images included in the image set in this example.

(S20: Is the quality good?)
If it is determined in step S19 that the quality of the image set is good (S20: Yes), the operation proceeds to step S23. On the other hand, if it is determined in step S19 that the quality of the image set is not good (S20: No), the operation proceeds to step S21.

(S21: Prompt to retake the photo)
If it is determined in step S19 that the quality of the image set is not good (S20: No), the control unit 7 displays and/or outputs sound to prompt the user to take a new photograph.

(S22: Should we take the photo again?)
In response to the information displayed and/or output as sound in step S21, the user performs an instruction operation to start re-shooting or an instruction operation to not perform re-shooting.

If the user performs an instruction operation to start re-imaging (S22: Yes), the operation returns to step S11. In this case, the slit lamp microscope system 1 executes the series of processes up to this point again. Note that re-imaging is repeated, for example, up to a predetermined number of times.

On the other hand, if the user performs an instruction not to retake the image (S22: No), the operation proceeds to step S23.

(S23: Transmit image set)
If the quality of the image set is determined to be good in step S19 (S20: Yes), or if the user performs an instruction operation in step S22 not to re-take the image (S22: No), the control unit 7 controls the communication unit 9 to send the image set to the reading terminal and/or reading device.

In this operation example, the steps up to this point are executed by the slit lamp microscope (and a computer connected thereto) included in the slit lamp microscope system 1, and the steps thereafter (the steps shown in FIG. 11C) are executed by an information processing device capable of communicating with this slit lamp microscope (or computer). The steps shown in FIG. 11C are performed using a computer (image reading terminal) used by a doctor to perform image reading. Those skilled in the art will understand that similar processing can be executed even when a computer (image reading device) with image reading capabilities is used. An example of a system configuration in which this operation example can be adopted will be described later.

(S24: Patient ID is accepted)
The slit lamp microscope system 1 receives an identifier (patient ID) of a subject via the reception unit 11 .

(S25: Obtaining location of interest information)
The position of interest information acquiring unit 86 acquires the position of interest information associated with the patient ID accepted in step S24 from the associated information 100 stored in the storage unit 10.

(S26: Start of interpretation)
A doctor using the image reading terminal starts reading the series of images included in the image set transmitted from the slit lamp microscope (or computer) in step S23 at a desired timing. The images to which the following steps are applied may be, for example, all images included in the image set, one or more images selected from the image set by the doctor, or one or more images selected from the image set by the image reading terminal.

(S27: Identify a partial area of the image)
The identifying unit 87 identifies a partial region of an image included in the image set transmitted from the slit lamp microscope (or computer) in step S23, based on the position of interest information acquired in step S25.

(S28: Analyze partial regions)
The analysis unit 88 analyzes the partial region identified in step S27 and generates partial analysis data.

Note that steps S27 and S28 in this operation example correspond to steps S5A and S6A in the first operation example, respectively, but instead, steps corresponding to steps S5B and S6B in the second operation example may be executed.

(S29: Display partial analysis data)
The control unit 7 causes the display device to display the partial analysis data generated in step S28. The control unit 7 may cause the display device to display the anterior eye image, a partial region thereof, and the like together with the partial analysis data.

(S30: Create image interpretation report)
The doctor can perform image interpretation while referring to the partial analysis data and images displayed in step S29. The doctor can apply the processes of steps S27 to S29 to a desired image among a series of images included in the image set. In this way, the doctor interprets the anterior segment image of the subject's eye E and inputs the result into a predetermined report template. In this way, an image interpretation report on the subject's eye E is created.

(S31: Save image set, image interpretation report, and partial analysis data)
The image set, image interpretation report, and partial analysis data are provided to and stored in, for example, a facility where the slit lamp microscope system 1 is installed, a medical institution designated by the subject, etc., an information processing device used by a doctor designated by the subject, etc., an address registered by the subject (email address, mailing address, etc.), etc. In addition, the image set and image interpretation report may be sent to a specified database system for storage and management (end).

Some effects of a slit lamp microscope system capable of executing this operation example will be described. A slit lamp microscope system capable of executing this operation example can prevent deviation of the subject's eye between scans by matching the anterior segment image at the start of scanning during alternating repetition of scanning and image selection. To this end, a slit lamp microscope system capable of executing this operation example includes a video capture unit 40 (image capture unit) that captures the anterior segment of the subject's eye from a fixed position. The control unit 7 is configured to cause the scanning unit to start a second scan in response to the video capture unit 40 acquiring an image that is substantially the same as the reference image acquired by the video capture unit 40 in response to the start of the first scan.

A slit lamp microscope system capable of performing this operation example can use video images (frame group) acquired by the video capture unit 40 in evaluating the quality of the image set. To that end, a slit lamp microscope system capable of performing this operation example includes a video capture unit 40 that captures video of the anterior segment of the subject's eye from a fixed position in parallel with the application of a scan to the anterior segment of the subject's eye. The image set evaluation unit 83 is configured to evaluate the image set quality based on the video images acquired by the video capture unit 40.

At this time, the image set evaluation unit 83 may be configured to evaluate the quality of the image set based on the correspondence between the series of images included in the image set and the series of frames included in the video captured by the video capture unit 40.

Furthermore, the image set evaluation unit 83 may be configured to evaluate the quality of the image set based on landmarks in a series of frames included in a video captured by the video shooting unit 40 and the correspondence between a series of images included in the image set and a series of frames included in the video.

An exemplary embodiment of a slit lamp microscope system capable of executing the third operation example is shown in FIG. 13. The slit lamp microscope system 200 of this embodiment includes a slit lamp microscope 210 and an information processing device 220. The information processing device 220 is, for example, the image reading terminal described above, and elements (hardware, software) for providing the image reading application are omitted from the illustration.

The slit lamp microscope 210 includes an image acquisition unit 211, a control unit 212, a data processing unit 213, and a communication unit 214. The control unit 212 may have the same configuration and function as the control unit 7, and the data processing unit 213 may have the same configuration and function as the data processing unit 8. The data processing unit 213 may include, for example, the image group processing unit 80 and the processed image construction unit 85, which are elements of the data processing unit 8 shown in FIG. 1B.

The image acquisition unit 211 has the configuration and function to acquire an image by scanning the anterior segment of the subject's eye with slit light, and includes, for example, the illumination system 2, the imaging system 3, and the movement mechanism 6 of the slit lamp microscope system 1.

The image acquired by the slit lamp microscope 210 may be, for example, a single image, a group of images, a set of images, a three-dimensional image, a rendered image, etc.

The communication unit 214 performs data communication between the slit lamp microscope 210 and other devices (external devices). The information processing device 220 is one of these external devices.

The control unit 212 of the slit lamp microscope 210 performs various controls executed in steps S11 to S23 of the third operation example.

The data processing unit 213 of the slit lamp microscope 210 executes various data processing operations performed in steps S11 to S23 of the third operation example. For example, the data processing unit 213 of the slit lamp microscope 210 may include an image group processing unit 80, an image group evaluation unit 81, an image set creation unit 82, a selection unit 821, an image set evaluation unit 83, etc. The image group evaluation unit 81 of the slit lamp microscope 210 may include an inference unit 811A, an inference model 812A, a three-dimensional image construction unit 811B, a comparison unit 812B, an evaluation processing unit 813B, an evaluation data generation unit 811C, an evaluation processing unit 812C, etc.

The information processing device 220 includes a memory unit 221, a reception unit 222, a location of interest information acquisition unit 223, an image analysis unit 224, a user interface 225, and a communication unit 226.

The memory unit 221 stores related information 221a. The memory unit 221 may be similar to the memory unit 10, and the related information 221a may be similar to the related information 100.

The reception unit 222 may be similar to the reception unit 11 and performs step S24 of the third operation example.

The location of interest information acquisition unit 223 may be similar to the location of interest information acquisition unit 86, and performs step S25 of the third operation example.

The image analysis unit 224 generates partial analysis data corresponding to a partial region of the image acquired by the slit lamp microscope 210 based on the position of interest information acquired by the position of interest information acquisition unit 223. The image analysis unit 224 includes, for example, an identification unit 87 and an analysis unit 78. The image analysis unit 224 performs steps S27 and S28 of the third operation example.

The user interface 225 includes, for example, a display device and an operation device. Step S29 of the third operation example is performed using the display device of the user interface 225.

The communication unit 226 performs data communication between the information processing device 220 and other devices (external devices). The slit lamp microscope 210 is one of these external devices.

Step S23 of the third operation example is performed by transmitting an image set from the communication unit 214 of the slit lamp microscope 210 to the information processing device 220. The information processing device 220 receives the image set transmitted from the slit lamp microscope 210 via the communication unit 226. The received image set is input to the image analysis unit 224.

Some examples of processing that can be performed by the slit lamp microscope 210 are described below. First, the slit lamp microscope 210 can process the image group collected by the image acquisition unit 211 using the image group processing unit 80.

The image group processing unit 80 of the slit lamp microscope 210 can evaluate the quality of the image group by the image group evaluation unit 81. If the image group evaluation unit 81 evaluates the quality of the image group to be good, the communication unit 214 can transmit this image group to the information processing device 220. If the image group evaluation unit 81 evaluates the quality of the image group to be bad, the control unit 7 of the slit lamp microscope 210 can execute control to cause the image acquisition unit (scanning unit) to execute a new scan of the anterior segment of the examinee's eye.

The image group processing unit 80 of the slit lamp microscope 210 can use the image set creation unit 82 to select a series of images corresponding to the scan range from two or more image groups, including an image group collected in a previous scan and a new image group collected in a new scan, to create an image set.

The image acquisition unit 211 (scanning unit) of the slit lamp microscope 210 can collect two or more image groups by applying two or more scans (e.g., consecutively) to the anterior segment of the subject's eye. The image group processing unit 80 of the slit lamp microscope 210 can create an image set by using the image set creation unit 82 to select a series of images corresponding to the scan range from the two or more image groups collected by the two or more scans.

The image group processing unit 80 of the slit lamp microscope 210 can evaluate the quality of the image set by the image set evaluation unit 83. If the image set evaluation unit 83 evaluates the quality of the image set to be good, the communication unit 214 can transmit this image set to the information processing device 220. If the image set evaluation unit 83 evaluates the quality of the image set to be bad, the control unit 7 of the slit lamp microscope 210 can execute control to cause the image acquisition unit (scanning unit) to execute a new scan of the anterior segment of the subject's eye.

＜Effects＞
Several advantages provided by the slit lamp microscope system according to the exemplary embodiment will now be described.

The exemplary slit lamp microscope system 1 includes a reception unit 11, a position of interest information acquisition unit 86, an image acquisition unit (illumination system 2, imaging system 3, moving mechanism 6, etc.), and an image analysis unit (data processing unit 8). Note that although the slit lamp microscope system 1 includes a memory unit 10, the storage means may be an external device accessible from the slit lamp microscope system.

The reception unit 11 receives patient identification information for identifying a patient. The position of interest information acquisition unit 86 acquires position of interest information associated with the patient identification information received by the reception unit 11 from a storage means (memory unit 10) in which position of interest information indicating the position of a region of interest is pre-associated with each of a plurality of patient identification information and stored. The image acquisition unit acquires an image by scanning the anterior segment of the subject eye of the patient corresponding to the patient identification information received by the reception unit 11 with slit light. The image analysis unit generates partial analysis data corresponding to a partial region of the image acquired by the image acquisition unit based on the position of interest information acquired by the position of interest information acquisition unit.

With the slit lamp microscope system 1 configured in this way, it is possible to generate analysis data for the region of interest set for the target patient from an image acquired by scanning the anterior segment with slit light. This makes it possible to continuously observe the region of interest for each patient, and to capture small local changes in the examinee's eye regardless of the type of disease, stage, or condition.

The image analysis unit of the slit lamp microscope system 1 includes, for example, an identification unit 87 (first identification unit) and an analysis unit 88 (first analysis unit). The identification unit 87 (first identification unit) may be configured to identify a partial region of the image acquired by the image acquisition unit based on the position of interest information acquired by the position of interest information acquisition unit 86. The analysis unit 88 (first analysis unit) may be configured to analyze the partial region identified by the identification unit 87 and generate partial analysis data.

The position information of interest may include, for example, first position information indicating the position of an area in a reference image of the anterior segment of the subject's eye previously acquired. In this case, the identification unit 87 (first identification unit) may include a deviation calculation unit 871 that calculates the deviation between the image acquired by the image acquisition unit and the reference image. Furthermore, the identification unit 87 (first identification unit) may be configured to identify the partial area based on the first position information and the deviation calculated by the deviation calculation unit 871. With this configuration, the partial area can be accurately identified by referring to the reference image, and it becomes possible to accurately capture changes in the analysis data corresponding to the area of interest of the patient.

The position information of interest may include, for example, second position information indicating the position of a specific site of the anterior eye. In this case, the identification unit 87 (first identification unit) may be configured to identify, as the above-mentioned partial region, an area corresponding to the position indicated by the second position information in the image acquired by the image acquisition unit. With such a configuration, even if the region of interest is not set with respect to the reference image, it is possible to accurately identify the partial region, and it is possible to accurately capture changes in the analysis data corresponding to the patient's region of interest.

The identification unit 87 (first identification unit) may be configured to identify the partial region described above so as to include an area corresponding to the patient's region of interest in the image acquired by the image acquisition unit. Furthermore, the identification unit 87 (first identification unit) may be configured to identify an area larger than the area corresponding to the patient's region of interest in the image acquired by the image acquisition unit as the partial region described above. With this configuration, it becomes possible to provide an analysis result of the entire region of interest of the patient.

The slit lamp microscope system 1 includes an update unit 89 (first update unit). The update unit 89 (first update unit) operates when a new partial region is identified by the identification unit 87 (first identification unit). The update unit 89 (first update unit) is configured to update the position of interest information associated with the patient identification information of the patient based on this new partial region. This makes it easier to follow up even if the position or size of the partial region changes, and makes it possible to capture small local changes.

The image analysis unit of the slit lamp microscope system 1 includes, for example, an analysis unit 88 (second analysis unit) and an identification unit 87 (second identification unit). The analysis unit 88 (second analysis unit) may be configured to analyze the image acquired by the image acquisition unit to generate analysis data. The identification unit 87 (second identification unit) may be configured to identify partial analysis data from the analysis data generated by the analysis unit 88 (second analysis unit) based on the position of interest information acquired by the position of interest information acquisition unit 86.

The position information of interest may include, for example, first position information indicating the position of an area in a reference image of the anterior segment of the subject's eye previously acquired. In this case, the identification unit 87 (second identification unit) may include a deviation calculation unit 871 that calculates the deviation between the image acquired by the image acquisition unit and the reference image. Furthermore, the identification unit 87 (second identification unit) may be configured to identify partial analysis data from the analysis data generated by the analysis unit 88 (second analysis unit) based on the first position information and the deviation calculated by the deviation calculation unit 871. With this configuration, the partial analysis data can be accurately identified by referring to the reference image, and it becomes possible to accurately capture changes in the analysis data corresponding to the area of interest of the patient.

The position information of interest may include, for example, second position information indicating the position of a specific site of the anterior eye. In this case, the identification unit 87 (second identification unit) may be configured to identify the region of the analysis data corresponding to the position indicated by the second position information as the above-mentioned partial analysis data. With such a configuration, even if the region of interest is not set with respect to the reference image, it is possible to accurately identify the partial analysis data, and it is possible to accurately capture changes in the analysis data corresponding to the patient's region of interest.

The identification unit 87 (second identification unit) may be configured to identify the partial analysis data so as to include a data area corresponding to the patient's region of interest in the analysis data generated by the analysis unit 88 (second analysis unit). Furthermore, the identification unit 87 (second identification unit) may be configured to identify a data area wider than the data area corresponding to the patient's region of interest in the analysis data generated by the analysis unit 88 (second analysis unit) as the partial analysis data. With this configuration, it becomes possible to provide an analysis result of the entire region of interest of the patient.

The slit lamp microscope system 1 includes an update unit 89 (second update unit). The update unit 89 (second update unit) operates when a new range of partial analysis data is identified by the identification unit 87 (second identification unit). The update unit 89 (second update unit) is configured to update the position of interest information associated with the patient identification information of the patient based on this new range. This makes it easier to follow up even if the position or size represented by the partial analysis data changes, and makes it possible to capture small local changes.

Slit lamp microscope system 200, like slit lamp microscope system 1, is capable of capturing small local changes in the eye regardless of disease type, disease stage, or pathological condition. In addition, slit lamp microscope system 200 allows the slit lamp 210 and information processing device 220 to be remotely located from each other, making it possible to put into practical use remote medical care using a slit lamp microscope, which was previously difficult.

The present disclosure provides a method for controlling a device or system according to any of the aspects. The present disclosure also provides a program for causing a computer to execute the control method, and a computer-readable non-transitory recording medium on which the program is recorded. The non-transitory recording medium may be in any form, and examples of such a recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

The present disclosure provides a method relating to any of the aspects (such as an imaging method, a data processing method, or a data communication method). The present disclosure also provides a program for causing a computer to execute the method, and a computer-readable non-transitory recording medium having the program recorded thereon.

The above-described aspects are exemplary embodiments and are not intended to limit the present invention. Any modifications (omissions, substitutions, additions, etc.) within the scope of the gist of the present invention may be made to the above-described aspects as appropriate.

REFERENCE SIGNS LIST 1 Slit lamp microscope 2 Illumination system 3 Photography system 4 Optical system 5 Image sensor 6 Movement mechanism 7 Control unit 8 Data processing unit 80 Image group processing unit 81 Image group evaluation unit 82 Image set creation unit 83 Image set evaluation unit 85 Processed image construction unit 86 Interest position information acquisition unit 87 Identification unit 88 Analysis unit 89 Update unit 9 Communication unit 10 Memory unit 11 Reception unit 100 Related information 200 Slit lamp microscope system 210 Slit lamp microscope 220 Information processing device

Claims (25)

a reception unit that receives patient identification information for identifying a patient;
a position of interest information acquiring unit that acquires position of interest information associated with the patient identification information accepted by the accepting unit from a storage unit in which position of interest information indicating a position of a region of interest is previously associated with each of a plurality of patient identification information and stored;
an image acquisition unit that acquires an image by scanning an anterior segment of an eye to be examined of a patient corresponding to the patient identification information with a slit light;
an image analysis unit that generates partial analysis data corresponding to a partial region of the image acquired by the image acquisition unit based on the location of interest information acquired by the location of interest information acquisition unit,
The image analysis unit
A second analysis unit that analyzes the image acquired by the image acquisition unit and generates analysis data;
a second identification unit that identifies partial analysis data from the analysis data based on the location of interest information acquired by the location of interest information acquisition unit;
Including,
Slit lamp microscope system. the position information of interest includes first position information indicating a position of a region in a reference image of the anterior eye segment previously acquired,
The second identification unit includes a deviation calculation unit that calculates a deviation between the reference image and the image, and identifies the partial analysis data from the analysis data based on the first position information and the deviation.
2. The slit lamp microscope system of claim 1 . the position of interest information includes second position information indicating a position of a predetermined site of an anterior segment of the eye,
The second identification unit identifies an area of the analysis data corresponding to the position indicated by the second position information as the partial analysis data.
2. The slit lamp microscope system of claim 1 . the second identification unit identifies the partial analysis data so as to include a region of the analysis data corresponding to the region of interest;
The slit lamp microscope system according to any one of claims 1 to 3 . The second identification unit identifies, as the partial analysis data, a region that is larger than the region of the analysis data corresponding to the region of interest.
5. The slit lamp microscope system of claim 4 . and a second update unit configured to update the position of interest information associated with the patient identification information of the patient based on a new range of partial analysis data identified by the second identification unit.
The slit lamp microscope system according to any one of claims 1 to 5 . the image acquisition unit includes a scanning unit that scans the anterior segment with the slit light to collect a group of images;
The scanning unit includes:
an illumination system that irradiates the anterior segment with the slit light;
an imaging system that images the anterior segment from a direction different from that of the illumination system;
a moving mechanism for moving the illumination system and the imaging system,
the imaging system repeatedly performs imaging in parallel with the movement of the illumination system and the imaging system by the movement mechanism;
The slit lamp microscope system according to any one of claims 1 to 6 . The imaging system includes:
an optical system for guiding light from the anterior segment irradiated with the slit light;
an imaging element that receives the light guided by the optical system on an imaging surface,
an object surface along the optical axis of the illumination system, the optical system, and the imaging surface satisfy the Scheimpflug condition;
The slit lamp microscope system of claim 7 . the moving mechanism moves the illumination system and the imaging system so as to move the slit light irradiated by the illumination system to the anterior eye segment in a width direction of the slit light.
9. A slit lamp microscope system according to claim 7 or 8 . The image analysis unit generates partial analysis data from at least one image included in the image group.
A slit lamp microscope system according to any one of claims 7 to 9 . The image acquisition unit further includes a processed image construction unit that constructs a processed image from the image group.
A slit lamp microscope system according to any one of claims 7 to 10 . The processed image construction unit constructs a three-dimensional image as the processed image.
The slit lamp microscope system of claim 11 . The processed image constructing unit constructs a rendering image of the three-dimensional image as the processed image.
13. The slit lamp microscope system of claim 12 . The image analysis unit generates partial analysis data from the processed image.
A slit lamp microscope system according to any one of claims 11 to 13 . A slit lamp microscope and an information processing device,
the slit lamp microscope includes the image acquisition unit and a transmission unit that transmits data to the information processing device,
The information processing device includes a receiving unit that receives the data transmitted by the transmitting unit, the accepting unit, the position of interest information acquiring unit, and the image analyzing unit.
A slit lamp microscope system according to any one of claims 7 to 14 . The slit lamp microscope further includes an image group processing unit that processes the image group collected by the scanning unit.
16. The slit lamp microscope system of claim 15 . The image group processing unit includes a first evaluation unit that evaluates a quality of the image group.
17. The slit lamp microscope system of claim 16 . When the first evaluation unit evaluates that the quality of the image group is good, the transmission unit transmits the image group to the information processing device.
20. The slit lamp microscope system of claim 17 . a first control unit that controls the scanning unit to perform a new scan of the anterior segment when the first evaluation unit evaluates that the image group has poor quality,
19. A slit lamp microscope system according to claim 17 or 18 . The image group processing unit further includes an image set creation unit that selects a series of images corresponding to a scan range from two or more image groups including the image group and a new image group collected by the new scan to create an image set.
20. The slit lamp microscope system of claim 19 . The scanning unit applies two or more scans to the anterior segment to collect two or more sets of images;
The image group processing unit includes an image set creation unit that selects a series of images corresponding to a scan range from the two or more image groups and creates an image set.
17. The slit lamp microscope system of claim 16 . The image group processing unit further includes a second evaluation unit that evaluates the quality of the image set created by the image set creation unit.
22. A slit lamp microscope system according to claim 20 or 21 . When the second evaluation unit evaluates that the quality of the image set is good, the transmission unit transmits the image set to the information processing device.
23. The slit lamp microscope system of claim 22 . and a second control unit that controls the scanning unit to perform a new scan of the anterior segment when the second evaluation unit evaluates that the quality of the image set is not good.
24. A slit lamp microscope system according to claim 22 or 23 . The region of interest includes a region corresponding to any one of the cornea, the iris, the pupil, the crystalline lens, the intraocular lens, the ciliary body, the angle, the trabecular meshwork, and the Schlemm's canal.
A slit lamp microscope system according to any one of claims 1 to 24 .

Images