JP7517903B2 - Slit Lamp Microscope System - Google Patents

Description

The present invention relates to a slit lamp microscope system.

Diagnostic imaging plays an important role in the field of ophthalmology. A variety of ophthalmic imaging devices are used for diagnostic imaging. Types of ophthalmic imaging devices include slit lamp microscopes, fundus cameras, scanning laser ophthalmoscopes (SLO), and optical coherence tomography (OCT).

Of these various ophthalmic devices, the most widely and frequently used is the slit lamp microscope. Slit lamp microscopes are used to illuminate the subject's eye with a slit of light, and to observe and photograph the illuminated cross-section from an oblique or lateral angle using a microscope (see, for example, Patent Documents 1 and 2).

One of the main uses of slit lamp microscopes is for observing the anterior segment of the eye. In this case, doctors observe the entire anterior segment while moving the illumination field and focus position using slit light to determine whether there are any abnormalities. Slit lamp microscopes are also used when prescribing vision correction devices, such as checking the fitting of contact lenses. Furthermore, slit lamp microscopes are also used by people other than doctors, such as optometrists, paramedical staff, and eyeglass store clerks, for the purpose of screening for eye diseases and follow-up (progression observation).

Furthermore, with recent advances in information and communication technology, research and development into telemedicine is progressing. Telemedicine is the practice of providing medical treatment to patients in remote locations using a communication network such as the Internet. Patent documents 3 and 4 disclose technology for operating a slit lamp microscope from a remote location.

To obtain good images using a slit lamp microscope, delicate and complicated operations such as adjusting the illumination angle and the shooting angle are required. However, with the technology disclosed in Patent Documents 3 and 4, even when observing the eye of a subject who is right in front of the examiner, an examiner in a remote location must perform difficult operations, which results in problems such as longer examination times and failure to obtain good images.

As mentioned above, slit lamp microscopes are effective for screening, follow-up, and other examinations, but there is currently a shortage of people skilled in operating the equipment to provide high-quality slit lamp microscopy examinations to many people.

Furthermore, in remote medical care and screening, the doctor who interprets the images (image diagnosis) is often not present at the imaging location. In such cases, images that are not suitable for interpretation may be provided to the doctor. Considering the difficulty of operation and the length of time it takes to perform imaging, it is expected that this problem will be more likely to occur with slit lamp microscopes.

When observing the anterior segment of the eye using a slit lamp microscope, in addition to observing areas such as the cornea, iris (pupil), lens, angle, and anterior chamber, it is also possible to observe floaters in the aqueous humor, lesions, and artificial objects. Artificial objects include intraocular lenses (IOLs) that are replacements for the lens, phakic intraocular lenses (phakic IOLs), intraocular contact lenses (ICLs), minimally invasive glaucoma surgery (MIGS) devices, and stents.

When an image is displayed that depicts multiple objects (such as body parts or man-made objects), it may not be possible to observe all of the depicted objects in detail. For example, when multiple objects with different optical properties (such as reflectance, transmittance, scattering intensity, wavelength characteristics) are depicted, it is conceivable that the depiction manner will differ depending on the object.

JP 2016-159073 A JP 2016-179004 A JP 2000-116732 A JP 2008-284273 A

One objective of the present invention is to provide images that optimally depict various objects to be observed.

An exemplary embodiment of a slit lamp microscope system includes a memory unit, an image acquisition unit, a partial area identification unit, a display condition selection unit, and a display control unit. The memory unit pre-stores display condition information in which display conditions including a display color and a display gradation are associated with one or more parts of the anterior segment. The image acquisition unit acquires an image by scanning the anterior segment of the subject's eye with slit light. The partial area identification unit analyzes the image acquired by the image acquisition unit to identify a partial area corresponding to a predetermined part of the anterior segment of the subject's eye. The display condition selection unit selects a display condition corresponding to this predetermined part from the display condition information. The display control unit causes the image acquired by the image acquisition unit to be displayed on the display means based on the display condition selected by the display condition selection unit.

According to an exemplary embodiment, it is possible to provide images that suitably depict various objects to be observed.

1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram for explaining the operation of a slit lamp microscope system according to an exemplary embodiment. FIG. 1 is a schematic diagram for explaining the operation of a slit lamp microscope system according to an exemplary embodiment. FIG. 1 is a schematic diagram for explaining the operation of a slit lamp microscope system according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. A schematic diagram for explaining the construction of an inference model of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment. 1 is a flowchart illustrating the operation of a slit lamp microscope system according to an exemplary embodiment. 1 is a flowchart illustrating the operation of a slit lamp microscope system according to an exemplary embodiment. 1 is a flowchart illustrating the operation of a slit lamp microscope system according to an exemplary embodiment. 1 is a flowchart illustrating the operation of a slit lamp microscope system according to an exemplary embodiment. 1 is a schematic diagram for explaining the operation of a slit lamp microscope system according to an exemplary embodiment. FIG. 1 is a schematic diagram for explaining the operation of a slit lamp microscope system according to an exemplary embodiment. FIG. 1 is a schematic diagram for explaining the operation of a slit lamp microscope system according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating a configuration of a slit lamp microscope system according to an exemplary embodiment.

Some exemplary aspects will be described in detail with reference to the drawings. Any known technology, such as those disclosed in the documents cited in this specification, may be combined with the exemplary aspects. The entire disclosure of other applications filed by the applicant may be incorporated into this disclosure. For example, at least a portion of the disclosure of an application filed by the applicant relating to a slit lamp microscope, at least a portion of the disclosure of an application filed by the applicant relating to a system including a slit lamp microscope, or at least a portion of the disclosure of an application filed by the applicant relating to a system connectable to a slit lamp microscope may be incorporated into this disclosure.

The slit lamp microscope system according to the exemplary embodiment may be either stationary or portable. The slit lamp microscope system according to the exemplary embodiment may be used in a situation or environment where there is no expert (expert) with expertise in the device or the examination, or where an expert is nearby, or where an expert can monitor, give instructions, and operate from a remote location. Examples of facilities in which the slit lamp microscope system according to the exemplary embodiment may be installed include eyeglass stores, optometrists, medical institutions, health checkup locations, medical examination locations, patients' homes, welfare facilities, public facilities, medical examination vehicles, etc.

When the slit lamp microscope system according to the exemplary embodiment includes two or more devices, these devices may be located in the same location or in different locations. For example, when the slit lamp microscope system includes a slit lamp microscope and an information processing device, the slit lamp microscope and the information processing device may be located remotely.

The slit lamp microscope system according to the exemplary embodiment has at least the imaging function of a slit lamp microscope and a predetermined information processing function, and may optionally further have imaging functions (modalities) other than the slit lamp microscope, an eye characteristic measurement function, other information processing functions, etc.

The system according to the exemplary embodiment (first system) may include one or more slit lamp microscopes, one or more information processing devices, and one or more image reading terminals, and may be usable for remote medical care, for example. One or more of the slit lamp microscope, the information processing device, and the image reading terminal may have an information processing function according to any of the exemplary embodiments. The slit lamp microscope may be a slit lamp microscope according to any of the exemplary embodiments, or may be a slit lamp microscope equipped with at least a part of the slit lamp microscope. The information processing device receives an image acquired by the slit lamp microscope and transmits it to the image reading terminal. The information processing device may also have a function of managing the image acquired by the slit lamp microscope. Note that an image may be transmitted from the slit lamp microscope to the image reading terminal without providing an information processing device. The image reading terminal is a computer used by a doctor (typically a specialist such as an ophthalmologist or an image reading doctor) to read an image acquired by the slit lamp microscope (observe the image to obtain a medical finding). Information input by the image reader to the image reading terminal may be converted into an image reading report or electronic medical record information by the image reading terminal or another computer, and transmitted to the information processing device. Another exemplary embodiment may be configured such that the image reading terminal accepts information input by the image reader, the image reading terminal transmits the accepted information to an information processing device, the information processing device receives the information transmitted from the image reading terminal, and the information processing device or another computer generates an image reading report or electronic medical record information from the received information. The information processing device may have a function of managing the image reading report or electronic medical record information. The information processing device may also have a function of transferring the image reading report or electronic medical record information to another device or another medical system (e.g., an electronic medical record system).

A system according to another exemplary embodiment (a second system) may include one or more slit lamp microscopes, one or more information processing devices, and one or more image reading devices. One or more of the slit lamp microscope, the information processing device, and the image reading device may have an information processing function according to any of the exemplary embodiments. The slit lamp microscope may be the same as the slit lamp microscope of the first system, and the information processing device may be the same as the information processing device of the first system. In addition, an image may be transmitted from the slit lamp microscope to the image reading device without providing an information processing device. The image reading device is, for example, an image processing processor that operates according to a predetermined program, and/or a computer that performs image reading processing of an image acquired by the slit lamp microscope using an artificial intelligence engine (such as an inference engine, an inference model, or a learned model). The image reading device or another computer can generate an image reading report or electronic medical record information from information derived from the image by the image reading device. The generated image reading report or electronic medical record information may be transmitted to the information processing device. In another example, the image reading device can transmit information derived from the image to the information processing device. In this case, the information processing device or another computer can generate an image reading report or electronic medical record information from the information derived from the image by the image reading device. In yet another example, the information derived from the image by the image reading device and/or information generated based on this information (image reading report, electronic medical record information, etc.) can be provided to a doctor as diagnostic support information.

A system according to yet another exemplary embodiment (a third system) may include one or more slit lamp microscopes and one or more information processing devices. The slit lamp microscope and/or the information processing device may have an information processing function according to any of the exemplary embodiments. The slit lamp microscope may be similar to the slit lamp microscope of the first system, and the information processing device may be similar to the information processing device of the first system and/or the information processing device of the second system. In other words, the information processing device may be capable of data communication with an image reading terminal or a similar computer of the first system, and/or may be capable of data communication with an image reading device or a similar computer of the second system.

The slit lamp microscope system according to the exemplary embodiment can be used for remote medical care, but as mentioned above, it is not easy to obtain good images with a slit lamp microscope, and in order to effectively interpret and diagnose images, it is necessary to obtain images of a wide range of the anterior segment "in advance." For these reasons, it can be said that effective remote medical care using a slit lamp microscope has not yet been realized. The exemplary embodiment can provide technology that contributes to the realization of this. However, the use of the exemplary embodiment is not limited to remote medical care, and the exemplary embodiment can also be applied to other uses.

The slit lamp microscope system according to the exemplary embodiment can be used to obtain a series of images (image group, image set) that represent a wide area of the anterior segment with good image quality.

The exemplary embodiment also focuses on the following problem. That is, in the application form (such as remote medical care) assumed by the exemplary embodiment, a series of images (a group of images, a set of images) from a sufficiently wide range of the anterior segment is directly or indirectly provided to the reader using a slit lamp microscope. Even if the quality of the series of images provided is low, it is difficult to retake them, and as a result, there is a risk of problems such as no interpretation being possible or only insufficient interpretation being possible. Therefore, it is necessary to obtain images of "good" quality "in advance." In other words, it is necessary to obtain a series of images with a quality that allows effective diagnosis (such as interpretation) all at once before providing them to the reader. However, in addition to the difficulty of operating a slit lamp microscope, it is extremely difficult to obtain a series of images that represent the entire target area of observation or interpretation with good image quality, taking into account the occurrence of blinking and eye movement during shooting. The present disclosure also takes into consideration how to deal with such problems.

Below, several exemplary aspects are described. Any two or more of these aspects can be at least partially combined. In addition, any known technology can be combined with such combinations, and modifications (substitutions, omissions, etc.) can be made based on them.

At least a portion of the functionality of the elements disclosed in this specification is implemented using circuitry or processing circuitry. The circuitry or processing circuitry may be a general purpose processor, a special purpose processor, an integrated circuit, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), a Field Programmable Gate Array (FPGA), a FPGA, a FPGA, a FPGAs ... Array), conventional circuitry, and any combination thereof. A processor is considered to be a processing circuitry or circuitry, including transistors and/or other circuitry. In this disclosure, a circuitry, unit, means, or similar term is hardware that performs at least some of the disclosed functions or hardware that is programmed to perform at least some of the disclosed functions. The hardware may be hardware disclosed herein or known hardware that is programmed and/or configured to perform at least some of the described functions. If the hardware is a processor that can be considered as a type of circuitry, the circuitry, unit, means, or similar term is a combination of hardware and software, and the software is used to configure the hardware and/or the processor.

<Configuration of the slit lamp microscope system>
The slit lamp microscope system according to the exemplary subject may be, for example, a slit lamp microscope equipped with an information processing function, or a combination of a slit lamp microscope and an information processing device. The configuration of a slit lamp microscope system according to one exemplary embodiment is shown in Figs. 1A and 1B. Fig. 1A shows an example of the overall configuration of a slit lamp microscope system 1, and Fig. 1B shows an example of the configuration of a control unit 7, a data processing unit 8, and a storage unit 10 of the slit lamp microscope system 1. The slit lamp microscope system 1 is capable of selectively applying display conditions according to a target (such as a site or an artificial object) to display an image in the observation or follow-up of a subject's eye E. In Fig. 1A, the symbol C indicates the cornea, the symbol CL indicates the crystalline lens, the symbol IR indicates the iris, and the symbol CA indicates the angle of the eye.

The slit lamp microscope system 1 includes an illumination system 2, an imaging system 3, a movement mechanism 6, a control unit 7, a data processing unit 8, a communication unit 9, and a storage unit 10. The slit lamp microscope system 1 may be a single device, or may be a system including two or more devices. As an example of the latter, the slit lamp microscope system 1 includes a main unit including the illumination system 2, the imaging system 3, and the movement mechanism 6, a computer including the control unit 7, the data processing unit 8, the communication unit 9, and the storage unit 10, and a communication interface that handles data communication between the main unit and the computer. This computer may be installed, for example, together with the main unit, or may be installed on a network. Examples of this computer include the above-mentioned information processing device, image reading terminal, and image reading device.

<Storage unit 10>
The storage unit 10 stores various types of data. For example, the storage unit 10 stores an image group collected by scanning the anterior segment of the subject's eye E with slit light, an image set created from two or more image groups, analysis data generated by image analysis, a processed image constructed from an image group or an image set, and the like. The control unit 7 executes a process of storing data in the storage unit 10 and a process of reading data from the storage unit 10. The storage process and read process are executed as appropriate in various operations of the slit lamp microscope system 1.

Furthermore, the storage unit 10 prestores display condition information 100. In the display condition information 100, display conditions are associated with one or more parts of the eye.

The portion of the eye for which the display condition is set may include, for example, any portion of the anterior segment and/or any portion of the posterior segment. In some exemplary aspects, the portion of the eye for which the display condition is set may include one or more portions of the anterior segment, for example, any of the cornea, iris, lens, angle, anterior chamber, and floaters in the aqueous humor.

The eye part for which the display condition is set may include sub-tissues of a certain part. For example, the eye part for which the display condition is set may include sub-tissues of the cornea (anterior corneal surface, posterior corneal surface, corneal epithelium, Bowman's membrane, lamina propria, Duar's layer, Descemet's membrane, corneal endothelium, etc.), sub-tissues of the lens (anterior lens surface, posterior lens surface, lens nucleus, epithelium, lens capsule, etc.), etc.

The part of the eye for which the display conditions are set may include an artificial object implanted in the eye. Such artificial objects include an IOL, a phakic IOL, an ICL, a MIGS device, a stent, etc.

The objects for which display conditions are set are not limited to parts of the eye (including artificial objects). For example, display conditions can be set for objects depicted as the background of the eye in the image. Such a background is the space (air) outside the eye.

The display conditions represent the conditions (parameters) for displaying an image obtained by the slit lamp microscope system 1 (or another device or system). The display conditions include, for example, a display color and a display gradation. A combination of a display color and a display gradation is sometimes called a color code. The display conditions in this embodiment include a color code.

The display color may be any color that the display device 11 can display, and may be monochromatic or multicolor. Examples of monochromatic colors include black and white, red, green, and blue. Examples of multicolor colors include the three primary colors of light (red, green, and blue).

The display gradation indicates the gradation (the number of shades) when displaying an image obtained by the slit lamp microscope system 1 (or another device or system). The display gradation may include at least one of color gradation and brightness (luminance) gradation. The number of gradations may be any number equal to or greater than 1. For example, there may be 1 gradation, 2 gradations, ..., 256 gradations, etc.

The display gradation may be set, for example, based on the gradation of an image generated by the imaging element 5 described below. The image generated by the imaging element 5 may be, for example, any one of a color image, a monochrome image, and a grayscale image. A color image is, for example, an image represented by a red image, a green image, and a blue image. A monochrome image is, for example, any one of a red image, a green image, and a blue image. A grayscale image is an image represented by white, black, and gray, which is an intermediate color between them. In this embodiment, the gradation of the image generated by the imaging element 5 may be referred to as a reference gradation.

The case of a grayscale image will be described. The same applies to the case of a monochromatic image. The reference gradation _A0 is, for example, 256 gradations (for example, black pixel value=0, white pixel value=255). The display gradation includes one or more gradation values as described above. When the display gradation is one gradation, the same display gradation value is assigned to any gradation value in the reference gradation _A0 . When the display gradation is two gradations, all gradation values included in the reference gradation _A0 are divided into two subsets, the first subset is assigned to the first display gradation value, and the second subset is assigned to the second display gradation value. Similarly, when the display gradation is N gradations, all gradation values included in the reference gradation _A0 are divided into N subsets, and the nth subset is assigned to the nth display gradation value (n=1, 2, ..., N). The number of gradations N of the display gradation is set to any value equal to or less than the number of gradations of the reference gradation _A0 , for example.

Each of the multiple subsets of gradation values included in the reference gradation A ₀ may be a connected set. In other words, for any n ₁ =1 to N, n ₂ =1 to N (n ₁ ≠ n ₂ ), there is a certain magnitude relationship between any gradation value included in the n _{1 -th} subset and any gradation value included in the n ₂ -th subset (i.e., one is always greater than the other or always less than the other).

Several examples of the correspondence between the reference gradation _A0 and the display gradation are shown in Figs. 2A to 2C. A graph 101 shown in Fig. 2A shows a first example (display condition information) 100a of the correspondence between the reference gradation _A0 and the display gradation A. The horizontal axis represents the reference gradation _A0 , and its range is from 0 to _αK . That is, the reference gradation _A0 in this example is the K+1 gradation. An arbitrary gradation value in the reference gradation _A0 is represented as _αk (k = 0, 1, 2, ..., K). On the other hand, the vertical axis represents an arbitrary display gradation A, and its range is from 0 to _aN . That is, the display gradation A in this example is the N+1 gradation. An arbitrary gradation value in the display gradation A is represented as _an (n = 0, 1, 2, ..., N). Here, K and N may or may not be equal. In this example, N is equal to or smaller than K (N≦K) (if N>K, then two or more a _n are associated with a certain α _k ). The graph 101 shows that one display gray scale value a _n is associated with each gray scale value α _k in the reference gray scale A _0. Although the graph 101 is expressed as a continuous straight line, it is actually a discrete set (multiple points arranged on the straight line graph 101) showing the correspondence between a plurality of reference gray scale values α _k and a plurality of display gray scale values a _n . Such a graph 101 is used as a lookup table for converting pixel values (expressed by reference gray scale values α _k ) of an image obtained by the slit lamp microscope system 1 into display gray scale values a _n .

2B shows a second example (display condition information) 100b of the correspondence between the reference grayscale _A0 and the display grayscale. A graph 102 shown in FIG. 2B shows an example of the correspondence between the reference grayscale _A0 and the display grayscale A, similar to the graph 101 in FIG. 2A, but differs from the graph 101 in that it is a curved graph. The coordinate axes and the number of grayscales may be the same as those in the graph 101 in FIG. 2A. Such a graph 102 can be used as a lookup table for converting the pixel value α _k of an image obtained by the slit lamp microscope system 1 into a display grayscale value a _n .

FIG. 2C shows a third example (display condition information) 100c of the correspondence between the reference grayscale _A0 and the display grayscale. The graph 103 shown in FIG. 2C may have the same coordinate axes and the number of grayscales as those of the graphs 101 and 102. Unlike the graphs 101 and 102, which are entirely continuous, the graph 103 is a graph having a plurality of discontinuous points. More specifically, the graph 103 in this example is a graph in which a plurality of portions ₁₀₃ (n) (n=0, 1, 2, ..., N) parallel to the horizontal axis representing the reference grayscale A0 are arranged in a stepped manner. An arbitrary portion 103(n) of the graph 103 defines a range of reference grayscale values _αk1 < _{αk ≦ αk2} as its domain, and the same display grayscale value a _n is associated with any reference grayscale value _αk within this range _αk1 < αk ≦ _αk2 . 2A and 2B show a one-to-one correspondence between the reference grayscale value and the display grayscale value, whereas the graph 103 of Fig. 2C shows a many-to-one correspondence between the reference grayscale value and the display grayscale value. Such a graph 103 can be used as a lookup table for converting the pixel value α _k of the image obtained by the slit lamp microscope system 1 into the display grayscale value a _n .

The form of the information representing the correspondence between the reference grayscale _A0 and the display grayscale is not limited to a graph, but may be, for example, a table in which a plurality of reference grayscale values are associated with a plurality of display grayscale values.

An example of the configuration of the display condition information 100 including the above-mentioned color codes (display color, display gradation) is shown in FIG. 3. The display condition information 100d in this example includes the following parts of the eye: "cornea", "iris", "crystalline lens", "angle", "anterior chamber", "floating matter (in aqueous humor)", "artificial object 1", "artificial object 2", "region of interest", and "background". Furthermore, the display condition information 100d associates the display color "C ₁ " and the display gradation "A ₁ " with the part "cornea", and associates the display color "C ₂ " and the display gradation "A ₂ " with the part "iris". Please refer to FIG. 3 for other parts.

The display color corresponding to a certain part may be grayscale (white, black, and intermediate colors), and the display gradation may be the same as the graph or table information shown in any one of Figures 2A to 2C. For example, as the display gradation "A ₁ " of the part "cornea", the graph or table information shown in any one of Figures 2A to 2C created so as to associate the reference gradation A ₀ with the display gradation A = A ₁ can be used.

Note that if both the display color and display gradation included in the display conditions are the same as the color (grayscale) and gradation (reference gradation) of the image generated by the imaging element 5, that is, if (at least) a part of the image generated by the imaging element 5 is displayed as is without color conversion, there is no need to set display conditions for that part. For example, color conversion can be applied only to the characteristic areas of that part (contours, center, etc.). Furthermore, the conversion indicated by the lookup table set up for that part may be an identity conversion.

Moreover, the display color and display gradation corresponding to a certain part may provide a pseudo-color representation. Pseudo-color is a color representation technique that assigns a predetermined color to each gradation value (luminance value) of the grayscale reference gradation A ₀ , and enables subtle luminance differences that are difficult to distinguish in grayscale (shades of black and white) to be emphasized and represented by changing colors. A lookup table that converts a grayscale representation (reference gradation A ₀ ) into a pseudo-color representation (display gradation) may be, for example, a graph or table information shown in any of Figures 2A to 2C. For example, as the display gradation "A ₁ " of the part "cornea", it is possible to use the graph or table information shown in any of Figures 2A to 2C that is created to associate the reference gradation A ₀ with the display gradation A=A _1. The display gradation A in this example is obtained, for example, by assigning a combination of red (R), green (G), and blue (B) to each gradation value a _{n on} the vertical axis A in Figure 2A. This provides a lookup table for converting each gradation value α _n of the reference gradation A ₀ on the horizontal axis _into a gradation value an of the pseudo color.

In this way, the display conditions for grayscale images (and monochrome images) include a color code defined by one or more display colors that are different from white, black, and intermediate colors thereof, and a display gradation that includes multiple gradation values set based on a reference gradation. Display conditions for color images can be set in a similar manner. That is, the display condition information for grayscale images includes conditions for converting the grayscale values of pixels of the grayscale image to color values, while the display condition information for color images includes conditions for converting the color values of pixels of the color image to another color value (which may be an identity conversion).

Sometimes, a part of a color image is essentially a grayscale image. In that case, pixel value conversion can be performed by applying display condition information related to grayscale images to the part that is essentially a grayscale image, while pixel value conversion can be performed by applying display condition information related to color images to the other part.

The storage unit 10 includes any storage device, and includes at least one of a non-volatile storage device and a volatile storage device. The storage unit 10 includes, for example, at least one of a hard disk drive and a semiconductor memory. Display condition information 100a to 100d in FIGS. 2A to 3 are examples of the display condition information 100 in FIG. 1B. The form, mode, and configuration of the display condition information 100 are not limited to these examples. The display condition information 100 in this embodiment may be any information that represents the correspondence between one or more parts of the anterior eye and display conditions including display color and display gradation.

<Lighting system 2>
The illumination system 2 irradiates slit light onto the anterior segment of the subject's eye E. Reference symbol 2a indicates the optical axis (illumination optical axis) of the illumination system 2. The illumination system 2 may have a configuration similar to that of an illumination system of a conventional slit lamp microscope. Although not shown in the figure, the exemplary illumination system 2 includes, in order from the side farthest from the subject's eye E, an illumination light source, a positive lens, a slit forming unit, and an objective lens.

The illumination light source outputs illumination light. The illumination system 2 may include multiple illumination light sources. For example, the illumination system 2 may include an illumination light source that outputs continuous light and an illumination light source that outputs flash light. The illumination system 2 may also include an illumination light source for the anterior segment and an illumination light source for the posterior segment. The illumination system 2 may also include two or more illumination light sources with different output wavelengths. A typical illumination system 2 includes a visible light source as an illumination light source. The illumination system 2 may also include an infrared light source. The illumination light output from the illumination light source passes through a positive lens and is projected onto the slit forming section.

The slit forming unit generates slit light by passing a portion of the illumination light. A typical slit forming unit has a pair of slit blades. The width of the area (slit) through which the illumination light passes is changed by changing the distance between these slit blades (slit width), and thus the width of the slit light is changed. The slit forming unit may also be configured to change the length of the slit light. The length of the slit light is the cross-sectional dimension of the slit light in a direction perpendicular to the cross-sectional width direction of the slit light corresponding to the slit width. The width and length of the slit light are typically expressed as the dimensions of the image of the slit light projected onto the anterior segment of the eye, but are not limited to this. For example, they can be expressed as the dimensions of the cross section of the slit light at any position, or as the dimensions of the slit formed by the slit forming unit.

The slit light generated by the slit forming unit is refracted by the objective lens and irradiated onto the anterior segment of the subject's eye E.

The illumination system 2 may further include a focusing mechanism for changing the focus position of the slit light. The focusing mechanism, for example, moves the objective lens along the illumination optical axis 2a. The movement of the objective lens can be performed automatically and/or manually. Note that a focusing lens may be disposed at a position on the illumination optical axis 2a between the objective lens and the slit forming section, and the focus position of the slit light may be changed by moving this focusing lens along the illumination optical axis 2a.

Note that FIG. 1A is a top view, and as shown in the figure, in this embodiment, the direction along the axis of the subject's eye E is the Z direction, the direction perpendicular to this that is the left-right direction for the subject is the X direction, and the direction perpendicular to both the X and Z directions is the Y direction. Typically, the X direction is the alignment direction of the left and right eyes, and the Y direction is the direction along the subject's body axis (body axis direction).

<Photography Series 3>
The imaging system 3 images the anterior segment of the eye illuminated with the slit light from the illumination system 2. Reference numeral 3a denotes an optical axis (imaging optical axis) of the imaging system 3. The imaging system 3 in this embodiment includes an optical system 4 and an image sensor 5.

The optical system 4 guides light from the anterior segment of the subject's eye E, which is irradiated with the slit light, to the image sensor 5. The image sensor 5 receives the light guided by the optical system 4 on the imaging surface.

The light guided by the optical system 4 (light from the anterior segment of the subject's eye E) includes the return light of the slit light irradiated on the anterior segment, and may further include other light. Examples of the return light include reflected light, scattered light, and fluorescence. Examples of other light include light from the environment (room light, sunlight, etc.). When an anterior segment illumination system for illuminating the entire anterior segment is provided separately from the illumination system 2, the return light of this anterior segment illumination light is included in the light guided by the optical system 4.

The optical system 4 may have a configuration similar to that of the imaging system of a conventional slit lamp microscope. For example, the optical system 4 includes, in order from the side closest to the subject's eye E, an objective lens, a variable magnification optical system, and an imaging lens. Light from the anterior segment of the subject's eye E irradiated with the slit light passes through the objective lens and the variable magnification optical system, and is imaged on the imaging surface of the image sensor 5 by the imaging lens.

The imaging element 5 is an area sensor having a two-dimensional imaging area, and may be, for example, a charge-coupled device (CCD) image sensor or a complementary metal-oxide semiconductor (CMOS) image sensor.

The imaging system 3 may further include a focusing mechanism for changing the focus position. The focusing mechanism, for example, moves the objective lens along the imaging optical axis 3a. The movement of the objective lens can be performed automatically and/or manually. Note that the focus position can be changed by disposing a focusing lens at a position on the imaging optical axis 3a between the objective lens and the imaging lens, and moving this focusing lens along the imaging optical axis 3a.

The illumination system 2 and the photographing system 3 function as a Scheimpflug camera. That is, the illumination system 2 and the photographing system 3 are configured so that the object plane along the illumination optical axis 2a, the optical system 4, and the imaging plane of the image sensor 5 satisfy the so-called Scheimpflug condition. More specifically, the YZ plane (including the object plane) passing through the illumination optical axis 2a, the principal plane of the optical system 4, and the imaging plane of the image sensor 5 intersect on the same straight line. This allows photographing to be performed with all positions in the object plane (all positions in the direction along the illumination optical axis 2a) in focus.

In this embodiment, for example, imaging is performed with the imaging system 3 focused on a three-dimensional region including at least a portion of a specific area of the anterior eye. With regard to the Z direction, for example, imaging can be performed with the imaging system 3 focused on the entire range from the apex of the anterior surface of the cornea C (Z = Z1) to the apex of the posterior surface of the crystalline lens CL (Z = Z2). Note that Z = Z0 indicates the Z coordinate of the intersection of the illumination optical axis 2a and the imaging optical axis 3a.

Such conditions are typically realized by the configuration and arrangement of elements included in the illumination system 2, the configuration and arrangement of elements included in the imaging system 3, and the relative positions of the illumination system 2 and the imaging system 3. Parameters indicating the relative positions of the illumination system 2 and the imaging system 3 include, for example, the angle θ between the illumination optical axis 2a and the imaging optical axis 3a. The angle θ is set to, for example, 17.5 degrees, 30 degrees, or 45 degrees. Note that the angle θ may be variable.

<Moving mechanism 6>
The moving mechanism 6 moves the illumination system 2 and the imaging system 3. The moving mechanism 6 includes, for example, a movable stage on which the illumination system 2 and the imaging system 3 are mounted, an actuator that operates according to a control signal input from the control unit 7, and a mechanism that moves the movable stage based on a driving force generated by the actuator. In another example, the moving mechanism 6 includes a movable stage on which the illumination system 2 and the imaging system 3 are mounted, and a mechanism that moves the movable stage based on a force applied to an operation device (not shown). The operation device is, for example, a lever. The movable stage is movable at least in the X direction, and may further be movable in the Y direction and/or the Z direction.

In this embodiment, the moving mechanism 6 moves the illumination system 2 and the imaging system 3 together in the X direction, for example. That is, the moving mechanism 6 moves the illumination system 2 and the imaging system 3 in the X direction while maintaining a state in which the above-mentioned Scheimpflug condition is satisfied. In parallel with this movement, the imaging system 3 captures video at a predetermined time interval (imaging rate), for example. As a result, a three-dimensional region of the anterior segment of the subject's eye E is scanned with slit light, and multiple images (image group) corresponding to multiple cross sections within this three-dimensional region are collected.

In a typical example, the longitudinal direction (length direction) of the slit light irradiated to the anterior segment by the illumination system 2 is aligned with the Y direction, and the transverse direction (width direction) is aligned with the X direction. Furthermore, the movement mechanism 6 moves the illumination system 2 and the imaging system 3 integrally in the X direction. In other words, the movement mechanism 6 moves the illumination system 2 and the imaging system 3 so that the slit light irradiated to the anterior segment by the illumination system 2 moves in the width direction of the slit light. This makes it possible to simply and efficiently scan the three-dimensional region of the anterior segment. Note that any combination of the orientation of the slit light and the movement direction may be used.

<Control Unit 7>
The control unit 7 controls each unit of the slit lamp microscope system 1. For example, the control unit 7 controls elements of the illumination system 2 (illumination light source, slit forming unit, focusing mechanism, etc.), elements of the imaging system 3 (focusing mechanism, image sensor, etc.), the moving mechanism 6, the data processing unit 8, the communication unit 9, etc. The control unit 7 may also be capable of executing control for changing the relative position between the illumination system 2 and the imaging system 3. Furthermore, the control unit 7 may also be capable of controlling a device (e.g., a display device) included in the slit lamp microscope system 1 or an apparatus (e.g., a display device) connected to the slit lamp microscope system 1.

The control unit 7 includes a processor, a main memory device, an auxiliary memory device, etc. The auxiliary memory device stores a control program, etc. The control program, etc. may be stored in a computer or a memory device that is accessible to the slit lamp microscope system 1. The functions of the control unit 7 are realized by the cooperation of software such as the control program and hardware such as a processor.

The control unit 7 can apply the following control to the illumination system 2, the imaging system 3, and the movement mechanism 6 in order to scan the three-dimensional area of the anterior segment of the subject's eye E with slit light.

First, the control unit 7 controls the movement mechanism 6 to place the illumination system 2 and the imaging system 3 at a predetermined scan start position (alignment control). The scan start position is, for example, a position corresponding to the end (first end) of the cornea C in the X direction, or a position further away from the axis of the subject's eye E.

The symbol X0 in FIG. 4A shows an example of a scan start position that corresponds to the first end of the cornea C in the X direction. Also, the symbol X0' in FIG. 4B shows an example of a scan start position that is farther away from the axis EA of the test eye E than the position that corresponds to the first end of the cornea C in the X direction.

The control unit 7 controls the illumination system 2 to start irradiating the anterior segment of the subject's eye E with slit light (slit light irradiation control). Note that the slit light irradiation control may be performed before or during the execution of the alignment control. The slit light is typically continuous light, but may also be intermittent light (pulsed light). The control of the pulsed light is synchronized with the imaging rate of the imaging system 3. Furthermore, the slit light is typically visible light, but may also be infrared light, or a mixture of visible light and infrared light.

The control unit 7 controls the imaging system 3 to start imaging a moving image of the anterior segment of the subject's eye E (imaging control). Note that imaging control may be performed before or during the execution of alignment control. Typically, imaging control is performed simultaneously with or after slit light irradiation control.

After performing the alignment control, the slit light irradiation control, and the photographing control, the control unit 7 controls the moving mechanism 6 to start moving the illumination system 2 and the photographing system 3 (movement control). The movement control moves the illumination system 2 and the photographing system 3 as a unit. That is, the illumination system 2 and the photographing system 3 are moved while maintaining the relative position (angle θ, etc.) between the illumination system 2 and the photographing system 3. Typically, the illumination system 2 and the photographing system 3 are moved while maintaining a state in which the above-mentioned Scheimpflug condition is satisfied. The illumination system 2 and the photographing system 3 are moved from the above-mentioned scan start position to a predetermined scan end position. The scan end position is, for example, a position corresponding to the end (second end) of the cornea C opposite the first end in the X direction, similar to the scan start position, or a position further away from the axis of the subject's eye E. In such a case, the range from the scan start position to the scan end position is the scan range.

Typically, a slit of light with the X direction as the width direction and the Y direction as the length direction is irradiated onto the anterior segment, and the illumination system 2 and the imaging system 3 are moved in the X direction while the imaging system 3 captures a moving image.

Here, the length of the slit light (i.e., the dimension of the slit light in the Y direction) is set, for example, to be greater than or equal to the diameter of the cornea C on the surface of the subject's eye E. That is, the length of the slit light is set to be greater than or equal to the corneal diameter in the Y direction. Also, as described above, the movement distance of the illumination system 2 and the imaging system 3 by the movement mechanism 6 (i.e., the scanning range) is set to be greater than or equal to the corneal diameter in the X direction. This makes it possible to scan at least the entire cornea C with the slit light.

By performing such a scan, multiple anterior eye images with different irradiation positions of the slit light are obtained. In other words, a moving image is obtained that depicts the irradiation position of the slit light moving in the X direction. An example of multiple such anterior eye images (i.e., a group of frames that make up a moving image) is shown in Figure 5.

Figure 5 shows multiple anterior eye images (frame groups, image groups) F1, F2, F3, ..., FN. The subscript n of these anterior eye images Fn (n = 1, 2, ..., N) represents the chronological order. That is, the anterior eye image acquired nth is represented by the symbol Fn. The anterior eye image Fn includes a slit light irradiation area An. As shown in Figure 5, the slit light irradiation areas A1, A2, A3, ..., AN move to the right in the chronological order.

In the example shown in FIG. 5, the longitudinal direction (length direction) of the slit light coincides with the Y direction (up-down direction, body axis direction), the transverse direction (width direction) coincides with the X direction (left-right direction, horizontal direction), and the moving direction of the slit light coincides with the X direction. In addition, the scan start position is a position outside (left side) of the first end (left end) of the cornea C, and the scan end position is a position outside (right side) of the second end (right end) of the cornea C. According to such a scanning mode, a three-dimensional region including the entire cornea C can be photographed. Note that the scan start position and/or scan end position are not limited to this example, and for example, the scan start position and/or scan end position may be the end of the cornea C in the X direction. In addition, various scanning parameters such as the direction of the slit light, the moving direction of the slit light, and the number of scans can also be set arbitrarily.

The exemplary control unit 7 shown in FIG. 1B includes a display condition selection unit 71 and a display control unit 72.

<Display condition selection unit 71>
The display condition selection unit 71 selects from the display condition information 100 a display condition corresponding to a part of the subject's eye E (anterior eye part) designated by the user or the slit lamp microscope system 1. For example, when the display condition information 100d shown in FIG. 3 is applied and the designated part is the "cornea", the display condition selection unit 71 searches for "cornea" in the "part" column of the display condition information 100d, and obtains the display color "C ₁ " and the display gradation "A ₁ " corresponding to the searched part "cornea". In this way, the display color and the display gradation corresponding to the designated part are selected from the display condition information 100d. When the designated part is the "angle", the display condition selection unit 71 can select the display color "C ₄ " and the display gradation "A ₄ " corresponding to the part "angle" from the display condition information 100d by performing the same process.

The part of the eye E to be examined is specified by the user or the slit lamp microscope system 1 as described above. When the user specifies the part of the eye E to be examined, for example, a graphical user interface (GUI) for specifying the part is displayed on the display device 11 by the control unit 7. The user can specify the desired part using this GUI. The GUI is operated, for example, by a touch operation and/or via an operation device (not shown). In another aspect of the user's specification, an image acquired by the slit lamp microscope system 1 (or an image acquired from the outside) is displayed on the display device 11 by the control unit 7. The user can specify the desired area in the displayed image by a touch operation or using an operation device. At this time, the user can input an identifier (such as the name of the part) of the part corresponding to the specified area. In another exemplary aspect, the data processing unit 8 may be configured to analyze an image in which the user specifies an area and generate information about the specified area (such as the shape, size, brightness, positional relationship with other areas, etc.), identify the part corresponding to the specified area based on this information, and generate an identifier for the identified part.

In an exemplary embodiment in which the slit lamp microscope system 1 specifies a part of the subject's eye E, the data processing unit 230 may be configured to specify a part corresponding to an image area specified by the partial area specification unit 86 described below. The processing for this purpose includes, for example, a process of generating information (shape, size, brightness, positional relationship with other areas, etc.) about the image area specified by the partial area specification unit 86 by image analysis, and a process of generating an identifier of the part corresponding to the specified image area from the results of this image analysis.

<Display control unit 72>
The display control unit 72 controls the display device 11 and processes (e.g., processes, converts, etc.) the information and data to be displayed. For example, the display control unit 72 processes an image based on the display conditions selected by the display condition selection unit 71 and displays the image on the display device 11. The display device 11 may be included in the slit lamp microscope system 1, or may be a peripheral device connected to the slit lamp microscope system 1.

In some exemplary aspects, the display control unit 72 may be configured to apply the display color and display gradation included in the display conditions selected by the display condition selection unit 71 to an image area identified by a partial area identification unit 86 described below from the image of the subject's eye E. For example, when a display color "C ₁ " and a display gradation "A ₁ " corresponding to the part "cornea" are selected from the display condition information 100d, the display condition information 100 can be applied to the image area corresponding to the cornea.

As described above, in some exemplary aspects, the slit lamp microscope system 1 acquires a gradation image expressed in a predetermined reference gradation, and the display condition information 100 includes information (display color code) defined by one or more display colors and a display gradation including multiple gradation values set based on the reference gradation. In such aspects, the display control unit 72 may be configured to convert the gradation values of pixels in an image area identified from the image of the subject's eye E by the partial area identification unit 86 described below, based on a display color code associated with a part corresponding to this image area.

The display color code may be a color code. The color code may be information defined by one or more display colors different from white, black, or intermediate colors thereof, and a display gradation including a plurality of gradation values set based on the reference gradation described above. Another aspect of the display color code may be information for determining a new color value from the pixel value (color value) of a color image. The new color value may be white, black, or an intermediate color thereof.

When the slit lamp microscope system 1 acquires a grayscale image, the display control unit 72 can convert the grayscale values of the pixels of the image area identified from the image of the subject's eye E by the partial area identification unit 86 described below into color values based on the color code associated with the part corresponding to this image area. In other words, this exemplary embodiment converts a partial area of the grayscale image into a color value corresponding to this partial area.

When the slit lamp microscope system 1 acquires a color image, the display control unit 72 can convert the color values of the pixels in the image area identified from the image of the subject's eye E by the partial area identification unit 86 described below into new color values based on the display color code associated with the part corresponding to this image area. In other words, this exemplary embodiment converts a partial area in a color image into another color value corresponding to this partial area.

<Data Processing Unit 8>
The data processing unit 8 executes various types of data processing. The data to be processed may be either data acquired by the slit lamp microscope system 1 or data input from the outside. The data processing unit 8 can process images acquired by the imaging system 3.

The data processing unit 8 includes a processor, a main memory device, an auxiliary memory device, etc. The auxiliary memory device stores a data processing program, etc. The data processing program, etc. may include a model constructed by machine learning (a trained model, an inference model, etc.). The data processing program, etc. may be stored in a computer or a memory device accessible to the slit lamp microscope system 1. The functions of the data processing unit 8 are realized by cooperation between software such as the data processing program and hardware such as a processor.

The exemplary data processing unit 8 shown in FIG. 1B includes an image group processing unit 80, a processed image construction unit 85, and a partial region identification unit 86.

<Image Group Processing Unit 80>
The image group processor 80 processes at least some of the images collected by the anterior eye scan using slit light. Several examples of the image group processor 80 will be described. Figures 6A, 6B, 6C, and 6D show image group processors 80A, 80B, 80C, and 80D, which are first, second, third, and fourth examples of the image group processor 80, respectively. Note that the configuration of the image group processor 80 is not limited to these. For example, the image group processor 80 may include a combination of any two or more of the four image group processors 80A, 80B, 80C, and 80D. In addition, it is possible to provide the image group processor 80 with any element for obtaining the same type of result or a similar type of result.

The slit lamp microscope system 1 of this embodiment applies one or more scans to the anterior segment of the subject's eye E. Each scan results in an image group such as the multiple anterior segment images F1 to FN shown in FIG. 5. The image group processing unit 80 can process one or more image groups obtained in this manner.

The slit lamp microscope system 1 may be capable of applying two or more scans to the anterior segment. For example, if a good set of images is not obtained in the first scan, a second scan is performed. More generally, if a good set of images is not obtained in the first through v scans, a v+1 scan is performed (v is an integer equal to or greater than 1). In some exemplary aspects, if a good set of images (image set) is not obtained from the v image sets obtained in the first through v scans, a v+1 scan is performed. In some other exemplary aspects, if none of the v image sets obtained in the first through v scans is a good image set, a v+1 scan is performed. Note that the manner in which two or more scans are performed is not limited to these.

When performing two or more scans, the start and end positions of these scans (i.e., the scan range) may be the same, or one or both of the start and end positions may be different. Typically, two or more scans are applied to the same scan range. As a result, a group of images such as the multiple anterior eye images F1 to FN shown in FIG. 5 is obtained in the same number as the number of scans.

Taking into consideration the eye movement of the subject's eye E, the scan range (scan start position, scan end position) is typically defined by the operation of the slit lamp microscope system 1, not by the position or range in the anterior segment. For example, the start point of the movement of the illumination system 2 and the imaging system 3 during scanning is set as the scan start position, and the end point is set as the scan end position.

On the other hand, it is also possible to define the scan range based on the position and range in the anterior segment. In this case, an operation (tracking) is applied in which the illumination system 2 and the imaging system 3 are moved so as to follow the movement of the subject's eye E. The slit lamp microscope system 1 of this example has, for example, a tracking function similar to that of a conventional system. The tracking function includes, for example, video recording of the anterior segment, extraction of landmarks from each frame, calculation of deviation between frames using the landmarks, and movement control of the illumination system 2 and the imaging system 3 to cancel out the deviation.

As another example of defining (setting) the scan range, it is possible to set the scan range for an image that has already been collected. In other words, it is possible to configure the scan range to be set retroactively (after scanning). Note that the scan range in this example represents the range of the image that will be provided for subsequent processing.

A first example of such a post-event scan range setting will be described. The slit lamp microscope of this example is capable of anterior segment photography, and is capable of recognizing the positional relationship between the anterior segment photography optical system and the scanning optical system (illumination system 2 and photography system 3). In this example, first, an image group (wide-area image group) is collected by scanning a sufficiently wide range (particularly in the X direction and Y direction) of the anterior segment, while performing anterior segment photography. Next, a scan range is set for the anterior segment image obtained by anterior segment photography. This setting may be performed either manually or automatically. The automatic setting includes, for example, a process of analyzing the anterior segment image to detect a landmark, and a process of setting the scan range by referring to this landmark. This landmark may be, for example, the corneal limbus, the angle, or the pupil margin. When the scan range is set, a partial area of each wide-area image corresponding to this scan range is specified based on the positional relationship of the optical system described above. Finally, an image corresponding to the scan range is formed by cropping this specified partial area from the wide-area image. As a result, a group of images corresponding to the set scan range is obtained. In this example, when the fixation of the subject's eye is stable (or when it is assumed or can be assumed that the fixation is stable), the anterior segment may be captured as still images. When the anterior segment is captured as video, the wide-area image group and the frame group in the video are associated with each other based on, for example, control of the scan and video capture (synchronization information, etc.), and the same processing as described above is performed for each associated pair of wide-area image and frame.

A second example of the post-scan range setting will be described. In this example, the anterior eye does not need to be photographed in parallel, and a sufficiently wide range of the anterior eye (particularly in the X and Y directions) is scanned to collect a group of wide-area images. Next, a partial area corresponding to the desired scan range is specified for each wide-area image. This specification may be performed either manually or automatically. The automatic specification includes, for example, a process of analyzing the wide-area image to detect a landmark (for example, the corneal limbus or the iridocorneal angle) and a process of setting the scan range by referring to the landmark. In addition, a landmark may be manually specified for one of the wide-area images (reference wide-area image), and the reference wide-area image and an adjacent wide-area image may be analyzed to specify a landmark for the adjacent wide-area image. By sequentially applying such processing, it is possible to specify landmarks for all wide-area images. Furthermore, a partial area of the wide-area image corresponding to the scan range can be specified based on the landmark, and an image corresponding to the scan range can be formed by cropping the specified partial area from the wide-area image. This allows a group of images corresponding to the set scan range to be obtained.

A first example of the image group processing unit 80 will be described. The image group processing unit 80A shown in FIG. 6A includes an image group evaluation unit 81. The image group evaluation unit 81 evaluates the quality of an image group collected in one scan of the subject's eye E. In some exemplary aspects, the image group evaluation unit 81 is configured to determine whether the image group satisfies a predetermined condition, and is typically configured to determine whether each image included in the image group satisfies the predetermined condition.

The specified conditions (image group evaluation conditions) are, for example, conditions related to the image quality required for effective interpretation and diagnosis, such as conditions related to the subject's eye E, conditions related to the slit lamp microscope system 1, and conditions related to the environment. The image group evaluation conditions may include, for example, either or both of "the image to be evaluated is not affected by blinking (blink condition)" and "the image to be evaluated is not affected by eye movement (eye movement condition)." The image group evaluation conditions may be any index related to image quality evaluation, and may include, for example, any of brightness, contrast, noise, signal-to-noise ratio (SNR), tone reproduction, dynamic range, sharpness, vignetting, aberration (spherical aberration, astigmatism, coma aberration, field curvature, distortion aberration, chromatic aberration, etc.), color accuracy, exposure accuracy, lens flare, moire, and artifacts. The image selection conditions are not limited to these examples and may be selected arbitrarily.

The blinking condition will now be described. For an image included in the image group, the image group evaluation unit 81 judges, for example, whether the image contains a reflected image of slit light irradiated onto the anterior segment of the subject's eye E. This judgment is a process that utilizes the fact that the reflected image of the slit light is not depicted in an image captured during a blink, and that the reflected image of the slit light is depicted significantly brighter than other regions, and is performed, for example, based on the luminance distribution in the image.

As an example, the image group evaluation unit 81 creates a luminance histogram from the image and determines whether there are any pixels with a luminance equal to or greater than a predetermined threshold. If it is determined that there is a luminance equal to or greater than the predetermined threshold, it is determined that the image contains a reflected image of slit light. This example has the advantage that the processing is extremely simple, but there is a risk of erroneous detection of high-luminance noise or reflections of external light.

As another example, the image group evaluation unit 81 creates a brightness histogram from the image and determines whether there are a predetermined number of pixels with a brightness equal to or greater than a predetermined threshold. If it is determined that there are a predetermined number of pixels with a brightness equal to or greater than a predetermined threshold, it is determined that the image contains a reflected image of slit light. This example has the advantage that the above-mentioned erroneous detection can be prevented by simple processing.

Explains eye movement conditions. For an image included in an image group, the image group evaluation unit 81 determines whether or not the image is affected by eye movement by, for example, comparing the image with adjacent images. This determination is a process that utilizes the fact that "image jumps" occur when eye movement occurs during video capture.

As an example, the image group evaluation unit 81 detects landmarks from each of the image in question and the adjacent images, calculates the amount of deviation of these landmarks, and determines whether this amount of deviation is equal to or greater than a predetermined threshold. If it is determined that the amount of deviation is equal to or greater than the predetermined threshold, it is determined that eye movement has occurred. Here, the landmarks may be, for example, the cornea, iris, pupil, or angle of the eye. The threshold is calculated based on predetermined scanning conditions, such as the shooting rate of the shooting system 3 and the moving speed of the moving mechanism 6.

In another example, the image group evaluation unit 81 may be configured to determine the presence or absence of eye movement from a single image. For example, when the shooting rate of the shooting system 3 is slow, rapid eye movement may cause "blur" in the image. The image group evaluation unit 81 can determine the presence or absence of eye movement by using blur detection. Blur detection is typically performed using known techniques such as edge detection.

The image group evaluation unit 81 may include an artificial intelligence engine for determining whether the input image satisfies the above image selection conditions. This artificial intelligence engine typically includes a convolutional neural network (CNN), which is trained in advance using training data including a large number of images acquired by a slit lamp microscope and a determination result of whether each image satisfies the image selection conditions. The images included in the training data are not limited to images acquired by a slit lamp microscope, and may be, for example, images acquired by other ophthalmic modalities (fundus cameras, OCT devices, SLOs, surgical microscopes, etc.), images acquired by image diagnostic modalities of other medical departments (ultrasound diagnostic devices, X-ray diagnostic devices, X-ray CT devices, magnetic resonance imaging (MRI) devices, etc.), images generated by processing actual images, pseudo images, etc. In addition, the methods and technologies (for example, hardware, software, machine learning methods, types of neural networks, etc.) used in the artificial intelligence engine are arbitrary.

A series of images constituting an image group collected in one scan may be associated with multiple positions in the scan range. This association is performed, for example, by the image group processing unit 80. A specific example will be described. The scan range along the X direction is divided into (N-1) sections, the scan start position is set to the first position, and the scan end position is set to the Nth position (N is an integer equal to or greater than 2). This sets N positions in the scan range. The N positions are represented by B1, B2, B3, ..., BN (not shown). The intervals between the N positions B1 to BN may be equal or different. The number (N) of positions B1 to BN is set based on, for example, the scan start position, the scan end position, the moving speed (scanning speed) of the slit light, the frame rate of the imaging system 3, and the like. The number (N) of positions B1 to BN set in this example is equal to the number of images collected in one scan. Note that the number (N) of positions B1 to BN is not limited to this example, and the setting method is also not limited to this example. Assume that one scan is applied to the anterior segment of the subject's eye E, and a group of images F1, F2, F3, ..., FN shown in FIG. 5 is obtained. The image group processing unit 80 can assign an image Fn to a position Bn. This results in N images Fn (n = 1, 2, ..., N) corresponding to the N positions Bn (n = 1, 2, ..., N).

The image group may include only a series of images, or may further include other information. Examples of information included in the image group along with the series of images include various types of additional information, such as subject information, information about the examined eye, shooting date and time, and shooting conditions. It is also possible to include images obtained with other modalities and examination data acquired by an examination device in the image group. Some examples of the configuration and operation of the image group evaluation unit 81 will be described later with reference to Figures 7A to 7D.

A second example of the image group processing unit 80 will be described. The effectiveness of this example is particularly evident when two or more scans are applied to the subject's eye E. The image group processing unit 80B shown in FIG. 6B includes an image set creation unit 82 in addition to the image group evaluation unit 81 similar to that in FIG. 6A. The image group processing unit 80B is provided with two or more image groups collected in two or more scans applied to the subject's eye E. The image group evaluation unit 81 in this example may perform quality evaluation on each of these image groups, or may perform quality evaluation only on the image group collected in the initial scan (first scan). The image set creation unit 82 selects a series of images corresponding to the scan range from the two or more image groups input to the image group processing unit 80B to create an image set.

The range represented by the series of images constituting the image set may be, for example, a three-dimensional area to which any one of the two or more scans is applied, or a three-dimensional area set based on the scan ranges in at least two of the two or more scans. As an example of the former, the maximum or minimum scan range among the two or more scan ranges to which the two or more scans are respectively applied may be adopted. As an example of the latter, the union or intersection of at least two scan ranges may be adopted.

The image set may include only the series of images, or may include other information as well. Examples of information included in the image set along with the series of images include various types of additional information, such as subject information, information about the examined eye, shooting date and time, and shooting conditions. Images obtained with other modalities and examination data acquired by an examination device can also be included in the image set.

A third example of the image group processing unit 80 will be described. As with the second example, the effectiveness of this example is particularly evident when two or more scans are applied to the subject's eye E. The image group processing unit 80C shown in FIG. 6C includes an image group evaluation unit 81 similar to that in FIG. 6A, as well as an image set creation unit 82 similar to that in FIG. 6B, but the image set creation unit 82 in this example includes a selection unit 821.

The image group processing unit 80C is provided with two or more image groups collected in two or more scans applied to the subject's eye E. The image group evaluation unit 81 in this example may perform a quality evaluation on each of these image groups, or may perform a quality evaluation only on the image group collected in the initial scan (first scan). The image set creation unit 82 selects a series of images corresponding to the scan range from the two or more image groups input to the image group processing unit 80B to create an image set. In creating this image set, the selection unit 821 selects images that satisfy predetermined conditions from the two or more image groups.

The specified conditions (image selection conditions) may be the same as the image group evaluation conditions described above, or may be different. For example, if the image group evaluation unit 81 is configured to provide two or more image groups to the selection unit 821 after the quality evaluation of each image is performed by the image group evaluation unit 81, the selection unit 821 may be configured to select images taking into consideration conditions related to the image arrangement (e.g., blinking conditions, eye movement conditions, etc.). Note that the image selection conditions are not limited to these examples, and the relationship between the image group evaluation conditions and the image selection conditions is not limited to these examples.

The following describes the cases where the selection unit 821 takes blink conditions into account and the cases where it takes eye movement conditions into account. Note that when other conditions are taken into account, the selection unit 821 can execute processing similar to the specific examples (see Figures 7A to 7D) related to the image group evaluation unit 81 described below.

Image selection that takes blinking conditions into consideration will now be described. Note that blinking conditions are conditions for confirming that "the image to be evaluated is not affected by blinking." For an image included in two or more image groups, the selection unit 821 determines, for example, whether the image contains a reflected image of slit light irradiated onto the anterior segment of the subject's eye E. This determination is a process that utilizes the fact that the reflected image of the slit light is not depicted in an image captured during a blink, and that the reflected image of the slit light is depicted significantly brighter than other regions, and is performed, for example, based on the luminance distribution in the image.

As an example, the selection unit 821 creates a luminance histogram from the image and determines whether there are pixels with a luminance equal to or greater than a predetermined threshold. If it is determined that there is a luminance equal to or greater than the predetermined threshold, it is determined that the image contains a reflected image of slit light. This example has the advantage that the processing is extremely simple, but there is a risk of false detection of high-luminance noise or reflections of external light.

As another example, the selection unit 821 creates a luminance histogram from the image and determines whether there are a predetermined number of pixels with a luminance equal to or greater than a predetermined threshold. If it is determined that there are a predetermined number of pixels with a luminance equal to or greater than a predetermined threshold, it is determined that the image contains a reflected image of slit light. This example has the advantage that the above-mentioned erroneous detection can be prevented by simple processing.

Image selection that takes into account eye movement conditions is described below. The eye movement conditions are conditions for confirming that "the image to be evaluated is not affected by eye movement." For an image included in two or more image groups, the selection unit 821 determines whether or not the image is affected by eye movement, for example, by comparing the image with adjacent images. This determination is a process that utilizes the fact that "image jumps" occur when eye movement occurs during video capture.

As an example, the selection unit 821 detects landmarks from each of the image in question and the adjacent image, calculates the amount of deviation of these landmarks, and determines whether this amount of deviation is equal to or greater than a predetermined threshold. If it is determined that the amount of deviation is equal to or greater than the predetermined threshold, it is determined that eye movement has occurred. Here, the landmarks may be, for example, the cornea, iris, pupil, or angle of the eye. Furthermore, the threshold is calculated based on predetermined scanning conditions, such as the imaging rate of the imaging system 3 and the moving speed of the moving mechanism 6.

In another example, the selection unit 821 may be configured to determine the presence or absence of eye movement from one image. For example, when the shooting rate of the shooting system 3 is slow, the occurrence of rapid eye movement may cause "blur" in the image. The selection unit 821 can determine the presence or absence of eye movement by using blur detection. Blur detection is typically performed using a known technique such as edge detection.

As with the image group evaluation unit 81, the selection unit 821 may include an artificial intelligence engine for determining whether the input images satisfy the above image selection conditions.

A series of images included in the image set created by the image set creation unit 82 may be associated with multiple positions in the scan range. For example, the selection unit 821 may be configured to select images so as to assign one or more images to each of multiple positions in the scan range.

A specific example will be described. As described above, the scan range along the X direction is divided into (N-1) sections, the scan start position is set to the first position, and the scan end position is set to the Nth position (N is an integer equal to or greater than 2). In this way, N positions are set in the scan range. The N positions are represented as B1, B2, B3, ..., BN.

Assume that two or more scans are applied to the anterior segment of the subject's eye E, resulting in the acquisition of a group of images F1, F2, F3, ..., FN shown in FIG. 5. Note that in the above explanation, the N images F1 to FN shown in FIG. 5 are considered to be a group of images obtained from a single scan, but in this specification, for the sake of simplicity, the N images F1 to FN shown in FIG. 5 may be referred to as an arbitrary group of images (multiple images). For example, in the explanation of this example, the N images F1 to FN are referred to as a series of images included in an image set.

For such N positions B1 to BN and N images F1 to FN, the selection unit 821 can assign an image Fn to a position Bn. This provides N images Fn (n = 1, 2, ..., N) corresponding to the N positions Bn (n = 1, 2, ..., N), and creates an image set in which the image group Fn is a "series of images."

In some exemplary aspects, if the image group evaluation unit 81 evaluates that the quality of the image group obtained in the first scan is not good, a second scan is performed automatically or in response to a user's instruction. If the number of scans applied to the anterior segment is two or more, the slit lamp microscope system 1 performs an operation of applying two or more scans to the anterior segment and an operation of selecting a series of images from two or more image groups collected thereby, but the manner of performing these operations may be arbitrary. As a first example, the slit lamp microscope system 1 may be configured to alternately repeat a scan of the anterior segment and a selection of images from the image group acquired by this scan in response to the evaluation that the quality of the image group obtained in the first scan is not good. As a second example, the slit lamp microscope system 1 may be configured to perform two or more scans together and then select a series of images from two or more image groups collected thereby in response to the evaluation that the quality of the image group obtained in the first scan is not good. These two examples will be described below. Note that instead of or in addition to the selection of images from the image group, the image group evaluation unit 81 may evaluate the image group. As previously discussed, image group evaluation and image selection may be similar, analogous, or interchangeable processes. In some example aspects, matters described as image group evaluation may be applied to image selection, and conversely, matters described as image selection may be applied to image group evaluation.

A first example of the mode of scanning and image selection executed after the image group evaluation unit 81 evaluates that the quality of the image group obtained in the first scan is not good is alternating repetition of scanning and image selection. More specifically, in the first example, for example, a set of application of a scan to the anterior eye and selection of an image from the image group obtained by this scan is repeatedly executed a predetermined number of times. That is, in the first example, U sets of operations are executed in the order of the first set of operations (scanning and image selection), the second set of operations (scanning and image selection), ..., the Uth set of operations (scanning and image selection) (U is an integer equal to or greater than 1). In addition, the set of the first scan performed before this alternating repetition and the quality evaluation (image group evaluation unit 81) of the image group obtained in the first scan will be called the 0th set of operations.

Here, the number of scans in the u-th set may be any number equal to or greater than 1 (u = 0, 1, ..., U). Also, the number of scans in the _u1- th set and the number of scans in the _u2 -th set may be equal to or different from each other ( _u1 = 0, 1, ..., U; _u2 = 0, 1, ..., U; _u1 ≠ _u2 ).

In a first example, the selection unit 821 may be configured to select images from two or more image groups collected in two or more scans that have already been performed to create a provisional image set. That is, the slit lamp microscope system 1 may be configured to create a provisional image set from two or more image groups obtained in two or more scans performed up to that point at any point during the alternating repetition of scanning and image selection. For example, after the scan in the u-th set is performed, the selection unit 821 is configured to create a provisional image set from all images obtained in the 0-th set to the u-th set. With this configuration, a provisional image set can be created from the images obtained up to the current point in time to build a final image set.

When the above configuration for creating a provisional image set is applied, the following configuration can be combined. When a new scan is applied to the anterior segment of the subject's eye E, the selection unit 821 first selects an image from a new image group collected in the new scan. Next, the selection unit 821 creates a new provisional image set by adding an image selected from the new image group to a provisional image set based on one or more scans performed before the new scan. For example, after a scan in the (u+1)th set is performed, the selection unit 821 can first select an image from the image group obtained in the (u+1)th set. Furthermore, the selection unit 821 can create a new provisional image set by adding an image selected from the image group obtained in the (u+1)th set to a provisional image set based on the image groups obtained in the 0th to uth sets. With this configuration, each time a scan is applied to the anterior segment, the provisional image set can be sequentially updated based on the image group obtained in this scan. This makes it possible to reliably and efficiently construct a final image set.

When the above configuration for creating (and updating) a provisional image set is applied, the following configuration can be combined. The control unit 7 (or the image set creation unit 82 (selection unit 821)) includes an image number counter that counts the number of images included in the provisional image set. When the number of images included in the provisional image set reaches a predetermined number, the control unit 7 controls the scanning unit (illumination system 2, shooting system 3, movement mechanism 6) and the selection unit 821 to end the alternating repetition of applying the scan and selecting the images. Here, the predetermined number is the number of a series of images included in the final image set, and is set in advance or from the processing status. In addition, the control unit 7 determines whether the number of images included in the provisional image set has reached the predetermined number. This determination may be made only by comparing the numbers. Alternatively, when multiple positions in the scan range are associated with a series of images (as described above), it may be determined whether corresponding images have been assigned to all of the multiple positions. With this configuration, when the number required for the final image set is obtained, the alternating repetition of scanning and image selection can be automatically ended.

When the above configuration for creating (and updating) a provisional image set is applied, the following configuration can be further combined. The control unit 7 includes a repetition counter that counts the number of alternating repetitions of scanning and image selection. The counted number may be defined in units of pairs of scanning and image selection (the first pair to the Uth pair), or may be defined in units of the number of scans. When the number of repetitions reaches a predetermined number, the control unit 7 controls the scanning unit (illumination system 2, shooting system 3, moving mechanism 6) and the selection unit 821 to end the alternating repetition of scanning application and image selection. When the number of repetitions is defined in units of pairs of scanning and image selection, the predetermined number is equal to the total number of pairs (U+1) set in advance. When the number of repetitions is defined in units of the number of scans, the predetermined number is equal to the total number of scans set in advance. In addition, the control unit 7 determines whether the number of repetitions has reached the predetermined number. According to this configuration, scanning and image selection can be automatically ended at a stage where they have been repeated a predetermined number of times. Without this configuration, scanning and image selection are repeated until the number of images required to construct the final image set is selected, tiring the subject and reducing imaging efficiency. In particular, when imaging multiple subjects in succession, imaging throughput is significantly impaired.

As described above, this embodiment may be configured to automatically end the alternating repetition of scanning and image selection. The condition for this automatic end is not limited to the above two examples, and may be, for example, an instruction input from the user. Alternatively, the elapsed time from the start of the alternating repetition of scanning and image selection may be measured, and the alternating repetition may be ended when a predetermined time is reached. In addition, when the repetition rate of scanning and image selection is constant, the automatic end control based on the elapsed time is equivalent to the automatic end control based on the number of repetitions described above. The image set creation unit 82 can create an image set based on a provisional image set when the alternating repetition of scanning and image selection is ended. The provisional image set is included in the image set as, for example, a series of images corresponding to the scan range. Predetermined subject information, such as subject ID, official ID, name, age, and sex, is input separately to the slit lamp microscope system 1. The image set creation unit 82 can create an image set by configuring such subject information, subject eye information (information indicating left eye/right eye, etc.), shooting date and time, shooting conditions, etc. as incidental information of a series of images. The image set may also include other images obtained by the slit lamp microscope system 1, images obtained by other modalities, inspection data acquired by an inspection device, etc. This concludes the explanation of the first example of the execution mode of scanning and image selection.

Next, a second example of the scanning and image selection mode executed after the image group evaluation unit 81 evaluates that the quality of the image group obtained in the first scan is not good will be described. This example is configured to perform two or more scans together and then select a series of images from the two or more image groups collected thereby. Images are then selected from the two or more image groups collected in these two or more scans and the image group obtained in the first scan performed before that, and an image set including the selected series of images is created.

A specific example of such processing will be described. The selection unit 821 first associates a group of images corresponding to each scan with multiple positions (described above) in the scan range. As a result, two or more images corresponding to different scans are assigned to each of the multiple positions in the scan range.

Next, for each of the multiple positions in the scan range, the selection unit 821 selects one image from the two or more images assigned to that position. The image selection conditions applied in this example may be, for example, the blink condition and eye movement condition described above. As a result, one image is assigned to each of the multiple positions in the scan range. The multiple images associated with the multiple positions in this way are adopted as a series of images included in the image set. This concludes the explanation of the second example of the execution mode of scanning and image selection.

A fourth example of the image group processing unit 80 will be described. As with the second and third examples, the effectiveness of this example is particularly evident when two or more scans are applied to the subject eye E. The image group processing unit 80D shown in FIG. 6D includes an image group evaluation unit 81 similar to that in FIG. 6A, an image set creation unit 82 similar to that in FIG. 6B or FIG. 6C, and an image set evaluation unit 83.

The image set evaluation unit 83 evaluates the quality of the image set created by the image set creation unit 82. This evaluation is to determine whether the image set has sufficient quality to perform effective diagnosis (image interpretation), and the evaluation items and evaluation criteria are determined from this perspective. The evaluation items and evaluation criteria may be common to at least a portion of the image group evaluation conditions and/or at least a portion of the image selection conditions, but are not limited to them.

The image set evaluation unit 83 may be configured to perform different evaluations depending on the aspect of the image set. For example, the evaluation applied to an image set created in response to the number of images included in the provisional image set reaching a predetermined number may be different from the evaluation applied to an image set created in response to the number of alternating repetitions of scanning and image selection reaching a predetermined number. Note that the same evaluation may be applied regardless of the aspect of the image set.

Examples of quality assessment of an image set include assessment of the quality of each image (which may be similar to the assessment of a group of images), as well as assessment of the "arrangement order" of a series of images, assessment of "image skips (missing images)", and assessment of "misalignment". Defects in an image set, such as rearrangement of the arrangement order, skipping of images, and misalignment, arise due to eye movement, fixation misalignment, etc.

The evaluation of the arrangement order of a series of images will now be described. In some examples, the aforementioned correspondence (one-to-one correspondence) is set between the series of images and multiple positions in the scan range. The image set evaluation unit 83 can use this correspondence to evaluate the arrangement order.

Here, the multiple positions in the scan range are ordered according to their positional relationships in real space. An example will be described. As described above, the scan range along the X direction is divided into (N-1) sections, and N positions B1, B2, B3, ..., BN are set in order from the scan start position toward the scan end position. In other words, the N positions B1 to BN are ordered according to their positional relationships in real space. Also, N images F1 to FN (a series of images) are associated with the N positions B1 to BN in a one-to-one relationship.

Under these assumptions, the image set evaluation unit 83 arranges the N images F1 to FN, for example, according to the arrangement order (relative positional relationship) of the N positions B1 to BN. This process is realized, for example, by setting the coordinates of the N positions B1 to BN in a certain three-dimensional coordinate system, and arranging (embedding) the N images F1 to FN according to the set N coordinates. More specifically, the image set evaluation unit 83 can, for example, extract slit light irradiation areas A1 to AN (two-dimensional cross-sectional images) from each of the N images F1 to FN, set the coordinates of the N positions B1 to BN in the three-dimensional coordinate system, and embed the N two-dimensional cross-sectional images A1 to AN according to the set N coordinates.

The image set evaluation unit 83 can evaluate whether the arrangement order is appropriate by analyzing the images F1-FN (two-dimensional cross-sectional images A1-AN) embedded in the three-dimensional coordinate system. For example, the image set evaluation unit 83 can detect areas of interest (image areas corresponding to areas of interest such as the anterior cornea, posterior cornea, iris, pupil, anterior crystalline lens, posterior crystalline lens, and angle) from the images F1-FN (two-dimensional cross-sectional images A1-AN), and perform an evaluation based on the shape (connectivity, continuity, etc.) of the areas of interest in the arrangement direction (X direction in this example) of the images F1-FN (two-dimensional cross-sectional images A1-AN). For example, if a gap of a predetermined dimension or more exists in the area of interest, it is determined that the arrangement order is inappropriate (there is a change in the arrangement order).

In another example, the image set evaluation unit 83 constructs a cross-sectional image along the X direction from images F1 to FN (2D cross-sectional images A1 to AN) embedded in a 3D coordinate system. Furthermore, the image set evaluation unit 83 can perform evaluation based on the form of this cross-sectional image (connectivity, continuity, etc.).

Evaluation of image skipping and misalignment can also be performed in the same way as evaluation of the array order.

As with the image group evaluation unit 81 and the selection unit 821, the image set evaluation unit 83 may include an artificial intelligence engine to evaluate whether the input image set has sufficient quality for effective diagnosis.

When the image set evaluation unit 83 evaluates the quality of the image set to be good, the control unit 7 may be configured to perform control to cause the communication unit 9 to transmit this image set. For example, the control unit 7 prepares transmission information including this image set, and controls the communication unit 9 to transmit this transmission information to a specified external device.

The manner in which an image set or the like is output from the slit lamp microscope system 1 is not limited to transmission. Examples of output manners other than transmission include saving in a storage device (such as a database), recording on a recording medium, and printing on a print medium.

If the image set evaluation unit 83 evaluates the quality of the image set as not good, the control unit 7 may be configured to control a new scan of the subject's eye E, that is, control the acquisition of a new image group (new image set). For example, the control unit 7 may be configured to display and/or output predetermined output information as sound. The predetermined output information has content to prompt the user to retake the image, such as, for example, that the image capture has failed or that retaking the image is necessary.

Alternatively, the control unit 7 may be configured to send a command to at least the scanning unit (illumination system 2, photographing system 3, movement mechanism 6) and the image set creation unit 82 to automatically start re-photographing (re-executing scanning and image set creation) when the image set evaluation unit 83 evaluates that the quality of the image set is not good.

Furthermore, several examples of the image group evaluation unit 81 will be described with reference to Figures 7A to 7D. Note that the image group evaluation unit 81 is not limited to these examples, and any modification (addition, substitution, omission, etc.) is possible. Also, two or more of these examples and modifications can be at least partially combined.

A first example of the image group evaluation unit 81 will be described with reference to Figures 7A and 7B. This example uses artificial intelligence technology to evaluate the quality of an image group. The image group evaluation unit 81A shown in Figure 7A includes an inference unit 811A that uses an inference model 812A to evaluate the quality of an image group.

The inference model 812A is constructed in advance by machine learning using training data including multiple anterior segment images. The device that constructs the inference model 812A (inference model construction device) may be provided in the slit lamp microscope system 1 (such as the data processing unit 8) or its peripheral device (such as a computer), or may be another computer.

The model construction unit 90 shown in FIG. 7B is an example of such an inference model construction device, and is provided in the slit lamp microscope system 1 or its peripheral devices. The model construction unit 90 includes a learning processing unit 91 and a neural network 92.

The neural network 92 typically includes a convolutional neural network (CNN). An example of the structure of a convolutional neural network is shown in FIG. 7B.

An image is input to the input layer. Behind the input layer, multiple pairs of convolutional layers and pooling layers are arranged. In the example shown in Figure 7B, three pairs of convolutional layers and pooling layers are provided, but the number of pairs is arbitrary.

In the convolution layer, a convolution operation is performed to grasp features (such as contours) from an image. The convolution operation is a multiplication and accumulation operation of a filter function (weighting factor, filter kernel) of the same dimension as the input image for the input image. In the convolution layer, the convolution operation is applied to each of multiple parts of the input image. More specifically, in the convolution layer, the value of each pixel of the partial image to which the filter function is applied is multiplied by the value (weight) of the filter function corresponding to that pixel to calculate the product, and the sum of the products is calculated over the multiple pixels of this partial image. The product-sum value obtained in this way is substituted for the corresponding pixel in the output image. By performing the product-sum operation while moving the part (partial image) to which the filter function is applied, the convolution operation result for the entire input image is obtained. According to this convolution operation, many images in which various features are extracted using many weighting factors are obtained. In other words, many filtered images such as smoothed images and edge images are obtained. The many images generated by the convolution layer are called feature maps.

In the pooling layer, the feature map generated by the immediately preceding convolutional layer is compressed (data thinning, etc.). More specifically, in the pooling layer, statistics of a specified number of neighboring pixels of a pixel of interest in the feature map are calculated at a specified pixel interval, and an image with dimensions smaller than the input feature map is output. The statistics applied to the pooling operation are, for example, the maximum value (max pooling) or the average value (average pooling). The pixel interval applied to the pooling operation is called the stride.

A convolutional neural network can extract many features from an input image by processing it using multiple pairs of convolutional layers and pooling layers.

After the last pair of convolutional and pooling layers, a fully connected layer is provided. In the example shown in FIG. 7B, two fully connected layers are provided, but the number of fully connected layers is arbitrary. In the fully connected layer, image classification, image segmentation, regression, and other processes are performed using features compressed by a combination of convolution and pooling. After the last fully connected layer, an output layer is provided to provide the output results.

In some exemplary embodiments, the convolutional neural network may not include a fully connected layer (e.g., a fully-layered convolutional network (FCN)), and may include a support vector machine, a recurrent neural network (RNN), or the like. The machine learning for the neural network 92 may be transfer learning. That is, the neural network 92 may include a neural network that has already been trained using other training data (training images) and has had its parameters adjusted. The model construction unit 90 (learning processing unit 91) may be configured to be capable of applying fine tuning to the trained neural network (92). The neural network 92 may be constructed using, for example, a known open source neural network architecture.

The learning processing unit 91 applies machine learning using training data to the neural network 92. When the neural network 92 includes a convolutional neural network, the parameters adjusted by the learning processing unit 91 include, for example, the filter coefficients of the convolutional layer and the connection weights and offsets of the fully connected layer.

As described above, the training data includes at least a plurality of anterior segment images. The plurality of anterior segment images are typically images acquired by a slit lamp microscope, but are not limited thereto, and may include, for example, images acquired by other ophthalmic modalities (fundus cameras, OCT devices, SLOs, surgical microscopes, etc.), images acquired by imaging diagnostic modalities of other medical departments (ultrasound diagnostic devices, X-ray diagnostic devices, X-ray CT devices, magnetic resonance imaging (MRI) devices, etc.), images generated by processing actual images, pseudo images, etc. Furthermore, the number of training data may be increased using techniques such as data expansion and data augmentation.

The training method for constructing an inference model may be any method, but may be, for example, supervised learning, unsupervised learning, or reinforcement learning, or a combination of two or more of these.

In some exemplary aspects, supervised learning is performed using training data in which input images are labeled with a final output label. For example, each of the multiple anterior segment images included in the training data is pre-labeled to indicate whether it is readable or unreadable. The labels are generated, for example, by a doctor or another inference model. The learning processing unit 91 can construct the inference model 812A by applying supervised learning using such training data to the neural network 92.

The inference model 812A of this example constructed in this manner is a trained model that takes as input the images obtained by scanning the anterior segment with slit light, and outputs the possibility of interpretation. Note that the possibility of interpretation, which is the output of the inference model 812A, may be any parameter indicating whether the image group to be evaluated is suitable for interpretation, and may be, for example, a determination of whether interpretation is possible or not, the probability that interpretation can be performed, the validity of the results when interpretation is performed (e.g., accuracy, precision), etc.

To prevent processing from concentrating on specific units of the neural network 92, the learning processing unit 91 may randomly select and disable some units and perform learning using the remaining units (dropout).

The methods used to build the inference model are not limited to the examples shown here. For example, any method such as a support vector machine, a Bayesian classifier, boosting, k-means, kernel density estimation, principal component analysis, independent component analysis, self-organizing map, random forest, or generative adversarial network (GAN) can be used to build the inference model.

The inference unit 811A shown in FIG. 7A uses such an inference model 812A to evaluate the quality of a group of images collected in one scan of the anterior segment of the subject's eye E. More specifically, the inference unit 811A first inputs the image group or each image contained therein to the inference model 812A. The inference model 812A derives information indicating the possibility of interpretation from the input image group or images.

The inference unit 811A may use the reading possibility information output from the inference model 812A as the inference result as is, or may generate an inference result based on the reading possibility information. As an example of the latter, the inference unit 811A may generate information for display from the reading possibility information, or may execute a predetermined statistical processing.

A second example of the image group evaluation unit 81 will be described with reference to FIG. 7C. In this example, an image group is converted into a three-dimensional image and quality evaluation is performed. The image group evaluation unit 81B shown in FIG. 7C includes a three-dimensional image construction unit 811B, a comparison unit 812B, and an evaluation processing unit 813B.

The three-dimensional image construction unit 811B constructs a three-dimensional image from a group of images acquired in a single scan of the anterior segment of the subject's eye E. For example, the three-dimensional image construction unit 811B may be configured to construct stack data by embedding the group of images in a single three-dimensional coordinate system. This stack data is constructed, for example, by embedding the N two-dimensional cross-sectional images A1 to AN shown in FIG. 5 in the three-dimensional coordinate system according to the arrangement (relative positional relationship) of the N positions B1 to BN described above.

The three-dimensional image construction unit 811B may be configured to construct volume data by applying a voxelization process to stack data constructed from the image group. Furthermore, the three-dimensional image construction unit 811B can apply a predetermined rendering to the stack data or the volume data. Examples of rendering include volume rendering and surface rendering.

The comparison unit 812B compares the 3D image (stack data, volume data, rendering image, etc.) constructed by the 3D image construction unit 811B with a predetermined reference 3D image. Any number of reference 3D images, one or more, are prepared.

The reference three-dimensional image may include one or more reference three-dimensional images corresponding to a normal eye (an eye in which no disease has been found (a healthy eye)). The reference three-dimensional image corresponding to a normal eye may be, for example, an image acquired by photographing a normal eye. The imaging modality for acquiring this reference three-dimensional image may be any, but is typically the slit lamp microscope system 1 or a similar slit lamp microscope. The reference three-dimensional image corresponding to a normal eye may also be an image acquired by photographing a model of a normal eye (such as a model eye), or an image created by computer graphics from a model of a normal eye or a clinical case.

The reference three-dimensional image may include one or more reference three-dimensional images corresponding to the diseased eye. The reference three-dimensional image corresponding to the diseased eye may be, for example, an image obtained by photographing an eye that has been definitively diagnosed with a specific disease. The imaging modality for obtaining this reference three-dimensional image may be any, but is typically the slit lamp microscope system 1 or a similar slit lamp microscope. The reference three-dimensional image corresponding to the diseased eye may also be an image obtained by photographing a model of the diseased eye (such as a model eye), or an image created by computer graphics from a model or clinical case of the diseased eye.

The comparison unit 812B performs matching between the 3D image constructed by the 3D image construction unit 811B and a reference 3D image to calculate the value of a predetermined parameter. This image matching may use any method, such as image correlation, feature-based matching, region-based matching, or machine learning (trained model). The calculated parameters may be any parameters, such as correlation values, matching parameters (angle, scale, similarity, compatibility, etc.), and output parameters of a trained model.

Such image matching is typically a process that quantitatively (i.e., numerically) expresses the degree to which the shape and structure of tissues and areas (cornea, iris, pupil, angle, etc.) depicted in a 3D image resembles the shape and structure of a standard normal eye and/or resembles the shape and structure of a standard diseased eye.

The evaluation processing unit 813B evaluates the quality of the image group based on the parameter values calculated by the comparison unit 812B. For example, the evaluation processing unit 813B may be configured to evaluate the quality of the image group by comparing the parameter values calculated by the comparison unit 812B with a predetermined threshold value. Alternatively, the evaluation processing unit 813B may be configured to evaluate the quality of the image group by determining whether the parameter values calculated by the comparison unit 812B are within a predetermined range. Note that the processing methods performed by the evaluation processing unit 813B are not limited to these, and may be any method that can be used to derive an evaluation result from the value of a certain parameter.

The quality evaluation of an image group using three-dimensional image construction is not limited to this example. For example, the image group evaluation unit 81 may be configured to be capable of performing one or more evaluation processes, in the same manner as the image set evaluation unit 83, of evaluating the arrangement order of a series of images constituting an image group, evaluating image skipping, and evaluating misalignment.

A third example of the image group evaluation unit 81 will be described with reference to FIG. 7D. In this example, the quality of an image group is evaluated by quantitatively evaluating the image quality of the image group. The image group evaluation unit 81C shown in FIG. 7D includes an evaluation data generation unit 811C and an evaluation processing unit 812C.

The evaluation data generating unit 811C obtains image quality evaluation data from images included in an image group collected in a single scan of the anterior segment of the subject's eye E. The image quality evaluation data quantitatively represents the image quality of the image group.

Several examples of image quality evaluation processing performed by the evaluation data generation unit 811C will be described. In some exemplary aspects, the image quality evaluation processing performed by the evaluation data generation unit 811C may be any processing, such as signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), root-mean-square (RMS) granularity, Wiener spectrum, modulation transfer function (MTF), quality index (QI), or other processing using any known technology.

For example, the evaluation data generation unit 811C calculates a value of a predetermined image quality evaluation index (image quality evaluation value) as image quality evaluation data for the image group. The image quality evaluation value may be any parameter that quantitatively expresses the quality of the image, and typically, the higher the image quality, the higher the image quality evaluation value.

As an example of a method for calculating an image quality evaluation value, a method for calculating an Image Quality value (IQ value) used in evaluating the image quality of an OCT image will be described below. First, the evaluation data generating unit 811C detects an image area (tissue image area) corresponding to the tissue (site) of the anterior eye and other image areas (background area, non-tissue image area) by applying a predetermined analysis process (e.g., segmentation) to the evaluation area set in the image to be evaluated. Next, the evaluation data generating unit 811C generates a histogram of brightness in the tissue image area and generates a histogram of brightness in the background area. Next, the evaluation data generating unit 811C calculates an image quality evaluation value (IQ value) from the degree of overlap of these two histograms. For example, the IQ value is defined in the range [0, 100] so that when both histograms completely overlap, the IQ value is 0, and when both histograms are completely separated, the IQ value is 100. This image quality evaluation calculation may include, for example, normalizing two histograms, generating a probability distribution function, and calculating an IQ value using a specified formula.

In this way, the evaluation data generating unit 811C may be configured to perform a process of identifying a tissue image region corresponding to the tissue of the anterior eye and a background region in the image to be evaluated, a process of creating a first histogram showing the frequency distribution of luminance in the tissue image region, a process of creating a second histogram showing the frequency distribution of luminance in the background region, and a process of calculating an image quality evaluation value (IQ value) as image quality evaluation data based on the first histogram and the second histogram.

The evaluation processing unit 812C evaluates the quality of the image group based on the image quality evaluation data generated by the evaluation data generation unit 811C. Several methods for quality evaluation based on image quality evaluation data are described below, but the quality evaluation method is not limited to these and may be any method.

A first example of quality evaluation based on image quality evaluation data will be described. When an IQ value is obtained for each image constituting an image group, the evaluation processing unit 812C compares each of the multiple IQ values obtained for this image group with a predetermined threshold. If all of the multiple IQ values are equal to or greater than the threshold, the evaluation processing unit 812C determines that the quality of this image group is good. On the other hand, if any of the multiple IQ values is less than the threshold, the evaluation processing unit 812C determines that the quality of this image group is not good.

A second example of quality evaluation based on image quality evaluation data will be described. When an IQ value is obtained for each image constituting an image group, the evaluation processing unit 812C selects the lowest IQ value from the multiple IQ values obtained for this image group, and compares this minimum IQ value with a predetermined threshold. If the minimum IQ value is equal to or greater than the threshold, the evaluation processing unit 812C determines that the quality of this image group is good. On the other hand, if the minimum IQ value is less than the threshold, the evaluation processing unit 812C determines that the quality of this image group is not good.

A third example of quality evaluation based on image quality evaluation data will be described. When an IQ value is obtained for each image constituting an image group, the evaluation processing unit 812C calculates a statistical value by applying a predetermined statistical operation to the multiple IQ values obtained for this image group. The type of statistical value may be any, and may be, for example, an average value, a minimum value, a maximum value, a mode value, a median value, etc. Note that the case where the statistical value is a minimum value corresponds to the second example above. The evaluation processing unit 812C compares the calculated statistical value with a predetermined threshold value. If the statistical value is equal to or greater than the threshold value, the evaluation processing unit 812C determines that the quality of this image group is good. On the other hand, if the statistical value is less than the threshold value, the evaluation processing unit 812C determines that the quality of this image group is not good.

The processing performed by the evaluation processing unit 812C is not limited to processing based on the image quality evaluation data generated by the evaluation data generation unit 811C. For example, the evaluation processing unit 812C may be configured to be capable of performing one or more of the following evaluation processes: evaluation of the arrangement order of a series of images constituting an image group, evaluation of image skipping, and evaluation of misalignment (all of which have been described above).

<Processed Image Construction Unit 85>
The processed image constructing unit 85 constructs a processed image from a group of images collected by scanning the anterior segment using a slit light. At least one image included in the group of images is used for constructing the processed image. The processed image construction may include any image processing. Some examples are described below, but the processing included in the processed image construction is not limited to these.

In some exemplary embodiments, the processed image construction unit 85 may be capable of performing three-dimensional image construction to construct a three-dimensional image from a group of images. The three-dimensional image constructed from the group of images may be, for example, stack data or volume data. The processed image construction unit 85 in this example may have a configuration similar to that of the above-mentioned three-dimensional image construction unit 811B. For example, the combination of hardware and software constituting the processed image construction unit 85 may be at least partially common to the combination of hardware and software constituting the three-dimensional image construction unit 811B.

In some exemplary embodiments, the processed image construction unit 85 may be capable of executing a rendering process that constructs a rendering image of a three-dimensional image. The processed image construction unit 85 may apply a predetermined rendering process to, for example, stack data or volume data obtained by constructing a three-dimensional image. Examples of the rendering process include volume rendering and surface rendering. The processed image construction unit 85 in this example may also have a similar configuration to the three-dimensional image construction unit 811B.

In some exemplary embodiments, the processed image construction unit 85 may be capable of performing processes such as identifying a partial region of an image (segmentation) and extracting the identified partial region (cropping, trimming). For example, the processed image construction unit 85 may be capable of extracting a region of interest corresponding to a specific part of the subject's eye E from an image included in an image group, a three-dimensional image, or a rendering image.

<Partial area specifying unit 86>
The partial region specifying unit 86 analyzes the image of the subject's eye E acquired by the slit lamp microscope system 1, and specifies a partial region corresponding to a predetermined part of the anterior eye. Reference numeral 71 denotes the anterior eye portion, which is input information for selecting the display conditions (display color, display gradation, etc.) from the display condition information 100 .

An example of a process performed by the partial region identification unit 86 is segmentation. Examples of segmentation include semantic segmentation using machine learning, segmentation using a non-machine learning algorithm such as edge detection, and a combination of these. The partial region identification unit 86 may be configured to adopt a region specified by the user on the image as a partial region corresponding to a specified part of the anterior eye, or may be configured to determine a partial region corresponding to a specified part of the anterior eye based on a region specified by the user on the image. These are merely examples of processes performed by the partial region identification unit 86, and other processes and methods may be used.

In some exemplary embodiments, the specified area is specified in advance or at the time of examination by the slit lamp microscope system 1, another device or system, or a user. In one embodiment, the specified area is specified by analyzing an image of the anterior segment of the subject's eye E. In another embodiment, the specified area is specified manually by referring to an image of the anterior segment of the subject's eye E. In yet another embodiment, a previously applied area is specified again as the specified area by referring to the subject's medical information (electronic medical record, image interpretation report, etc.). The area of the anterior segment specified in this manner (i.e., the area of the anterior segment that is the target of identification by the partial region identification unit 86) may be referred to as the target area.

The target area may be, for example, the cornea, iris, lens, angle, anterior chamber, floating matter in aqueous humor, area of interest, artificial object, etc. In a typical embodiment, the target area is any area for which display conditions are defined in the display condition information 100. Note that an area for which display conditions are not defined may be identified by the partial area identification unit 86. In this case, the display condition selection unit 71 can select, for example, display conditions for other areas located in the vicinity of the identified area from the display condition information 100. The number of target areas may be any number equal to or greater than 1.

The display condition selection unit 71 selects a display condition corresponding to the target site from the display condition information 100. Furthermore, the display control unit 72 causes the display device 11 to display the image of the subject's eye E acquired by the slit lamp microscope system 1 based on the display condition selected by the display condition selection unit 71. The display control unit 72 applies at least the display color and display gradation included in the display condition selected by the display condition selection unit 71 to the partial region to display the image. When a gradation image (which may be either a color image or a grayscale image) is acquired by the slit lamp microscope system 1 and the display condition includes a display color code, the display control unit 72 can convert the gradation value of the pixel of the partial region (image region corresponding to the target site) identified from this gradation image by the partial region identification unit 86 based on the display color code corresponding to the target site. As a result, the part of the gradation image corresponding to the target site is displayed with a gradation value corresponding to the target site, improving the display quality of the target site and making it possible to perform suitable observation.

Some examples of the processing performed by the partial region identification unit 86 will be described. The partial region identification unit 86 performs processing to divide an image of the anterior segment of the subject's eye E into two or more partial regions. Note that "dividing an image" includes at least classifying an image into a plurality of partial regions, that is, dividing a set of a plurality of pixels constituting the image into a plurality of subsets (or assigning a label corresponding to one of the partial regions to each pixel constituting the image). Note that the plurality of pixels that are the subject of the classification processing may be all of the pixels constituting the image, or may be a portion of the pixels. Furthermore, the image division processing may include processing to extract the partial regions obtained by the classification processing from the original image, that is, processing to extract the partial regions. In this case, for example, in a subsequent display processing, a display image is constructed by recombining the extracted partial regions.

The partial region identification unit 86 identifies image regions corresponding to one or more target regions of the anterior segment by analyzing an image of the anterior segment of the subject's eye E. The target regions are not limited to anatomical tissues or diseased areas, and may be a concept that includes regions obtained by analyzing an image of the anterior segment, regions that are manually set by referring to an image of the subject's eye, etc.

A case will be described in which the display condition information 100 includes display conditions related to one or more regions of interest, such as the display condition information 100d in FIG. 3. The region of interest is a location that is focused on in diagnosis, and may be, for example, a lesion, an abnormality, a region to be diagnosed, or a region to be treated. The partial region specifying unit 86 can analyze the image of the anterior segment acquired by the slit lamp microscope system 1 to specify a region of interest corresponding to a predetermined region of interest. For example, the partial region specifying unit 86 can specify the region of interest by applying image analysis processing such as segmentation, image matching, luminance analysis, and color analysis to the image of the anterior segment. The display condition selecting unit 71 can select a display condition corresponding to this region of interest from the display condition information 100d. The display control unit 72 can apply the display condition (display color "C ₉ ", display gradation "A ₉ ") selected by the display condition selecting unit 71 to the region of interest specified by the partial region specifying unit 86, and can display the image of the anterior segment. This improves the display quality of the region of interest (image of the region of interest), making it possible to preferably observe the region of interest.

A case will be described in which the display condition information 100 includes a display condition related to the background, as in the display condition information 100d of FIG. 3. As described above, the background region may be, for example, the space (air) outside the eye. The partial region specifying unit 86 can analyze the image of the anterior segment acquired by the slit lamp microscope system 1 to specify an image region corresponding to the background (background image region). For example, the partial region specifying unit 86 can specify the background image region by applying image analysis processing such as segmentation, image matching, luminance analysis, and color analysis to the image of the anterior segment. The display condition selecting unit 71 can select a display condition corresponding to the background from the display condition information 100d. The display control unit 72 can apply the display condition (display color "C ₁₀ ", display gradation "A ₁₀ ") selected by the display condition selecting unit 71 to the background image region specified by the partial region specifying unit 86, and display this image of the anterior segment. This improves the display quality of the background image region, making it possible to preferably observe not only the background but also the region other than the background. For example, it becomes easier to grasp the boundary between the background and the non-background.

A case will be described in which the display condition information 100 includes display conditions related to an artifact, such as the display condition information 100d in FIG. 3. As described above, the artifact may be an IOL, a phakic IOL, an ICL, a MIGS device, a stent, or the like. The partial region specifying unit 86 can analyze the image of the anterior segment acquired by the slit lamp microscope system 1 to specify an image region corresponding to the artifact (artificial object image region). For example, the partial region specifying unit 86 can specify the artificial object image region by applying image analysis processing such as segmentation, image matching, luminance analysis, and color analysis to the image of the anterior segment. The display condition selecting unit 71 can select a display condition corresponding to the artificial object from the display condition information 100d. When the display condition information includes display conditions related to two or more artifacts, such as the display condition information 100d, the slit lamp microscope system 1 (for example, the partial region specifying unit 86 or the display condition selecting unit 71) can identify the type of the artifact specified as the artificial object image region by the partial region specifying unit 86, and select a display condition corresponding to the identified type of artifact by the display condition selecting unit 71. The process of identifying the type of artifact can be performed using, for example, semantic segmentation. The display control unit 72 can apply the display conditions (e.g., display color "C ₇ " and display gradation "A ₇ ") selected by the display condition selection unit 71 to the artifact image region identified by the partial region identification unit 86, and display the image of the anterior eye segment. This improves the display quality of the artifact image region, making it possible to preferably observe the artifact.

The partial area identification unit 86 may analyze at least one image included in the image group described above to identify a partial area corresponding to the target area. For example, the partial area identification unit 86 may identify a partial area from any number of images included in the image group. The display control unit 72 can display at least one image identified from at least one image included in the image group by applying display conditions corresponding to the target area. In other words, the display control unit 72 can display each image in which a partial area is identified such that the partial area is displayed under conditions corresponding to the target area. When such processing is performed on two or more images included in the image group, it is possible to display the target areas depicted in these images under the same conditions, so that the user can preferably compare the target areas between images.

The partial area identification unit 86 may analyze the processed image constructed by the processed image construction unit 85 to identify a partial area corresponding to the target area. For example, the partial area identification unit 86 may be able to identify a partial area corresponding to the target area from a three-dimensional image, or to identify a partial area corresponding to the target area from a rendering image. The display control unit 72 can apply display conditions corresponding to the target area to the partial area identified from the processed image and display the processed image. This improves the degree of freedom in observing the target area. For example, it becomes possible to observe the target area three-dimensionally, observe the target area from a desired direction, observe a desired cross section of the target area, and observe a desired part of the target area in detail.

<Communication Unit 9>
The communication unit 9 performs data communication between the slit lamp microscope system 1 and other devices (external devices). That is, the communication unit 9 transmits data to other devices and receives data transmitted from other devices.

For example, the communication unit 9 can transmit at least a part of the image group collected by scanning the anterior segment of the subject's eye E with slit light to another device (first external device). The communication unit 9 can also transmit data generated from the image group by the data processing unit 8 to another device (second external device). The first external device and the second external device may be the same device or different devices. When the first external device and the second external device are the same device, the external device may be, for example, any of an image reading terminal, an image reading device, and a computer that executes a process different from that of the data processing unit 8. On the other hand, when the first external device and the second external device are different devices, the first external device may be, for example, any of an image reading terminal, an image reading device, a computer that executes a process similar to that of the data processing unit 8, and a computer that executes a process different from that of the data processing unit 8, and the second external device may be any of an image reading terminal, an image reading device, and a computer that executes a process different from that of the data processing unit 8. When the first external device and the second external device are different devices, the control unit 7 may be configured to select the external device to be the destination according to the type of data to be transmitted.

The data communication method implemented by the communication unit 9 is arbitrary. For example, the communication unit 9 includes one or more of various communication interfaces, such as a communication interface compliant with the Internet, a communication interface compliant with a dedicated line, a communication interface compliant with a LAN, and a communication interface compliant with short-range communication. The data communication may be wired communication or wireless communication.

The data transmitted by the communication unit 9 and/or the data received may be encrypted. When encryption is used, for example, the control unit 7 and/or the data processing unit 8 includes at least one of an encryption processing unit that encrypts the data transmitted by the communication unit 9 and a decryption processing unit that decrypts the data received by the communication unit 9.

<Other elements>
1A and 1B, the slit lamp microscope system 1 may include a user interface. An example of such a configuration is shown in FIG. 8. The configuration of this example can be applied instead of the configuration of FIG. 1B. The slit lamp microscope system 1 of this example includes a user interface (UI) 12. The user interface 12 includes a display device 121 and an operation device 122.

The display device 121 operates under the control of the display control unit 72. The display device 121 may be a flat panel display such as an LCD. The operation device 122 includes a device for operating the slit lamp microscope system 1 and a device for inputting information. The operation device 122 includes, for example, a button, a switch, a lever, a dial, a handle, a knob, a mouse, a keyboard, a trackball, an operation panel, and the like. The user interface 12 may include a device having both a display function and an operation function, such as a touch screen.

The user interface 12 can be used, for example, to specify and/or change a target area, specify and/or change a partial area of an image, specify and/or change display conditions, generate and/or edit display condition information 100, etc.

In some exemplary aspects, the display control unit 72 can display an image acquired by the slit lamp microscope system 1 on the display device 121. The user can use the operation device 122 to perform an operation to specify a partial area of the displayed image. The user can also perform an operation to change the display conditions corresponding to the specified partial area. The display control unit 72 can apply the display conditions specified by the user to the partial area specified by the user and display the image on the display device 121. This makes it possible to display a desired part of the image under desired conditions.

As described above, the slit lamp microscope system 1 is capable of acquiring multiple anterior segment images of the subject's eye E (such as an image group, a part of an image group, an image set, or a part of an image set). The user can specify a partial region for one of these anterior segment images (or two or more images). An image for which a partial region has been specified by the user may be referred to as a specified image.

The partial region identification unit 86 can identify partial regions of one or more images other than the specified image among the multiple anterior eye images based on the partial region of the specified image. This partial region identification process is performed, for example, by comparing the specified image with an image adjacent to it. Alternatively, this partial region identification process is performed by identifying image regions of the other images that correspond to the partial region of the specified image by image correlation or image matching between the specified image and the other images. This allows the partial regions of the other images to be automatically set by simply manually identifying the partial region of one (or more) images, thereby reducing the time and effort required for the work.

Furthermore, the display conditions of the partial area of the specified image can be assigned to the partial area of the other image specified in this way. This allows the display conditions of the other images to be set automatically by simply manually specifying the display conditions for one (or more) images, thereby reducing the time and effort required for the work.

<Alignment>
The following describes alignment of the slit lamp microscope system 1 with respect to the subject's eye E. Generally, alignment is an operation for disposing the device optical system at a suitable position for photographing and measuring the subject's eye E. Alignment in this embodiment is an operation for disposing the illumination system 2 and the photographing system 3 at suitable positions for acquiring a plurality of anterior segment images (such as a series of images, a moving image, an image group, or an image set) as shown in FIG.

There are various methods for aligning ophthalmic devices. Below, some alignment methods are given as examples, but the methods that can be applied to this embodiment are not limited to these.

One alignment method that can be applied to this embodiment is stereo alignment. Stereo alignment can be applied to an ophthalmic device capable of photographing the anterior eye from two or more different directions, and a specific method thereof is disclosed in, for example, Japanese Patent Application Laid-Open Publication No. 2013-248376 filed by the present applicant. Stereo alignment includes, for example, the following steps: a step in which two or more anterior eye cameras photograph the anterior eye from different directions to obtain two or more photographed images; a step in which a processor analyzes these photographed images to determine the three-dimensional position of the subject's eye; and a step in which the processor controls the movement of the optical system based on the determined three-dimensional position. As a result, the optical system (illumination system 2 and photographing system 3 in this example) is positioned in a suitable position relative to the subject's eye. In a typical stereo alignment, the position of the pupil (center or center of gravity of the pupil) of the subject's eye is used as the reference.

In addition to this type of stereo alignment, any known alignment method can be used, such as a method using a Purkinje image obtained by alignment light, a method using an optical lever, or a method using an alignment index. In the method using a Purkinje image, or the method using an optical lever or alignment index, the position of the corneal apex of the test eye is used as the reference.

Note that while typical conventional alignment methods, including the above examples, are performed with the aim of aligning the axis of the subject's eye with the optical axis of the optical system, in this embodiment, it is possible to perform alignment so that the illumination system 2 and the imaging system 3 are positioned at a position corresponding to the scan start position.

As a first example of alignment in this aspect, after performing alignment based on the pupil or corneal apex of the test eye E by applying any of the alignment methods described above, the illumination system 2 and the imaging system 3 can be moved (in the X direction) by a distance equivalent to a preset standard value of the corneal radius, or a distance equivalent to a value obtained by adding a predetermined value to the preset standard value of the corneal radius. Note that instead of using the standard value, a previously obtained measured value of the corneal radius of the test eye E may be used.

As a second example, after performing alignment based on the pupil or corneal apex of the subject's eye E by applying any of the alignment methods described above, an image of the anterior segment of the subject's eye E is analyzed to measure the corneal radius, and the illumination system 2 and the imaging system 3 can be moved (in the X direction) by a distance equivalent to this measurement value or a distance equivalent to the measurement value plus a predetermined value. The image of the anterior segment analyzed in this example is, for example, an anterior segment image obtained by the imaging system 3, or another image. The other image may be any image, such as an image obtained by an anterior segment camera or an anterior segment OCT.

As a third example, an image of the anterior eye obtained by an anterior eye camera or imaging system 3 for stereo alignment can be analyzed to determine the first end of the cornea, and by applying stereo alignment, the illumination system 2 and imaging system 3 can be moved to a position corresponding to this first end, or to a position shifted outward a predetermined distance from this first end.

It is also possible to apply any of the above-mentioned alignment methods to perform alignment based on the pupil or corneal apex of the subject's eye E, and start scanning with the slit light from the position determined thereby. Even in this case, the scan sequence can be set to scan a three-dimensional area including the cornea C (e.g., a three-dimensional area including the entire cornea C). For example, the scan sequence is set to scan to the left from the position determined by alignment based on the pupil or corneal apex, and then scan to the right.

<Image display>
Several examples of displaying images acquired by the slit lamp microscope system 1 will now be described.

In the first example of image display, when multiple anterior segment images are acquired by the slit lamp microscope system 1, the same target site is displayed under the same conditions. In this example, the slit lamp microscope system 1 acquires two or more anterior segment images of the subject's eye. The two or more anterior segment images may be (part of) an image group, (part of) an image set, or other than these. The partial area identification unit 86 identifies a partial area corresponding to the target site from each of these anterior segment images. The display control unit 72 applies a display condition corresponding to the target site to each of the two or more partial areas identified from these anterior segment images. Furthermore, the display control unit 72 causes the display device 11 (display device 121) to display two or more anterior segment images with the same display condition applied to each partial area. The manner in which these anterior segment images are displayed is arbitrary, and may be, for example, parallel display, sequential display, or a combination thereof. Parallel display is a method of displaying at least two of these anterior segment images side by side. Sequential display is a method of switching (replacing) these anterior segment images for display. The trigger for switching images in sequential display is issued automatically or manually. An example of sequential display with automatic issuance is a slide show.

The second example of image display allows comparative observation of images, and is useful for follow-up, etc. In this example, an anterior eye image (first anterior eye image) is stored in the storage unit 10. The first anterior eye image may be any image, for example, an image of the anterior eye of the subject eye E acquired in the past, an image of the anterior eye of another eye, or a simulated image of the anterior eye (an image of a model eye, an image created by computer graphics, etc.). The first anterior eye image may be, for example, an image of a normal eye, or an image of an eye (disease eye) suffering from a specific disease. The first anterior eye image may be an image acquired by the slit lamp microscope system 1, or an image acquired or created by another device or system. The partial area identification unit 86 identifies a partial area (first partial area) of the first anterior eye image corresponding to the target area. In addition, as described above, the partial area identification unit 86 identifies a partial area (second partial area) of the first anterior eye image corresponding to the target area in the anterior eye image (second anterior eye image) acquired (newly) by the slit lamp microscope system 1. The display control unit 72 applies display conditions corresponding to the target site to each of the first partial region and the second partial region. That is, the same display conditions are applied to the first partial region and the second partial region. Furthermore, the display control unit 72 causes the display device 11 (display device 121) to display the first anterior eye image and the second anterior eye image to which the same display conditions are applied to the first partial region and the second partial region. The manner in which the first anterior eye image and the second display image are displayed is arbitrary and may be the same as in the first example. Note that, although the processing related to two anterior eye images has been described in this example, the same processing may be applied to three or more anterior eye images.

<Other matters>
The slit lamp microscope system 1 may include a fixation system that outputs light (fixation light) for fixating the subject's eye E. The fixation system typically includes at least one visible light source (fixation light source) or a display device that displays an image such as a landscape chart or a fixation target. The fixation system is arranged, for example, coaxially or non-coaxially with the illumination system 2 or the imaging system 3. The fixation system may include an internal fixation system that presents a fixation target to the subject through an optical path of the device optical system, and/or an external fixation system that presents a fixation target to the subject from outside the optical path.

The types of images that can be acquired by the slit lamp microscope system 1 are not limited to multiple anterior segment images as shown in FIG. 5. For example, the slit lamp microscope system 1 can acquire three-dimensional images based on multiple anterior segment images, rendering images based on these three-dimensional images, transillumination images, moving images showing the movement of a contact lens worn on the subject's eye, and images showing the gap between the contact lens and the corneal surface due to the application of a fluorescent agent. Rendering images will be described in other aspects. Transillumination images are images obtained by a transillumination method that uses the retinal reflection of illumination light to depict opacities or foreign bodies within the eye. Fundus photography, corneal endothelial cell photography, meibomian gland photography, and the like may also be possible.

The slit lamp microscope system 1 (data processing unit 8) may be capable of performing analysis processing. For example, the slit lamp microscope system 1 may be capable of calculating any anterior segment parameter. The anterior segment parameter is a value that represents the shape of the anterior segment. Examples of the anterior segment parameter include the radius of curvature of the anterior cornea, the radius of curvature of the posterior cornea, the radius of curvature of the anterior lens, the radius of curvature of the posterior lens, the corneal diameter (vertical diameter, horizontal diameter (white-to-white)), the corneal thickness (central thickness, peripheral thickness), the lens thickness, the anterior chamber depth, the anterior chamber volume, the pupil diameter, and the pupil center (eccentricity). The anterior segment parameter may be shape distribution data, and may be, for example, various corneal shape maps such as an axial curvature map, a tangential curvature map, an elevation map, a refractive power map, a thickness map (pachymetry map), and a wavefront aberration map. The anterior segment parameter may also include an angle parameter. Known examples of angle parameters include AOD (angle opening distance), ACA (anterior chamber angle), TISA (trabecular iris space area), ARA (angle recess area), and AtA (angle-to-angle distance). Methods for calculating anterior segment parameters are known. Typically, anterior segment parameter calculation includes a process for identifying the area and/or position to be measured (e.g., segmentation, feature point detection, etc.) and a measurement process for determining the value of the anterior segment parameter based on the identified area and/or position (e.g., distance measurement, ratio calculation, angle calculation, etc.).

<Other Aspects of the Optical System>
A slit lamp microscope system according to some exemplary aspects may have the configuration shown in Fig. 9. The configuration in Fig. 9 is one example of the optical system in Fig. 1A. Note that the slit lamp microscope system of this aspect includes, in addition to the group of elements shown in Fig. 9, for example, the control unit 7, data processing unit 8, communication unit 9, storage unit 10, and user interface 12 described above. Unless otherwise specified, the matters related to the slit lamp microscope system 1 described above can be applied to this aspect.

The illumination system 20 shown in FIG. 9 is an example of the illumination system 2 of the slit lamp microscope system 1, and the left imaging system 30L and the right imaging system 30R are examples of the imaging system 3. Hereinafter, the illumination system 20 may be described as the illumination system 2, and the left imaging system 30L and/or the right imaging system 30R may be described as the imaging system 3. In some embodiments, only one of the left imaging system 30L and the right imaging system 30R may be provided and configured as an example of the imaging system 3. Symbol 20a indicates the optical axis (illumination optical axis) of the illumination system 20, symbol 30La indicates the optical axis (left imaging optical axis) of the left imaging system 30L, and symbol 30Ra indicates the optical axis (right imaging optical axis) of the right imaging system 30R. The left imaging optical axis 30La and the right imaging optical axis 30Ra are arranged in different directions. The angle between the illumination optical axis 20a and the left photographing optical axis 30La is indicated as θL, and the angle between the illumination optical axis 20a and the right photographing optical axis 30Ra is indicated as θR. The angles θL and θR may be equal to each other or may be different. The illumination optical axis 20a, the left photographing optical axis 30La, and the right photographing optical axis 30Ra intersect at one point. As in FIG. 1A, the Z coordinate of this intersection is indicated as Z0.

The moving mechanism 6 may be the same element as the moving mechanism 6 shown in FIG. 1A, and can move the illumination system 20, the left imaging system 30L, and the right imaging system 30R in the direction indicated by the arrow 49 (X direction). Typically, the illumination system 20, the left imaging system 30L, and the right imaging system 30R are placed on a stage that is movable at least in the X direction, and the moving mechanism 6 moves this movable stage in accordance with a control signal from the control unit 7.

The illumination system 20 irradiates slit light onto the anterior segment of the subject's eye E. Similar to the illumination system of a conventional slit lamp microscope, the illumination system 20 includes, in order from the side furthest from the subject's eye E, an illumination light source 21, a positive lens 22, a slit forming section 23, and objective lens groups 24 and 25.

The illumination light (typically visible light) output from the illumination light source 21 is refracted by the positive lens 22 and projected onto the slit forming section 23. A portion of the projected illumination light passes through the slit formed by the slit forming section 23 to become slit light. The generated slit light is refracted by the objective lens groups 24 and 25, and then reflected by the beam splitter 47 and irradiated onto the anterior segment of the subject's eye E.

The left imaging system 30L includes a reflector 31L, an imaging lens 32L, and an image sensor 33L. The reflector 31L and the imaging lens 32L guide light from the anterior segment irradiated with slit light by the illumination system 20 (light traveling in the direction of the left imaging system 30L) to the image sensor 33L.

The light traveling from the anterior segment toward the left imaging system 30L is light from the anterior segment irradiated with the slit light, and is light traveling in a direction away from the illumination optical axis 20a. The reflector 31L reflects the light in a direction approaching the illumination optical axis 20a. The imaging lens 32L refracts the light reflected by the reflector 31L and forms an image on the imaging surface 34L of the imaging element 33L. The imaging element 33L receives the light on the imaging surface 34L.

As with the slit lamp microscope system 1 described above, the left imaging system 30L repeatedly captures images in parallel with the movement of the illumination system 20, the left imaging system 30L, and the right imaging system 30R by the movement mechanism 6. This allows multiple anterior segment images (image group) to be obtained.

Also, similar to the slit lamp microscope system 1 described above, the object plane along the illumination optical axis 20a, the optical system including the reflector 31L and the imaging lens 32L, and the imaging plane 34L satisfy the Scheimpflug condition. More specifically, when considering the deflection of the optical path of the imaging system 30L by the reflector 31L, the YZ plane (including the object plane) passing through the illumination optical axis 20a, the principal surface of the imaging lens 32L, and the imaging plane 34L intersect on the same straight line. This allows the left imaging system 30L to focus on all positions within the object plane (for example, in the range from the anterior surface of the cornea to the posterior surface of the crystalline lens) and perform imaging.

The right photographing system 30R includes a reflector 31R, an imaging lens 32R, and an image sensor 33R. Similar to the left photographing system 30L, the right photographing system 30R guides light from the anterior eye, which is irradiated with slit light by the illumination system 20, to the imaging surface 34R of the image sensor 33R by the reflector 31R and the imaging lens 32R. Furthermore, similar to the left photographing system 30L, the right photographing system 30R obtains multiple anterior eye images (image group) by repeatedly photographing in parallel with the movement of the illumination system 20, the left photographing system 30L, and the right photographing system 30R by the movement mechanism 6. Similar to the left photographing system 30L, the object surface along the illumination optical axis 20a, the optical system including the reflector 31R and the imaging lens 32R, and the imaging surface 34R satisfy the Scheimpflug condition.

The control unit 7 can synchronize the repeated photographing by the left photographing system 30L and the repeated photographing by the right photographing system 30R. This allows a correspondence relationship to be obtained between the multiple anterior eye images obtained by the left photographing system 30L and the multiple anterior eye images obtained by the right photographing system 30R. This correspondence relationship is a temporal correspondence relationship, and more specifically, it is a pairing of images obtained substantially simultaneously.

Alternatively, the control unit 7 or the data processing unit 8 can execute a process to determine the correspondence between multiple anterior segment images obtained by the left imaging system 30L and multiple anterior segment images obtained by the right imaging system 30R. For example, the control unit 7 or the data processing unit 8 can pair the anterior segment images sequentially input from the left imaging system 30L and the anterior segment images sequentially input from the right imaging system 30R based on the input timing of the images.

This embodiment further includes a video capture unit 40. The video capture unit 40 captures video of the anterior segment of the subject's eye E from a fixed position in parallel with the capture by the left and right imaging systems 30L and 30R. "Capturing video from a fixed position" means that the video capture unit 40 does not move when the illumination system 20, the left and right imaging systems 30L and 30R are moved to scan the anterior segment. The video capture unit 40 is also capable of capturing still images.

In this embodiment, the video capture unit 40 is arranged coaxially with the illumination system 20, but the arrangement is not limited to this. For example, the video capture unit can be arranged non-coaxially with the illumination system 20. In addition, an optical system may be provided that illuminates the anterior segment with illumination light in a band to which the video capture unit 40 is sensitive.

The light that passes through the beam splitter 47 is reflected by the reflector 48 and enters the video capture unit 40. The light that enters the video capture unit 40 is refracted by the objective lens 41, and then an image is formed on the imaging surface of the image sensor 43 by the imaging lens 42. The image sensor 43 is an area sensor. The image sensor 43 has sensitivity to, for example, either one or both of the visible light and infrared light bands.

If a video capture unit 40 is provided, it is possible to monitor and track the movement of the subject's eye E. It is also possible to perform alignment using the video capture unit 40.

Depending on the output wavelength of the illumination system 20 and the detection wavelength of the video capture unit 40, the beam splitter 47 is, for example, a dichroic mirror or a half mirror.

Some features, functions, and effects of this slit lamp microscope are described below.

As an example of the imaging system 3 of the slit lamp microscope system 1 described above, this embodiment provides a left imaging system 30L and a right imaging system 30R. The left imaging system 30L includes a reflector 31L and an imaging lens 32L (first optical system) that guide light from the anterior segment of the eye irradiated with slit light, and an imaging element 33L (first imaging element) that receives the guided light on an imaging surface 34L (first imaging surface). Similarly, the right imaging system 30R includes a reflector 31R and an imaging lens 32R (second optical system) that guide light from the anterior segment of the eye irradiated with slit light, and an imaging element 33R (second imaging element) that receives the guided light on an imaging surface 34R (second imaging surface).

The optical axis of the left imaging system 30L (left imaging optical axis 30La) and the optical axis of the right imaging system 30R (right imaging optical axis 30Ra) are arranged in different directions. Furthermore, the object surface along the optical axis of the illumination system 20 (illumination optical axis 20a), the reflector 31L, the imaging lens 32L, and the imaging surface 34L satisfy the Scheimpflug condition. Similarly, the object surface, the reflector 31L, the imaging lens 32L, and the imaging surface 34L satisfy the Scheimpflug condition.

The left imaging system 30L obtains a first group of images by repeatedly capturing images in parallel with the movement of the illumination system 20, the left imaging system 30L, and the right imaging system 30R by the movement mechanism 6. Similarly, the right imaging system 30R obtains a second group of images by repeatedly capturing images in parallel with the movement of the illumination system 20, the left imaging system 30L, and the right imaging system 30R by the movement mechanism 6.

The control unit 7 can control the illumination system 20, the left imaging system 30L, the right imaging system 30R, and the moving mechanism 6, for example, so that the left imaging system 30L and the right imaging system 30R apply one scan to the anterior segment of the subject's eye E in parallel. The data processing unit 8 can select a series of images corresponding to the scan range from the pair of left and right image groups collected in this scan to create one image group. The image group evaluation unit 81 can evaluate the quality of the created image group. If the quality of this image group is evaluated to be good, the control unit 7 can control the communication unit 9 to transmit this image group to a specified external device.

If the quality of this image group is evaluated as not good, the control unit 7 can control the illumination system 20, the left imaging system 30L, the right imaging system 30R, and the moving mechanism 6 to apply a new scan (one or more scans) to the anterior segment of the subject's eye E. By this additional scan, the left imaging system 30L collects one or more first image groups corresponding to one or more scans, and the right imaging system 30R collects one or more second image groups corresponding to one or more scans. The image set creation unit 82 can, for example, select a first series of images corresponding to the scan range from two or more first image groups obtained by the first scan and the additional scan to create a first image set, and select a second series of images corresponding to the scan range from two or more second image groups obtained by the first scan and the additional scan to create a second image set. Alternatively, the image set creation unit 82 may be configured to select a series of images corresponding to the scan range from two or more first image groups and two or more second image groups obtained by the first scan and the additional scan to create an image set. Any aspect of the slit lamp microscope system 1 described above (configuration, control, processing, action, function, etc.) can be combined with this embodiment.

According to this aspect, the anterior segment of the eye irradiated with slit light can be captured in video from different directions. Even if an image acquired by one imaging system contains an artifact, an image acquired by the other imaging system substantially simultaneously with the image may not contain the artifact. Furthermore, even if both of a pair of images acquired substantially simultaneously by both imaging systems contain artifacts and an artifact in one image overlaps with a region of interest (e.g., a region irradiated with slit light), an artifact in the other image may not overlap with the region of interest. This increases the likelihood of acquiring a suitable image. This makes it possible to further improve the probability of acquiring a group and/or set of images of good quality.

The imaging system 3 may include a third imaging system, ..., a Kth imaging system (K is an integer equal to or greater than 3) of a similar configuration in addition to the left imaging system 30L (first imaging system) and the right imaging system 30R (second imaging system). This complicates the structure of the optical system, but it is possible to further improve the probability of obtaining a good quality image group and/or image set. Those who wish to implement this embodiment can design a slit lamp microscope taking into account these trade-off factors (complexity of the optical system and probability of obtaining a high quality image set).

The left imaging system 30L in this embodiment includes a reflector 31L and an imaging lens 32L. The reflector 31L reflects light from the anterior segment irradiated with the slit light, traveling in a direction away from the illumination optical axis 20a, in a direction toward the illumination optical axis 20a. Furthermore, the imaging lens 32L forms an image of the light reflected by the reflector 31L on the imaging surface 34L. Here, the imaging lens 32L includes one or more lenses.

Similarly, the right imaging system 30R includes a reflector 31R and an imaging lens 32R. The reflector 31R reflects light from the anterior segment irradiated with the slit light, traveling in a direction away from the illumination optical axis 20a, in a direction toward the illumination optical axis 20a. Furthermore, the imaging lens 32R forms an image of the light reflected by the reflector 31R on the imaging surface 34R. Here, the imaging lens 32R includes one or more lenses.

This configuration allows the device to be made more compact. That is, the image acquired by the image sensor 33L (33R) is output through a cable extending from the surface opposite the image sensor 34L (34R), but with this configuration, the cable can be arranged from the back surface of the image sensor 33L (33R), which is located relatively close to the illumination optical axis 20a, toward the opposite direction to the subject's eye E. This allows the cable to be routed in an optimal manner, making it possible to make the device more compact.

In addition, with this configuration, since it is possible to set the angles θL and θR large, when an image acquired by one imaging system contains an artifact, it is possible to increase the likelihood that the image acquired by the other imaging system at substantially the same time as the image contains no artifact. Also, when a pair of images acquired substantially simultaneously by both imaging systems contain artifacts, and an artifact in one image overlaps with a region of interest (e.g., a region irradiated with slit light), it is possible to reduce the likelihood that an artifact in the other image overlaps with the region of interest.

This embodiment includes a video capture unit 40. The left and right capture systems 30L and 30R repeatedly capture images of the anterior segment in parallel with the movement of the illumination system 20, the left and right capture systems 30L and 30R by the movement mechanism 6. In parallel with this repeated capture, the video capture unit 40 captures video of the anterior segment from a fixed position.

With this configuration, video recording is performed from a fixed position (e.g., the front) in parallel with scanning the anterior segment with slit light, making it possible to grasp the condition of the subject's eye E during scanning and to perform control according to the condition of the subject's eye E.

For example, when the slit lamp microscope of this embodiment includes the image group evaluation unit 81 (image set evaluation unit 83) of the slit lamp microscope system 1 described above, the slit lamp microscope of this embodiment can evaluate whether the image group (image set) acquired by the left imaging system 30L and/or the right imaging system 30R is of a quality suitable for interpretation. The image group evaluation unit 81 (image set evaluation unit 83) can perform the following operations in combination with the video capture unit 40. Note that similar operations can also be performed using two or more anterior eye cameras for stereo alignment or similar capture means.

The video capture unit 40 captures video of the anterior segment of the subject's eye E from a fixed position in parallel with the application of a scan to the anterior segment of the subject's eye E. This video capture is performed, for example, under the control of the control unit 7. In other words, the control unit 7 can coordinately control the illumination system 2 (illumination system 20), the capture system 3 (left capture system 30L and/or right capture system 30R), the moving mechanism 6, and the video capture unit 40 to scan the anterior segment of the subject's eye E.

In this linked control, the control unit 7 can synchronize the shooting rate of the imaging system 3 with the shooting rate of the video shooting unit 40. For example, the shooting rate of the imaging system 3 and the shooting rate of the video shooting unit 40 are set to be equal, and the shooting timing of the imaging system 3 and the shooting timing of the video shooting unit 40 are matched. This makes it possible to temporally correspond the frame group acquired by the imaging system 3 during scanning with the frame group acquired by the video shooting unit 40.

Even if one or both of the imaging rate and the imaging timing are different, it is possible to temporally correspond the frame group acquired by the imaging system 3 during scanning with the frame group acquired by the video imaging unit 40, for example, by allowing a time difference within a predetermined range.

A pair of frames that are temporally associated (a pair of a frame obtained by the imaging system 3 and a frame obtained by the video imaging unit 40) can be considered to have been acquired substantially simultaneously. Therefore, when considering a pair of frames that are temporally associated, positional deviations due to eye movement can be ignored.

Under such a premise, the image group evaluation unit 81 (image set evaluation unit 83) may be configured to evaluate the quality of a series of images included in the image group (image set) based on the video images (frame group) acquired by the video shooting unit 40.

At this time, the image group evaluation unit 81 (image set evaluation unit 83) may be configured to evaluate the quality of the image group (image set) based on the correspondence between a series of images included in the image group (image set) and a series of frames included in the video captured by the video capture unit 40. In other words, the image group evaluation unit 81 (image set evaluation unit 83) may be configured to evaluate the quality of the image group (image set) based on the temporal correspondence between the frame group obtained by the imaging system 3 and the frame group acquired by the video capture unit 40. Furthermore, the image group evaluation unit 81 (image set evaluation unit 83) may be configured to evaluate the quality of the image group (image set) based on the landmarks in the series of frames and the correspondence.

One specific example will be described. In this example, for a group of frames F1 to FN (described above) obtained by the imaging system 3 and a group of frames D1 to DN obtained in parallel by the video imaging unit 40, frame Fn is associated with frame Dn (n=1, 2, ..., N).

The image group evaluation unit 81 (image set evaluation unit 83) identifies landmarks in each of the frame groups D1 to DN. The landmarks may be, for example, image regions corresponding to the iris (iris regions).

Next, the image group evaluation unit 81 (image set evaluation unit 83) determines the arrangement order (spatial arrangement order) of the frame groups D1 to DN based on the positions (e.g., changes in spatial position) of the N landmarks identified in each of the frame groups D1 to DN.

As mentioned above, frame groups D1 to DN are arranged in this order over time (temporal arrangement order). If the temporal arrangement order and the spatial arrangement order differ, the order may have been changed or the positions may have shifted due to the influence of eye movement. Also, a skipped frame may have occurred due to the influence of a blink.

If such a defect is detected, that is, if the temporal arrangement order and the spatial arrangement order differ, the image group evaluation unit 81 (image set evaluation unit 83) determines that the quality of this image group (image set) is not good.

With this configuration, a moving image that depicts a wide area of the anterior segment (at least in the X and Y directions) can be acquired in parallel with scanning the anterior segment with slit light, and defects in a series of images included in an image group (image set) can be detected using the moving image.

Each of the series of images included in the image group (image set) is an image that extends in the depth direction (Z direction). Therefore, in order to recognize the arrangement and alignment of the series of images in the X and Y directions perpendicular to the Z direction from the series of images themselves, it is necessary to use image processing (image analysis) similar to that of the slit lamp microscope system 1 described above.

In this example, the quality of a series of images is evaluated using a video image acquired separately in parallel with the scan, without applying image analysis to the series of images. Note that the evaluation items in this example may be any of the arrangement order of the series of images, missing images, and misalignment, as in the slit lamp microscope system 1 described above. It is also possible to combine the evaluation process of this example with any of the evaluation processes described for the slit lamp microscope system 1 described above.

Another application of the video capture unit 40 will now be described. In this example, when two or more scans are performed after the quality of the image group collected in the first scan is evaluated as not good, the purpose is to prevent the position of the subject's eye E from shifting between scans by adjusting the start timing of these two or more additional scans.

In this example, the video capture unit 40 also captures an image of the anterior segment of the subject's eye E from a fixed position. The control unit 7 is configured to cause the scanning unit to start a second scan in response to the video capture unit 40 acquiring an image that is substantially the same as the reference image acquired by the video capture unit 40 in response to the start of a first scan of two or more additional scans.

To be more specific, any of the two or more scans applied to the anterior segment to create an image set is called the first scan. In this example, the first scan of the two or more scans is the first scan.

First, the control unit 7 stores an image (reference image) acquired by the video capture unit 40 at the start timing of the first scan. The reference image may be, for example, a frame acquired closest to the start of the first scan among a group of video frames whose capture began before the start of the first scan. Alternatively, the video capture unit 40 may acquire the reference image by capturing a still image of the anterior eye segment at the start of the first scan (immediately before, simultaneously with, or immediately after) under the control of the control unit 7.

Any scan (second scan) performed after the first scan is started while the video capture unit 40 is capturing video of the anterior segment. The control unit 7 (or data processing unit 8) compares the frames sequentially acquired by the video capture unit 40 with a reference image. This comparison may include any image processing, such as segmentation to identify landmarks, image matching, image correlation, etc.

When it is determined that a frame that is substantially the same as the reference image has been obtained, the control unit 7 controls the illumination system 2, the imaging system 3, and the movement mechanism 6 to start a second scan.

The slit lamp microscope of this example repeatedly executes the above-mentioned series of processes until the number of scans applied to the anterior segment reaches the number of scan iterations required to create an image set. This reduces the positional shift of the subject's eye E between scans, and prevents degradation of the image set quality due to eye movement. It also makes it possible to make image selection more efficient and easier.

<Operation>
Several examples of the operation of a slit lamp microscope according to exemplary embodiments will now be described.

Although not shown in the figure, at any stage, a user (subject, examiner, assistant, etc.) can input subject information into the slit lamp microscope system 1. The input subject information is stored in the memory unit 10 by the control unit 7. The subject information typically includes identification information of the subject (subject ID).

Furthermore, background information of the subject can be input. Background information is any information related to the subject, and examples include information about the subject's medical interview, information written by the subject on a designated sheet, and information recorded in the subject's electronic medical record. Typically, background information includes gender, age, height, weight, disease name, candidate disease name, test results (visual acuity value, eye refractive power value, intraocular pressure value, etc.), history of wearing and strength of refractive devices (glasses, contact lenses, etc.), test history, treatment history, etc. These are examples, and background information is not limited to these.

In preparation for imaging, the table on which the slit lamp microscope system 1 is placed, the chair on which the subject sits, and the chin rest of the slit lamp microscope system 1 are adjusted (all not shown). For example, the heights of the table, chair, and chin rest are adjusted. The chin rest is provided with a chin rest and a forehead rest to stably position the subject's face.

Once preparation is complete, the subject sits on a chair, rests his or her chin on the chin rest, and places his or her forehead on the forehead rest. Before or after these actions, the user performs an instruction operation to start photographing the subject's eye. This operation may be, for example, pressing an imaging start trigger button (not shown) or inputting an instruction voice. Alternatively, the control unit 7 may detect the completion of the preparation phase and automatically transition to the imaging phase. In addition, a fixation target (not shown) may be presented to the subject (test eye E or its fellow eye).

<First Operation Example>
A first operation example executed after the above preparation will be described with reference to Fig. 10. This operation example shows basic operations executed by the slit lamp microscope system 1. Those with ordinary knowledge in the technical field to which the present invention pertains (those skilled in the art) will be able to understand that a second operation example, which will be described later, is an application of this operation example.

(S1: Alignment)
In response to the start of imaging, the slit lamp microscope system 1 first aligns the illumination system 2 and the imaging system 3 with respect to the subject's eye E. Unlike general alignment for aligning the optical axis of the optical system with the corneal apex or pupil center of the subject's eye E, the alignment in step S1 is performed to position the illumination system 2 and the imaging system 3 at the start position of the scan performed in step S2. The alignment in step S1 may be performed in any manner, and may be any of the examples described above. Before the start of, during, and/or after the alignment, operations such as adjustment of the image sensor 5 and focus adjustment may be performed.

(S2: Acquire an image of the anterior segment)
After the alignment in step S1 is completed, the slit lamp microscope system 1 photographs the anterior segment of the subject's eye E to obtain an image. For example, the slit lamp microscope system 1 scans the anterior segment of the subject's eye E by performing a combination of irradiation of slit light by the illumination system 2, video shooting by the imaging system 3, and movement of the illumination system 2 and the imaging system 3 by the moving mechanism 6. This scan is applied to a three-dimensional area of the anterior segment. By performing one scan (scanning from the scan start position to the scan end position), for example, a group of images (multiple anterior segment images) F1 to FN shown in FIG. 5 can be obtained. Note that the imaging performed in step S2 is not limited to such a scan, and may be imaging for obtaining one image, for example.

The data processing unit 8 may perform a predetermined process on the acquired anterior eye image. For example, any signal processing or image processing such as noise removal, contrast adjustment, brightness adjustment, and color correction can be applied.

(S3: Identify a partial area of the image)
The partial region specifying part 86 analyzes the image acquired in step S2 and specifies a partial region corresponding to a target site of the anterior segment of the subject's eye E.

(S4: Select display conditions)
The display condition selection unit 71 selects the display condition corresponding to the target part from the display condition information 100 .

The timing of execution of partial region identification in step S3 and the timing of execution of display condition selection in step S4 may be arbitrary. For example, as in this operation example, partial region identification may be followed by display condition selection, or partial region identification may be performed after display condition selection, or at least a portion of the execution period of partial region identification may overlap with at least a portion of the execution period of display condition selection. Typically, partial region identification is performed at any timing after the target area is specified and after the anterior eye image is acquired, and display condition selection is performed at any timing after the anterior eye image is acquired.

(S5: Change the display conditions of the partial area)
The display control unit 72 applies the display condition selected in step S4 to the partial region specified in step S3. That is, the display control unit 72 changes the pixel values of the partial region specified in step S3 in accordance with the display condition selected in step S4.

(S6: Display image)
The display control unit 72 causes the display device 11 (display device 121) to display the anterior eye image in which the display conditions of the partial region have been changed in step S5 (END).

According to this first operation example, an image of a target area of the anterior segment of the subject's eye E can be displayed under conditions that correspond to this target area, improving the display quality of the target area and enabling optimal observation of the anterior segment of the subject's eye E.

<Second Operation Example>
A second operation example executed after the above-mentioned preparation will be described with reference to Figures 11A to 11C. This operation example shows an example of reading an anterior eye image acquired by the slit lamp microscope system 1. Those skilled in the art will understand that the same processing as this operation example can be applied to medical procedures other than image reading. In this operation example, the configuration of Figure 9 is adopted.

(S11: Alignment)
First, alignment is performed in the same manner as in step S1 of the first operation example. As a result, the illumination system 2 (illumination system 20) and the imaging system 3 (either the left imaging system 30L or the right imaging system 30R, or both) are disposed at a position corresponding to the scan start position, and the video imaging unit 40 is disposed, for example, in front of the subject's eye E.

(S12: Start video recording)
After the alignment is completed, video capture of the anterior segment is started by the video capture unit 40. The capture rate of this video capture may be equal to the capture rate of the imaging system 3 in the anterior segment scan described below.

The illumination system 2 and the imaging system 3 are moved to scan the anterior segment, but the video capture unit 40 captures video of the anterior segment from a fixed position. When an anterior segment scan is performed in parallel with video capture, the aforementioned correspondence is assigned between the image group collected in the anterior segment scan and the frame group collected by the video capture unit 40.

The timing for starting video recording can be arbitrary, for example, before or during alignment.

(S13: Anterior eye scan)
The slit lamp microscope system 1 scans the anterior segment of the subject's eye E by combining the irradiation of slit light by the illumination system 2, video capture by the imaging system 3, and movement of the illumination system 2 and the imaging system 3 by the movement mechanism 6.

In this example, 256 images are acquired in one scan (scan from the scan start position to the scan end position). The 256 images (image group) acquired in the first scan (the first scan performed in step S13; the same applies below) are indicated as H1(1) to H1(256). The 256 images H1(1) to H1(256) are assigned to 256 ordered positions in the scan range (the range in the X direction with the scan start position and scan end position at both ends).

(S14: Select a good quality image)
The selection unit 821 selects images that satisfy a predetermined image selection condition from the 256 images H1(1) to H1(256) acquired in step S13.

In this example, as shown in FIG. 12A, the 71st to 170th 100 images H1(71) to H1(170) do not satisfy the image selection condition, and the 1st to 70th and 171st to 256th 156 images H1(1) to H1(70) and H1(171) to H1(256) do satisfy the image selection condition. In this case, the selection unit 821 selects the 1st to 70th and 171st to 256th 156 images H1(1) to H1(70) and H1(171) to H1(256) from among the 256 images H1(1) to H1(256).

(S15: Create a provisional image set)
The selector 821 creates a provisional image set based on the image(s) acquired up to the current point in time. At this point in the example, a provisional image set is created that includes 156 images H1(1)-H1(70) and H1(171)-H1(256).

(S16: Number of selected images = predetermined number?)
The selection unit 821 compares the number of images included in the temporary image set created in the immediately preceding step S15 (the number of selected images) with a predetermined number. As described above, in this example, the predetermined number (the number of images included in the image set) is set to 256.

If the number of images included in the provisional image set reaches 256 (S16: Yes), the operation proceeds to step S18. If the number of images included in the provisional image set does not reach 256 (S16: No), the operation proceeds to step S17.

(S17: Number of scans = predetermined number?)
The selection unit 821 compares the number of scans performed up to the current stage (the number of times that scanning and image selection have been alternately repeated) with a predetermined number of times. As described above, in this example, the predetermined number of times is set to five times.

If the number of scans performed up to the current stage reaches five (S17: Yes), the operation proceeds to step S18. If the number of scans performed up to the current stage does not reach five (S17: No), the operation returns to step S13.

Steps S13 to S37 are repeated until step S16 or S17 returns "Yes." A "Yes" determination in step S16 corresponds to the number of images (256) required to create an image set being prepared. A "Yes" determination in step S17 corresponds to the scanning and image selection being repeated the maximum number of times.

(S18: Create image set)
If the determination in step S16 or S17 is "Yes", the image set creation unit 82 creates an image set based on the provisional image set created in the immediately preceding step S15.

Here, a specific example of steps S13 to S18 will be described. As described above, in the example of FIG. 12A, a provisional image set including 156 images H1(1) to H1(70) and H1(171) to H1(256) was created (step S15) based on the first scan (step S13) and the first image selection (step S14).

At this stage, the number of images included in the provisional image set has not reached 256 (S16: No), so the operation proceeds to step S17. At this stage, only one scan has been performed (S17: No), so the operation returns to step S13.

When the operation returns to step S13, a second anterior eye scan is performed. The 256 images obtained in the second scan are denoted by H2(1) to H2(256). As with the set of images obtained in the first scan, the 256 images H2(1) to H2(256) are assigned to 256 ordered positions in the scan range. Furthermore, the image H1(n) obtained in the first scan and the image H2(n) obtained in the second scan are associated with each other via the 256 positions (n = 1, 2, ..., 256). The same applies to the sets of images obtained in subsequent scans.

In step S14, the selection unit 821 selects images that satisfy a predetermined image selection condition from among the 256 images H2(1) to H2(256) acquired in step S13. In this example, as shown in FIG. 12B, it is assumed that the 100 images H2(1) to H2(100) from the 1st to 100th do not satisfy the image selection condition, and the 156 images H2(101) to H2(256) from the 101st to 256th do satisfy the image selection condition. In this case, the selection unit 821 selects the 156 images H2(101) to H2(256) from the 256 images H2(1) to H2(256).

Here, if two or more images corresponding to a certain position in the scan range are selected, the selection unit 821 can select one of those images. In the example shown in Figures 12A and 12B, for example, image H1 (256) and image H2 (256) are selected as images corresponding to the 256th position.

The selection unit 821 may be configured to execute a process of selecting one of the image H1 (256) and the image H2 (256). For example, the selection unit 821 may be configured to select the image H1 (256) acquired first, or to select the image H2 (256) acquired later. In this example, it is assumed that the selection unit 821 is configured to select the image acquired first.

As another example, the selection unit 821 may be configured to select one of the images H1 (256) and H2 (256) by comparing them. For example, the selection unit 821 may be configured to calculate an image quality evaluation value for the image H1 (256), calculate an image quality evaluation value for the image H2 (256), compare these image quality evaluation values, and select one of the images H1 (256) and H2 (256). Typically, the image with the highest image quality evaluation value is selected.

This process assigns up to one image to each of the 256 positions in the scan range. This means that the provisional image set contains a maximum of 256 images.

The selection unit 821 creates a provisional image set based on the two image groups obtained up to this point (image group H1(1) to H1(256) and image group H2(1) to H2(256)). At this stage in this example, a provisional image set is created that includes 156 images H1(1) to H1(70) and H1(171) to H1(256) based on the first scan, and 70 images H2(101) to H2(170) based on the second scan.

In other words, the provisional image set obtained at this stage includes 226 images H1(1) to H1(70), H2(101) to H2(170), and H1(171) to H1(256), which correspond to the 1st to 70th and 101st to 256th positions (226 positions) of the 256 positions in the scan range.

At this stage, the number of images included in the provisional image set has not reached 256 (S16: No), so the operation proceeds to step S17. At this stage, only two scans have been performed (S17: No), so the operation returns to step S13 again.

When operation returns to step S13, a third anterior segment scan is performed. The 256 images obtained in the third scan are shown as H3(1) to H3(256).

In step S14, the selection unit 821 selects images that satisfy a predetermined image selection condition from among the 256 images H3(1) to H3(256) acquired in step S13. In this example, as shown in FIG. 12C, it is assumed that the 227th to 256th 30 images H3(227) to H3(256) do not satisfy the image selection condition, and the 1st to 226th 226 images H3(1) to H3(226) do satisfy the image selection condition. In this case, the selection unit 821 selects the 1st to 226th 226 images H3(1) to H3(226) from among the 256 images H3(1) to H3(256).

Furthermore, the selection unit 821 selects 30 images H3(71) to H3(100) corresponding to the 71st to 100th positions from among the 226 images H3(1) to H3(226), excluding the 1st to 70th and 101st to 256th positions of the 256 positions in the scan range to which images have already been assigned.

The selection unit 821 creates a provisional image set based on the three image groups obtained up to this point (image group H1(1) to H1(256), image group H2(1) to H2(256), and image group H3(1) to H3(256)). At this stage in this example, a provisional image set is created that includes 156 images H1(1) to H1(70) and H1(171) to H1(256) based on the first scan, 70 images H2(101) to H2(170) based on the second scan, and 30 images H3(71) to H3(100) based on the third scan.

In other words, the provisional image set obtained at this stage includes 256 images H1(1) to H1(70), H3(71) to H3(100), H2(101) to H2(170), and H1(171) to H1(256) corresponding to all 256 positions in the scan range. Therefore, the answer in step S16 is "Yes," and the operation proceeds to step S18.

In step S18, the image set creation unit 82 creates an image set based on the 256 images H1(1) to H1(70), H3(71) to H3(100), H2(101) to H2(170), and H1(171) to H1(256) included in the provisional image set created in the immediately preceding step 35.

The operations shown in FIG. 11A provide an example process for creating an image set using alternating iterations of scanning and image selection, followed by the operations shown in FIG. 11B.

(S19: Evaluate the quality of the image set)
The image set evaluation unit 83 evaluates the quality of the image set created in step S18.

Alternatively, in this embodiment, the quality of the image set may be evaluated using the video captured by the video capture unit 40 in the manner described above. For example, the image set based on the example shown in Figures 12A to 12C includes a series of 256 images H1(1) to H1(70), H3(71) to H3(100), H2(101) to H2(170), and H1(171) to H1(256). Three scans were performed to collect these images, and video capture was performed by the video capture unit 40 in parallel with each scan.

Of the frames obtained by video capture in parallel with the first scan, the frame associated with image H1(n) is denoted by J1(n) (n = 1, 2, ..., 256). Of the frames obtained by video capture in parallel with the second scan, the frame associated with image H2(n) is denoted by J2(n) (n = 1, 2, ..., 256). Of the frames obtained by video capture in parallel with the third scan, the frame associated with image H3(n) is denoted by J3(n) (n = 1, 2, ..., 256).

Therefore, the 256 images included in the image set in this example, H1(1) to H1(70), H3(71) to H3(100), H2(101) to H2(170), and H1(171) to H1(256), correspond to frames J1(1) to J1(70), J3(71) to J3(100), J2(101) to J2(170), and J1(171) to J1(256), respectively.

The image set evaluation unit 83 can evaluate this image set by applying the same processing as that performed on the frame groups D1 to DN described above to the 256 frames corresponding to the 256 images included in the image set in this example.

(S20: Is the quality good?)
If it is determined in step S19 that the quality of the image set is good (S20: Yes), the operation proceeds to step S23. On the other hand, if it is determined in step S19 that the quality of the image set is not good (S20: No), the operation proceeds to step S21.

(S21: Prompt to retake the photo)
If it is determined in step S19 that the quality of the image set is not good (S20: No), the control unit 7 displays and/or outputs sound to prompt the user to take a new photograph.

(S22: Should we take the photo again?)
In response to the information displayed and/or output as sound in step S21, the user performs an instruction operation to start re-shooting or an instruction operation to not perform re-shooting.

If the user performs an instruction operation to start re-imaging (S22: Yes), the operation returns to step S11. In this case, the slit lamp microscope system 1 executes the series of processes up to this point again. Note that re-imaging is repeated, for example, up to a predetermined number of times.

On the other hand, if the user performs an instruction not to retake the image (S22: No), the operation proceeds to step S23.

(S23: Transmit image set)
If the quality of the image set is determined to be good in step S19 (S20: Yes), or if the user performs an instruction operation in step S22 not to re-take the image (S22: No), the control unit 7 controls the communication unit 9 to send the image set to the reading terminal and/or reading device.

In this operation example, the steps up to this point are executed by the slit lamp microscope (and a computer connected thereto) included in the slit lamp microscope system 1, and the steps thereafter (the steps shown in FIG. 11C) are executed by an information processing device capable of communicating with this slit lamp microscope (or computer). The steps shown in FIG. 11C are performed using a computer (image reading terminal) used by a doctor to perform image reading. Those skilled in the art will understand that similar processing can be executed even when a computer (image reading device) with image reading capabilities is used. An example of a system configuration in which this operation example can be adopted will be described later.

(S24: Start of interpretation)
A doctor using the image reading terminal starts reading the series of images included in the image set transmitted from the slit lamp microscope (or computer) in step S23 at a desired timing. The images to which the following steps are applied may be, for example, all images included in the image set, one or more images selected from the image set by the doctor, or one or more images selected from the image set by the image reading terminal.

(S25: Identifying a partial area of an image)
A partial area identification unit 86 provided in the reading terminal or other information processing device analyzes an image of the anterior segment of the subject's eye E (an image included in the image set) and identifies a partial area corresponding to the target area of the anterior segment of the subject's eye E.

(S26: Select display conditions)
A display condition selection unit 71 provided in the image interpretation terminal or another information processing device selects a display condition corresponding to a target site from the display condition information 100. Note that the display condition information 100 in this operation example is stored in a storage unit 10 provided in the image interpretation terminal or another information processing device.

Similar to the first operation example, the timing of performing the partial area identification in step S25 and the timing of performing the display condition selection in step S26 may be arbitrary. Furthermore, the information processing device that performs the partial area identification in step S25 and the information processing device that performs the display condition selection in step S26 may be the same or different.

(S27: Change display conditions of partial area)
The display control unit 72 provided in the image interpretation terminal or another information processing device applies the display condition selected in step S26 to the partial region specified in step S25. That is, the display control unit 72 provided in the image interpretation terminal or another information processing device changes the pixel values of the partial region specified in step S25 in accordance with the display condition selected in step S26.

(S28: Display image)
The display control unit 72 provided in the reading terminal or other information processing device displays the anterior eye image in which the display conditions of the partial area have been changed in step S27 on a display device 11 (display device 121) provided in the reading terminal or other information processing device.

(S29: Create image interpretation report)
The doctor can perform image interpretation while referring to the image displayed in step S28. The doctor can apply the processes of steps S25 to S28 to a desired image among a series of images included in the image set. In this way, the doctor interprets the anterior segment image of the subject's eye E and inputs the result into a predetermined report template. In this way, an image interpretation report on the subject's eye E is created.

(S30: Save image set and image interpretation report)
The image set and the image interpretation report are provided to and stored in, for example, a facility where the slit lamp microscope system 1 is installed, a medical institution designated by the subject, etc., an information processing device used by a doctor designated by the subject, etc., an address registered by the subject (email address, address, etc.), etc. In addition, the image set and the image interpretation report may be sent to a specified database system for storage and management (end).

Some effects of a slit lamp microscope system capable of executing this operation example will be described. A slit lamp microscope system capable of executing this operation example can prevent deviation of the subject's eye between scans by matching the anterior segment image at the start of scanning during alternating repetition of scanning and image selection. To this end, a slit lamp microscope system capable of executing this operation example includes a video capture unit 40 (image capture unit) that captures the anterior segment of the subject's eye from a fixed position. The control unit 7 is configured to cause the scanning unit to start a second scan in response to the video capture unit 40 acquiring an image that is substantially the same as the reference image acquired by the video capture unit 40 in response to the start of the first scan.

A slit lamp microscope system capable of performing this operation example can use video images (frame group) acquired by the video capture unit 40 in evaluating the quality of the image set. To that end, a slit lamp microscope system capable of performing this operation example includes a video capture unit 40 that captures video of the anterior segment of the subject's eye from a fixed position in parallel with the application of a scan to the anterior segment of the subject's eye. The image set evaluation unit 83 is configured to evaluate the image set quality based on the video images acquired by the video capture unit 40.

At this time, the image set evaluation unit 83 may be configured to evaluate the quality of the image set based on the correspondence between the series of images included in the image set and the series of frames included in the video captured by the video capture unit 40.

Furthermore, the image set evaluation unit 83 may be configured to evaluate the quality of the image set based on landmarks in a series of frames included in a video captured by the video shooting unit 40 and the correspondence between a series of images included in the image set and a series of frames included in the video.

An exemplary embodiment of a slit lamp microscope system capable of executing the second operation example is shown in FIG. 13. Note that elements of the slit lamp microscope system 200 of this embodiment that have the same configuration and function as the elements of the slit lamp microscope system 1 described above are given the same reference numerals as the corresponding elements of the slit lamp microscope system 1.

The slit lamp microscope system 200 of this embodiment includes a slit lamp microscope 210 and an information processing device 220. The information processing device 220 is, for example, the image reading terminal described above, and elements (hardware, software) for providing the image reading application are omitted from the illustration.

The slit lamp microscope 210 includes a control unit 7 and a data processing unit 8 similar to those of the slit lamp microscope system 1, as well as an image acquisition unit 211 and a communication unit 212.

The image acquisition unit 211 has a configuration and function for acquiring an image by scanning the anterior segment of the subject's eye with slit light, and includes, for example, the illumination system 2, the imaging system 3, and the moving mechanism 6 of the slit lamp microscope system 1. The image acquisition unit 211 may further include a processed image construction unit 85. The image acquired by the image acquisition unit 211 may be, for example, a single image, a group of images, an image set, a three-dimensional image, a rendered image, etc.

The communication unit 212 performs data communication between the slit lamp microscope 210 and other devices (external devices). The information processing device 220 is one of these external devices.

The control unit 7 of the slit lamp microscope 210 executes various controls performed in steps S11 to S23 of the second operation example. The control unit 7 of the slit lamp microscope 210 does not need to include the display condition selection unit 71 and/or the display control unit 72 shown in FIG. 1B.

The data processing unit 8 of the slit lamp microscope 210 executes various data processing operations performed in steps S11 to S23 of the second operation example. For example, the data processing unit 8 of the slit lamp microscope 210 may include an image group processing unit 80, an image group evaluation unit 81, an image set creation unit 82, a selection unit 821, an image set evaluation unit 83, etc. The image group evaluation unit 81 of the slit lamp microscope 210 may include an inference unit 811A, an inference model 812A, a three-dimensional image construction unit 811B, a comparison unit 812B, an evaluation processing unit 813B, an evaluation data generation unit 811C, an evaluation processing unit 812C, etc.

The information processing device 220 includes a communication unit 221 in addition to the memory unit 10, display condition selection unit 71, display control unit 72, partial area identification unit 86, and user interface 12 similar to those of the slit lamp microscope system 1. The memory unit 10 stores display condition information 100, which may include, for example, any of the display condition information 100a to 100d. The communication unit 221 performs data communication between the information processing device 220 and other devices (external devices). The slit lamp microscope 210 is one of these external devices.

Data (image, image group, image set, three-dimensional image, rendering image, front image, etc.) acquired by the image acquisition unit 211 of the slit lamp microscope 210 is transmitted to the information processing device 220 by the communication unit 212 (transmission unit). The information processing device 220 receives the data transmitted by the communication unit 212 of the slit lamp microscope 210 by the communication unit 221 (reception unit). Furthermore, the information processing device 220 executes the same processing as the slit lamp microscope system 1 by the memory unit 10, the display condition selection unit 71, the display control unit 72, the partial area identification unit 86, and the user interface 12.

Some examples of processing that can be performed by the slit lamp microscope 210 are described below. First, the slit lamp microscope 210 can process the image group collected by the image acquisition unit 211 using the image group processing unit 80.

The image group processing unit 80 of the slit lamp microscope 210 can evaluate the quality of the image group by the image group evaluation unit 81. If the image group evaluation unit 81 evaluates the quality of the image group to be good, the communication unit 212 can transmit this image group to the information processing device 220. If the image group evaluation unit 81 evaluates the quality of the image group to be bad, the control unit 7 of the slit lamp microscope 210 can execute control to cause the image acquisition unit (scanning unit) to execute a new scan of the anterior segment of the subject's eye.

The image group processing unit 80 of the slit lamp microscope 210 can use the image set creation unit 82 to select a series of images corresponding to the scan range from two or more image groups, including an image group collected in a previous scan and a new image group collected in a new scan, to create an image set.

The image acquisition unit 211 (scanning unit) of the slit lamp microscope 210 can collect two or more image groups by applying two or more scans (e.g., consecutively) to the anterior segment of the subject's eye. The image group processing unit 80 of the slit lamp microscope 210 can create an image set by using the image set creation unit 82 to select a series of images corresponding to the scan range from the two or more image groups collected by the two or more scans.

The image group processing unit 80 of the slit lamp microscope 210 can evaluate the quality of the image set by the image set evaluation unit 83. If the image set evaluation unit 83 evaluates the quality of the image set to be good, the communication unit 212 can transmit this image set to the information processing device 220. If the image set evaluation unit 83 evaluates the quality of the image set to be bad, the control unit 7 of the slit lamp microscope 210 can execute control to cause the image acquisition unit (scanning unit) to execute a new scan of the anterior segment of the examinee's eye.

Similar to the slit lamp microscope system 1, the slit lamp microscope system 200 can provide images that appropriately depict various target areas. In addition, the slit lamp microscope system 200 can remotely locate the slit lamp 210 and the information processing device 220, making it possible to put into practical use remote medical care using a slit lamp microscope, which was previously difficult.

The present disclosure provides a method for controlling a device or system according to any of the aspects. The present disclosure also provides a program for causing a computer to execute the control method, and a computer-readable non-transitory recording medium on which the program is recorded. The non-transitory recording medium may be in any form, and examples of such a recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

The present disclosure provides a method relating to any of the aspects (such as an imaging method, a data processing method, or a data communication method). The present disclosure also provides a program for causing a computer to execute the method, and a computer-readable non-transitory recording medium having the program recorded thereon.

The above-described aspects are exemplary embodiments and are not intended to limit the present invention. Any modifications (omissions, substitutions, additions, etc.) within the scope of the gist of the present invention may be made to the above-described aspects as appropriate.

REFERENCE SIGNS LIST 1 Slit lamp microscope 2 Illumination system 3 Photography system 4 Optical system 5 Image sensor 6 Movement mechanism 7 Control unit 71 Display condition selection unit 72 Display control unit 8 Data processing unit 80 Image group processing unit 81 Image group evaluation unit 82 Image set creation unit 83 Image set evaluation unit 85 Processed image construction unit 86 Partial region identification unit 9 Communication unit 10 Storage unit 100 Display condition information 200 Slit lamp microscope system 210 Slit lamp microscope 220 Information processing device

Claims (27)

a storage unit that stores in advance display condition information in which display conditions including a display color and a display gradation are associated with each of one or more parts of the anterior eye;
an image acquisition unit that acquires an image by scanning an anterior segment of a subject's eye with a slit light;
a partial region specifying unit that analyzes the image acquired by the image acquiring unit and specifies a partial region corresponding to a predetermined portion of the anterior segment of the subject's eye;
a display condition selection unit that selects a display condition corresponding to the predetermined portion from the display condition information;
a display control unit that causes a display means to display the image acquired by the image acquisition unit based on the display condition selected by the display condition selection unit,
the image acquisition unit includes a scanning unit that scans the anterior segment with the slit light to collect a group of images;
The scanning unit includes:
an illumination system that irradiates the anterior segment with the slit light;
an imaging system that images the anterior segment from a direction different from that of the illumination system;
a moving mechanism for moving the illumination system and the imaging system;
Including,
the imaging system repeatedly performs imaging in parallel with the movement of the illumination system and the imaging system by the movement mechanism;
A slit lamp microscope and an information processing device,
the slit lamp microscope includes the image acquisition unit and a transmission unit that transmits data to the information processing device,
the information processing device includes a receiving unit that receives the data transmitted by the transmitting unit, the storage unit, the partial region specifying unit, the display condition selecting unit, and the display control unit;
The slit lamp microscope further includes an image group processor that processes the image group collected by the scanning unit,
the image group processing unit includes a first evaluation unit that evaluates a quality of the image group;
When the first evaluation unit evaluates that the quality of the image group is good, the transmission unit transmits the image group to the information processing device.
Slit lamp microscope system. a first control unit that controls the scanning unit to perform a new scan of the anterior segment when the first evaluation unit evaluates that the image group has poor quality,
2. The slit lamp microscope system of claim 1 . a storage unit that stores in advance display condition information in which display conditions including a display color and a display gradation are associated with each of one or more parts of the anterior eye;
an image acquisition unit that acquires an image by scanning an anterior segment of a subject's eye with a slit light;
a partial region specifying unit that analyzes the image acquired by the image acquiring unit and specifies a partial region corresponding to a predetermined portion of the anterior segment of the subject's eye;
a display condition selection unit that selects a display condition corresponding to the predetermined portion from the display condition information;
a display control unit that causes a display means to display the image acquired by the image acquisition unit based on the display condition selected by the display condition selection unit,
the image acquisition unit includes a scanning unit that scans the anterior segment with the slit light to collect a group of images;
The scanning unit includes:
an illumination system that irradiates the anterior segment with the slit light;
an imaging system that images the anterior segment from a direction different from that of the illumination system;
a moving mechanism for moving the illumination system and the imaging system;
Including,
the imaging system repeatedly performs imaging in parallel with the movement of the illumination system and the imaging system by the movement mechanism;
A slit lamp microscope and an information processing device,
the slit lamp microscope includes the image acquisition unit and a transmission unit that transmits data to the information processing device,
the information processing device includes a receiving unit that receives the data transmitted by the transmitting unit, the storage unit, the partial region specifying unit, the display condition selecting unit, and the display control unit;
The slit lamp microscope further includes an image group processor that processes the image group collected by the scanning unit,
the image group processing unit includes a first evaluation unit that evaluates a quality of the image group;
a first control unit that controls the scanning unit to perform a new scan of the anterior segment when the first evaluation unit evaluates that the image group has poor quality,
Slit lamp microscope system. The image group processing unit further includes an image set creation unit that selects a series of images corresponding to a scan range from two or more image groups including the image group and a new image group collected by the new scan to create an image set.
4. A slit lamp microscope system according to claim 2 or 3 . a storage unit that stores in advance display condition information in which display conditions including a display color and a display gradation are associated with each of one or more parts of the anterior eye;
an image acquisition unit that acquires an image by scanning an anterior segment of a subject's eye with a slit light;
a partial region specifying unit that analyzes the image acquired by the image acquiring unit and specifies a partial region corresponding to a predetermined portion of the anterior segment of the subject's eye;
a display condition selection unit that selects a display condition corresponding to the predetermined portion from the display condition information;
a display control unit that causes a display means to display the image acquired by the image acquisition unit based on the display condition selected by the display condition selection unit,
the image acquisition unit includes a scanning unit that scans the anterior segment with the slit light to collect a group of images;
The scanning unit includes:
an illumination system that irradiates the anterior segment with the slit light;
an imaging system that images the anterior segment from a direction different from that of the illumination system;
a moving mechanism for moving the illumination system and the imaging system;
Including,
the imaging system repeatedly performs imaging in parallel with the movement of the illumination system and the imaging system by the movement mechanism;
A slit lamp microscope and an information processing device,
the slit lamp microscope includes the image acquisition unit and a transmission unit that transmits data to the information processing device,
the information processing device includes a receiving unit that receives the data transmitted by the transmitting unit, the storage unit, the partial region specifying unit, the display condition selecting unit, and the display control unit;
The slit lamp microscope further includes an image group processor that processes the image group collected by the scanning unit,
The scanning unit applies two or more scans to the anterior segment to collect two or more sets of images;
The image group processing unit includes an image set creation unit that selects a series of images corresponding to a scan range from the two or more image groups and creates an image set.
Slit lamp microscope system. The image group processing unit further includes a second evaluation unit that evaluates the quality of the image set created by the image set creation unit.
6. A slit lamp microscope system according to claim 4 or 5 . When the second evaluation unit evaluates that the quality of the image set is good, the transmission unit transmits the image set to the information processing device.
7. The slit lamp microscope system of claim 6 . and a second control unit that controls the scanning unit to perform a new scan of the anterior segment when the second evaluation unit evaluates that the quality of the image set is not good.
8. A slit lamp microscope system according to claim 6 or 7 . the display control unit applies at least a display color and a display gradation included in the display condition selected by the display condition selection unit to the partial region to display the image.
The slit lamp microscope system according to any one of claims 1 to 8 . The image acquisition unit acquires, as the image, a gradation image expressed in a predetermined reference gradation,
the display condition includes a display color code defined by one or more display colors and a display gradation including a plurality of gradation values set based on the reference gradation,
the display control unit converts the gradation values of the pixels of the partial region based on a display color code corresponding to the predetermined portion.
10. The slit lamp microscope system of claim 9 . the image acquisition unit acquires a grayscale image as the gradation image,
the display conditions include, as the display color code, a color code defined by one or more display colors different from white, black, and intermediate colors thereof, and a display gradation including a plurality of gradation values set based on the reference gradation,
the display control unit converts grayscale values of pixels in the partial region into color values based on a color code corresponding to the predetermined portion.
The slit lamp microscope system of claim 10 . the image acquisition unit acquires a color image as the gradation image,
the display control unit converts color values of pixels in the partial region into new color values based on a display color code corresponding to the predetermined portion.
The slit lamp microscope system of claim 10 . The image acquisition unit acquires two or more images of the anterior segment,
the partial region identifying unit identifies a partial region corresponding to the predetermined portion from each of the two or more images;
the display control unit applies the display condition corresponding to the predetermined part to two or more partial regions identified from the two or more images, and displays the two or more images.
A slit lamp microscope system according to any one of claims 1 to 12 . The storage unit stores an anterior eye image,
The partial region specifying unit specifies a partial region corresponding to the predetermined part from the anterior eye image,
the display control unit applies the display condition corresponding to the predetermined site to both the partial region specified from the anterior eye image and the partial region of the image acquired by the image acquisition unit, thereby displaying the anterior eye image and the image.
A slit lamp microscope system according to any one of claims 1 to 13 . a first user interface for performing an operation of specifying a partial area of the image displayed by the display control unit and an operation of changing a display condition;
the display control unit applies a display condition specified by the first user interface to the partial region specified by the first user interface.
A slit lamp microscope system according to any one of claims 1 to 14 . a second user interface for editing the display condition information;
A slit lamp microscope system according to any one of claims 1 to 15 . The one or more portions of the anterior eye include any one of the cornea, the iris, the lens, the angle, the anterior chamber, and floaters in the aqueous humor;
A slit lamp microscope system according to any one of claims 1 to 16 . the display condition information includes display conditions associated with one or more sites of interest,
The partial region specifying unit analyzes the image acquired by the image acquiring unit to specify a region of interest corresponding to a predetermined site of interest;
the display condition selection unit selects a display condition corresponding to the predetermined site of interest from the display condition information;
the display control unit applies the display condition corresponding to the predetermined site of interest to the region of interest and displays the image.
A slit lamp microscope system according to any one of claims 1 to 17 . the display condition information includes a display condition associated with a background,
The partial region specifying unit analyzes the image acquired by the image acquiring unit to specify a background image region corresponding to an outside of the subject's eye,
the display control unit applies the display condition corresponding to the background to the background image area to display the image.
A slit lamp microscope system according to any one of claims 1 to 18 . the display condition information includes display conditions associated with each of one or more artifacts,
the partial region specifying unit analyzes the image acquired by the image acquiring unit to specify an artifact image region corresponding to a predetermined artifact;
the display condition selection unit selects a display condition corresponding to the predetermined artificial object from the display condition information;
the display control unit applies the display condition corresponding to the predetermined artificial object to the artificial object image area to display the image.
A slit lamp microscope system according to any one of claims 1 to 19 . The imaging system includes:
an optical system for guiding light from the anterior segment irradiated with the slit light;
an imaging element that receives the light guided by the optical system on an imaging surface,
an object surface along the optical axis of the illumination system, the optical system, and the imaging surface satisfy the Scheimpflug condition;
A slit lamp microscope system according to any one of claims 1 to 20 . the moving mechanism moves the illumination system and the imaging system so as to move the slit light irradiated by the illumination system to the anterior eye segment in a width direction of the slit light.
A slit lamp microscope system according to any one of claims 1 to 21 . the partial region identifying unit identifies a partial region corresponding to the predetermined portion by analyzing at least one of the images included in the image group;
the display control unit applies the display condition corresponding to the predetermined part to at least one partial region specified from the at least one image, and displays the at least one image.
A slit lamp microscope system according to any one of claims 1 to 22 . The image acquisition unit further includes a processed image construction unit that constructs a processed image from the image group.
A slit lamp microscope system according to any one of claims 1 to 23 . The processed image construction unit constructs a three-dimensional image as the processed image.
25. The slit lamp microscope system of claim 24 . The processed image constructing unit constructs a rendering image of the three-dimensional image as the processed image.
26. The slit lamp microscope system of claim 25 . The partial region specifying unit analyzes the processed image to specify a partial region corresponding to the predetermined portion,
the display control unit applies the display condition corresponding to the predetermined part to the partial area specified from the processed image, and displays the processed image.
A slit lamp microscope system according to any one of claims 24 to 26 .

Images