JP2023032224A - Eye position abnormality detection system, eye position abnormality detection method, and eye position abnormality detection program - Google Patents

Abstract

To provide a simple eye position abnormality detection system that assists screening of an eye position abnormality by automating an eye position inspection by the "Cover-Test (ACT and CUT)" that is a representative inspection method of a qualitative inspection of discriminating mainly strabismus and heterophoria, an eye position abnormality detection method, and an eye position abnormality detection program.SOLUTION: Provided is an eye position abnormality detection system using a head-mounted display (HMD) with a line-of-sight tracking function. The eye position abnormality detection system includes the HMD with a line-of-sight tracking function and a controller that controls inspection operation of the HMD with a line-of-sight tracking function. The controller includes an inspection execution part that displays on the HMD a virtual inspection environment including a virtual inspection chamber that virtualizes a real inspection chamber, an indicator displayed in the virtual inspection chamber, and a shield for shielding right and left eyes.SELECTED DRAWING: Figure 14

Description

TECHNICAL FIELD The present invention relates to an eye position abnormality detection system, an eye position abnormality detection method, and an eye position abnormality detection program, and in particular, an HMD with a line-of-sight tracking function is used to automate an eye position examination and assist in screening for eye position abnormality. The present invention relates to a simple eye position abnormality detection system, an eye position abnormality detection method, and an eye position abnormality detection program.

In general, it is known that early detection and early treatment are necessary for eye (how to see) abnormalities related to eye diseases. Among them, eye position abnormalities such as strabismus and oblique position, which are typical eye diseases, are factors that later cause poor stereoscopic vision and asthenopia, so early detection and early treatment are necessary.

Here, strabismus refers to a state in which one eye is properly directed toward a target in binocular vision, but the line of sight of the other eye is off. Further, oblique position is a state in which no abnormalities are observed in binocular vision, but the line of sight deviates when one eye is occluded. In particular, strabismus is a risk factor that causes amblyopia, and how to detect strabismus at an early age and start treatment is a major research theme in pediatric ophthalmology.

However, it is difficult to diagnose eye position abnormalities, and examinations by ophthalmologists and orthoptists are necessary, but it is said that there is a chronic shortage of qualified persons for this type of diagnosis. From the above, it is of great significance to develop a simple system that can perform an eye position examination without a qualified person. However, visual function tests including current eye position tests have hardly been digitized and systematized. Although there are few digitized studies, the applicability of the oculometric device used in this study is limited to the diagnosis of exotropia. The reason for this is that it is difficult to determine the eye position during measurement except for exotropia, and it is difficult to uniquely specify the amount of eyeball deviation. However, the "movement of the eyeball during measurement" itself, which indicates an eye position abnormality, is not so small that it cannot be detected. For this reason, it is of great significance to digitize, systematize, and automate screening for the presence or absence of strabismus and obliques.

Here, patent documents as prior art related to the present invention will be described. In Patent Document 1, "For example, it is used in an ophthalmological clinic, etc., and relates to a sight line direction detection device that is applied to a perimeter, strabismus, oblique position, sight line direction, etc." (Specification paragraph "0001" of Patent Document 1 See.) In the technical field, for the purpose of "providing a line-of-sight direction detection device capable of detecting the line-of-sight direction with high precision regardless of individual differences" (see the same paragraph "0008"), " imaging means for imaging a pupil area with a two-dimensional imaging device; a light source for illuminating the pupil area including the iris; A parameter representing the relationship between the line-of-sight direction and the relative position for each subject eye based on the position of the presented visual target and the relative position between the pupil image captured by the imaging means at that time and the corneal reflection image of the light source. and the relative positional relationship between the pupil image of the eye to be examined and the corneal reflection image of the light source and the stored parameter at the time of subsequent measurement, to calculate the line-of-sight direction of the eye to be examined. A line-of-sight direction detection device characterized by having a calculation means." (See "Claim 1" of Patent Document 1.).

In addition, in Patent Document 2, "displacement measuring device, displacement measuring method, and program" (see paragraph "0001" of the specification of Patent Document 2). For the purpose of "providing a displacement measuring device" (see the same paragraph "0011"), "imaging that is relatively fixed with respect to a predetermined target and images at least the area around the eye of the subject Acquisition of captured images from the device, processing of detecting the three-dimensional position of the center of the pupil of the subject's left and right eyes, processing of detecting the three-dimensional position of the corneal vertices of the left and right eyes of the subject, and the left and right eyes A process of calculating a first angle formed by the center of the pupil with respect to the target, a process of calculating a second angle formed by the corneal vertices of the left and right eyes with respect to the target, and the first and calculating a first displacement angle that is the difference between the angle and the second angle” (see “Claim 1” of Patent Document 2). It is

In addition, in Patent Document 3, in the technical field “related to an ophthalmologic apparatus for inspecting the visual function of a subject” (see paragraph “0001” of the specification of Patent Document 3), “objective eye position-related state In order to provide an ophthalmologic apparatus capable of appropriately performing an appropriate examination" (see the same paragraph "0005"), "an ophthalmologic apparatus for examining the visual function of a subject, comprising: an anterior segment imaging unit configured to capture anterior segment images of left and right eyes of a subject; a fixation target presenting unit configured to present a fixation target to at least one of the left eye and right eye of the subject; a control unit for controlling an ophthalmologic apparatus, wherein the control unit processes the anterior eye image captured by the anterior eye imaging unit, thereby obtaining images of the subject's left eye and right eye. A timing after the presentation switching at which the eye position is measured and switching between presentation and non-presentation of the fixation target to at least one of the left eye and the right eye of the subject, and before the presentation switching. generating eye position state information indicating the state of the eye position of the subject based on the measurement results of the eye positions of the left eye and the right eye at at least two timings including the timing of device” (see “Claim 1” of Patent Document 3).

In addition, Patent Document 4 describes a technology "related to a visual target recognition determination system for determining whether or not a user is visually recognizing a visual target displayed on a display" (see paragraph "0001" of the specification of Patent Document 4). In the field, for the purpose of "providing a visual target recognition determination system capable of automatically and highly accurately determining whether or not a user visually recognizes a visual target" (see the same paragraph "0007"), " A visual target recognition determination system for determining whether or not a displayed visual target is visually recognized by a user, comprising: a display for displaying the visual target; a line-of-sight detection unit, a storage device for recording a log of the line-of-sight information, and a visual recognition determination unit for determining whether or not the user is visually recognizing the target based on the position information of the target and the line-of-sight information. a target proximity determination unit for determining whether or not the line of sight is close to the target, and a determination unit for determining whether or not the line of sight continues to be positioned near a predetermined coordinate position for a predetermined time and a gaze determination unit, wherein the user visually recognizes the target when the target proximity determination unit determines that the target proximity determination unit is in proximity and determines that the gaze determination unit continues to be positioned. and a visual recognition determination unit that determines that the eye is present” (see “Claim 1” of Patent Document 4).

Japanese Patent No. 3289953 Japanese Patent No. 6007543 Japanese Patent Application Laid-Open No. 2019-069049 Japanese Patent Application Laid-Open No. 2020-141848

Yuma Shinomiya, Yuji Takahashi, Goshi Nogami, Takahiro Niida, “Study of occlusion time in ocular position measurement with defusion using a line-of-sight analyzer”, The 71st Japanese Society of Amblyopia and Strabismus Society Ophthalmology Clinical Bulletin 9(3), 2016, p.231-233 Noriyuki Uchida, Kayoko Takatuka, Kouki Hinokuma, Konomi Hirata, Hisaaki-Yamaba, Naonobu Okazaki, “Automated cover-uncover test system using active LCD shutter glasses,” 23rd International Symposium on Artificial Life and Robotics (AROB), 2019, volume 24 , p.332-337 Yu Imaoka, Andri Flury, Eling D.de Bruin, “Assessing Saccadic Eye Movements With Head-Mounted Display Virtual Reality Technology”, Frontiers in Psychiatry11, 2020, p.1-19 Kunie Hashimoto, Nobuhisa Shiraishi et al., “Changes in biological functions over time”, (2) Values at single and complex reactions, Occupational Medicine, 1961, p.3-184. Ulrich Schiefer,Hans Strasburger,Stephan T.Becker,Reinhard Vonthein, Jan Schiller,Traugott J.Dietrich,William Hart,“Reaction time in automated ki-netic perimetry: effects of stimulus luminance,eccentricity,and movement direc-tion”,Vision Research Volume 41, Issue 16, 2001, p.2157-2164 Shigeo Maruo, Shinobu Awaya, Kazuo Kato, “Glossary of Orthoptics”, Kanehara Publishing, 2000, p.405-415 Shigeo Maruo, Shinobu Awaya, “Shigeo Maruo. Overview of strabismus Definition of strabismus”, Orthoptics 2nd edition, 2008, p.203

Regarding "squint measurement", the above-mentioned Patent Document 1 describes that "a fixation light source for strabismus measurement is sequentially presented to a normal eye and a patient eye, and the corneal reflection image and the pupil image of the patient eye are captured by the television camera 120 at that time. Then, the distances dx and dy are obtained from the obtained video signal by the calculator 122, the values of the angles θx and θy are obtained from the equations (2) and (3), and the angles θ and θ are obtained from the equations (4) and (5). Find the azimuth angle φ.Strabismus is measured from the difference between the eye position of the affected eye when fixed by a normal eye and the eye position of the affected eye when fixed by the affected eye.At this time, the angle θ is the squint This is the angle, that is, the inclination of the direction of the line of sight, and the direction is represented by the azimuth angle φ.The fixation light source for oblique measurement may be placed at a distant position outside the apparatus. When measuring , the fixation light source is placed near the eye to be examined E” (see paragraphs “0056” to “0057” of the specification). No measurements are mentioned. Further, for example, as can be read from FIGS. 1 and 5 and the description thereof, the “line-of-sight direction detection device” described in Patent Document 1 has a large device scale, and further simplification is required.

In the above Patent Document 2, "The state of eye position greatly affects binocular vision. The state of eye position includes, for example, binocular visual position (eye position when viewing with both eyes open) and Both eyes are upright with no deviation (eye position when fusion is eliminated by blocking one eye), no deviation in binocular visual position, but deviation in defusion eye position Squint, binocular visual position, and non-fusion eye position are deviated, etc. In addition, clinical γ angle (κ angle) abnormalities in which there is a discrepancy between the corneal reflex and the center of the pupil, macular deviation, etc. Apparent strabismus, such as macular deviation, in which misalignment can be seen is called pseudosquint" (see paragraph 0003). It has not been.
In addition, Patent Documents 1 and 2 do not disclose any ACT (Alternating-Cover Test) or CUT (Cover-Uncover Test) for determining strabismus or oblique posture.

Patent Document 3 includes a description of a "cover-an-cover test" (see paragraph "0004") and a description of specific inspection means and inspection methods for "squint" and "oblique" (see paragraph "0009"). - "0144", see Figures 1-16). However, the “ophthalmic device” described in Patent Document 3 is large in size, and further miniaturization is desired, as shown in FIG. 1 and its description (see paragraphs “0038” to “0039”).

In Patent Document 4, "The target visual recognition determination system 1 includes a head-mounted display (HMD) 10, a control device 30, and a communication cable 60 that connects the HMD 10 and the control device 30." (See paragraph “0014”.), and the size of the apparatus is reduced compared to the above Patent Documents 1 to 3. However, Patent Literature 4 does not describe or suggest eye position abnormalities such as strabismus and oblique position, and detection devices and methods thereof.

Therefore, in the present invention, the eye position test is automated by the "Cover-Test (ACT and CUT)", which is a typical qualitative test method that mainly distinguishes between strabismus and oblique, to screen for eye position abnormalities. An object of the present invention is to provide a simple eye position abnormality detection system, an eye position abnormality detection method, and an eye position abnormality detection program.

In addition, in the present invention, a head-mounted display (HMD) with eye-tracking added is used to create an examination environment under virtual reality. It is an object of the present invention to provide an eye position abnormality detection system, an eye position abnormality detection method, and an eye position abnormality detection program capable of quantifying an eye position abnormality.

Another object of the present invention is to provide an eye position abnormality detection system, an eye position abnormality detection method, and an eye position abnormality detection program that have a simple configuration and do not require experience or qualifications based on advanced specialized knowledge. and

Therefore, an eye position abnormality detection system according to claim 1 of the present invention is an eye position abnormality detection system using an HMD with a line-of-sight tracking function, wherein the eye position abnormality detection system includes the HMD with a line-of-sight tracking function (head mount display) and a controller for controlling the inspection operation of the HMD with eye-tracking function. , and shields for shielding the left and right eyes, respectively, and an examination execution unit for displaying a virtual examination environment on the HMD.

Further, the eye position abnormality detection system according to claim 2 of the present invention is the eye position abnormality detection system according to claim 1, wherein the procedure for shielding or removing the left and right eyes with the shielding object includes at least ACT ( Alternating-Cover Test) or CUT (Cover-Uncover Test),
The act of shielding or removing is configured to be performed by either visualization or non-visualization of the shield placed in front of the left and right eyes.

Further, the eye position abnormality detection system according to claim 3 of the present invention is the eye position abnormality detection system according to claim 2, wherein the controller controls the left and right eye positions accompanying the operation of shielding or removing the left and right eyes. and an evaluation unit that quantifies and calculates the degree of strabismus or oblique position based on the eyeball data, wherein the evaluation unit includes the ACT In the examination by CUT, the total amount of deviation of both eyes is calculated, and in the examination by the CUT, the amount of oblique deviation or the amount of oblique deviation is calculated, It is characterized in that a strabismus or oblique position is determined based on the amount of deviation.

Further, an eye position abnormality detection system according to claim 4 of the present invention is the eye position abnormality detection system according to claim 3, wherein the total deviation amount, the oblique deviation amount, and the oblique deviation amount is characterized by being calculated by the amount of prism.

Further, according to claim 5 of the present invention, there is provided an eye position abnormality detection system according to claim 3 or 4, wherein the eye position measurement unit measures the movement of the left and right eyes. It is characterized by removing invalid data from the acquired data, or correcting the effect of latency on the data acquired from the left and right eye movements.

Further, an eye position abnormality detection system according to claim 6 of the present invention is the eye position abnormality detection system according to any one of claims 1 to 5, wherein the HMD further includes a calibration unit. and the calibration unit executes at least one of adjustment of the mounting position of the HMD, adjustment of the interpupillary distance, and correction of line-of-sight information.

An eye position abnormality detection method according to claim 7 of the present invention is an eye position abnormality detection method using an HMD with a line-of-sight tracking function, wherein a controller for controlling an inspection operation of the HMD with a line-of-sight tracking function The HMD displays a virtual examination environment comprising a virtual examination room obtained by virtualizing the examination room, indices displayed in the virtual examination room, and shields for shielding the left and right eyes. A step is included.

Further, the eye position abnormality detection method according to claim 8 of the present invention is the eye position abnormality detection method according to claim 7, wherein in the examination execution step, the controller controls the movement of the left and right eyes by the shielding object. A shielding or removing procedure is performed according to at least either an ACT (Alternating-Cover Test) or a CUT (Cover-Uncover Test), and the shielding or removing operation is performed on the shield placed in front of the left and right eyes. It is characterized in that it is executed by either making objects visible or invisible.

Further, an eye position abnormality detection method according to claim 9 of the present invention is the eye position abnormality detection method according to claim 8, wherein the controller detects the left and right eye positions accompanying the operation of shielding or removing the left and right eyes. and an evaluation step of quantifying and calculating the degree of strabismus or obliqueness based on the eye data, wherein the controller calculates the total amount of deviation of both eyes in the examination by the ACT, calculates the amount of oblique deviation or the amount of oblique deviation in the examination by the CUT, and calculates the total amount of deviation and the amount of oblique deviation and determining whether the patient is oblique or oblique based on the amount of oblique deviation.

Further, an eye position abnormality detection method according to claim 10 of the present invention is the eye position abnormality detection method according to claim 9, wherein the total deviation amount, the oblique deviation amount, and the oblique deviation amount is characterized by being calculated by the amount of prism.

An eye position abnormality detection method according to claim 11 of the present invention is the eye position abnormality detection method according to claim 9 or 10, wherein in the eye position measurement step, the controller controls the left and right eye positions. It is characterized by removing invalid data from the data obtained from the eye movement, or correcting the effect of latency on the data obtained from the left and right eye movements.

A program according to claim 12 of the present invention is characterized by being a program for causing a computer to execute the method according to any one of claims 7 to 11.

According to the present invention, there are the following excellent effects,
(1) A virtual examination room was created using VR, a visual target was presented in the room, and the eye position was acquired while the open-shield operation was performed. By using the virtual environment, there is no need to prepare a large room in the real world, and there is a remarkable effect that the inspection can be performed in a small space.
(2) The HMD to be used is less uncomfortable to wear, and has a remarkable effect of reducing the burden on the subject.
(3) By constructing the system on VR, it is possible to completely control the wearing of the HMD, the presentation of the visual target, the opening and closing of the eyes, the acquisition of the eye position, the analysis, etc. by the program on the system side. It has a remarkable effect of being able to perform conversion.
(4) Since an eye position abnormality test can be performed without a qualified person, it is possible to solve the chronic shortage of medical technicians with national qualifications such as ophthalmologists and orthoptists. It is possible to perform a more appropriate screening than the current situation, which is a remarkable effect in medical checkups for children aged 10 and older, preschool medical checkups, and the like.
(5) Remarkable effect that it can contribute to the early detection of various serious adult diseases that cause ophthalmoplegia by utilizing it for screening for paralytic strabismus due to ophthalmoplegia in adults. play.

It is a figure explaining the definition of the squint angle in the clinical eye position test. (a) is a diagram for explaining the upright state, and (b) is a diagram for explaining the oblique (external oblique) state. It is a figure explaining the kind of strabismus. It is a figure explaining the examination example figure by ACT. It is a figure explaining the total displacement amount (angle). It is a figure explaining the examination example of internal obliques by CUT. It is a figure explaining the esotropia in CUT. It is a figure explaining the correction|amendment using a prism. It is a figure explaining binocular parallax and convergence. 1 is an external perspective view of an HMD according to one embodiment of the present invention; FIG. It is a figure explaining the internal structure of HMD of one Example of this invention. It is a figure explaining the positional relationship of the eyeball and HMD of one Example of this invention. 12 is a diagram for explaining the position of the origin and the coordinate relationship in FIG. 11; FIG. It is a figure explaining the structure of the system for implementing this invention. FIG. 14 is a diagram illustrating an inspection environment in the system of FIG. 13; 14A and 14B are diagrams for explaining how each eye sees and shields in an eye position test by the system of FIG. 13; FIG. FIG. 14 is a diagram for explaining an HMD mounting position adjustment screen in an eye position test by the system of FIG. 13; FIG. 14 is a diagram illustrating an interpupillary distance adjustment screen in an eye position test by the system of FIG. 13; 14A and 14B are diagrams for explaining a confirmation screen for eye tracking in an eye position test by the system of FIG. 13; FIG. FIG. 14 is a diagram illustrating removal of invalid frames in eye position examination by the system of FIG. 13; 14A and 14B are diagrams for explaining the timing of opening the shield and the actual eye movement in eye position examination by the system of FIG. 13; FIG. 14 is a diagram illustrating an example of Phase correction based on latency in an eye position test by the system of FIG. 13; 14A and 14B are diagrams for explaining eye movements and rotation points in an eye position test by the system of FIG. 13; 14A and 14B are diagrams for explaining the timing of opening the shield in the CUT and the actual movement of the eye in the eye position examination by the system of FIG. 13; 14A and 14B are diagrams for explaining an example of CUT correction in an eye position test by the system of FIG. 13; FIG. FIG. 14 is a diagram for explaining the number of pieces of latency data in an eye position test by the system of FIG. 13; FIG. 14 is a diagram for explaining a correspondence relationship of phases for viewing movement in the case of strabismus in eye position examination by the system of FIG. 13 ; FIG. 14 is a diagram for explaining a correspondence relationship of phases for observing movements in oblique posture in an eye position test by the system of FIG. 13 ; FIG. 14 is a diagram for explaining the correlation between the ACT eye position test result by the system of FIG. 13 and the ACT eye position test result by Maddox minor. FIG. 14 is a diagram for explaining the correlation between the CUT eye position test results obtained by the system of FIG. 13 and the CUT eye position test results obtained by Maddox's small culm; 2 is a block diagram showing the functional configuration of the eye position abnormality detection system 1. FIG. 2 is a block diagram showing the functional configuration of the HMD 10; FIG. 2 is a block diagram showing the functional configuration of a controller 20; FIG. FIG. 10 is a flowchart showing the overall detection processing of an eye position deviation amount and the like by an ACT examination; FIG. 4 is a flowchart showing calibration processing; 10 is a flowchart showing eyeball data acquisition processing during an ACT examination. 10 is a flow chart showing processing for calculating the amount and direction of eye misalignment by an ACT examination. 4 is a flow chart showing the overall processing for detecting an eye misalignment amount and the like by CUT examination. 10 is a flow chart showing eyeball data acquisition processing during a CUT examination. 10 is a flow chart showing processing for calculating the amount and direction of misalignment of the eyes and the amount of strabismus and the amount of strabismus by CUT examination.

EXAMPLES The present invention will be specifically described below with reference to Examples, but the present invention is not limited to these Examples.

In one embodiment of the present invention, a simple test support system is developed that automates eye position testing and aids in screening for eye position abnormalities. As an eye position test, the “Cover-Test”, which is a typical qualitative test method that mainly distinguishes between strabismus and obliques, is used. In addition, the developed system uses "VIVE Pro Eye" (trade name), which is a head-mounted display (HMD) with eye tracking added. A test environment is created under virtual reality, and Cover-Test is performed in it. At the same time, eye position abnormality is quantified using eyeball data obtained by eye tracking. The system can be used in small rooms and is very simple to operate. Therefore, even an unqualified person can easily perform the examination, which contributes to an increase in examination opportunities and early detection of eye position abnormality.

In the following, strabismus and squint, which are eye position abnormalities to be detected in the present invention, and examination methods thereof will be described. Eye position abnormalities such as strabismus and oblique position are eye diseases that require early detection and early treatment. Eye position examinations for examining these eye position abnormalities are roughly divided into two types: qualitative examinations and quantitative examinations. In the "Cover-Test", which is a typical qualitative test, the movement of the eyeballs when the subject covers and uncovers each eye with an ocular occluder determines whether or not they are strabismus or oblique, and what kind of characteristics they have. squint. From the results, it is possible to infer the presence or absence of abnormal binocular vision such as fusion and stereopsis. On the other hand, a quantitative test is a test that measures the degree of strabismus/oblique posture, and the results are expressed numerically. Specifically, as shown in FIG. 1, an angle (perspective angle) formed by a line connecting the line of sight of the abnormal eye, the squint eye rotation point, and the fixation point is obtained numerically. There are objective examination methods and subjective examination methods for examination methods, but the problem with objective examination methods is that the results depend on the years of experience and skills of the ophthalmologists and orthoptists who conduct the examination. There is In addition, the subjective examination method has a problem that the patient's cooperation with the examination (whether he or she carefully observes the index) is greatly affected.
Below are the basic concepts of the Cover-Test, an objective qualitative test method for strabismus and obliques, eye position test using the Maddox rod as a subjective quantitative test method, and the necessity of the proposed system. about.

[Strabismus/Oblique]
Eye position abnormalities include strabismus and oblique position (see, for example, Non-Patent Document 6). Strabismus is a syndrome of constant ocular deviation (bias) accompanied by abnormal binocular vision and amblyopia. Oblique position, on the other hand, is an eye position abnormality in which eyeball deviation appears only when fusion, which is the function to combine two images captured by both eyes, is disturbed. On the other hand, a person who keeps looking at the fixation point even when one eye is shielded (eye position to remove fusion) is called Orthophoria (for example, see Non-Patent Document 7). The same viewing direction is important to distinguish between these pathologies.

The visual direction is the direction of the eye toward a subjective object, and in the upright position, the visual direction is the same for both the left and right eyes, as shown in FIG. 2(a). However, as shown in FIG. 2(b), in the oblique or oblique position, the viewing direction is deviated due to shielding. As shown in Fig. 3, depending on the direction of deviation, if it is inward, it is esotropia (position), if it is outside, it is exotropia (position), if it is facing upward, it is oblique (position), and if it is facing down, it is oblique (position). ). There are various causes of strabismus and oblique position, and they are caused by abnormalities in the muscles that move the eyeballs, brain nerves, bones around the eyeballs, visual acuity abnormalities, and binocular visual function abnormalities. The Cover-Test is used to distinguish strabismus and squint.

[Cover-Test]
The Cover-Test is a test method that distinguishes between strabismus and squint by eye movement when one eye is covered. If one eye is shielded and the opposite eye that is not shielded moves, it is determined to be strabismus, and if the shielded eye moves when the eye is unshielded, it is determined to be oblique. The shielding time for one time is about 2 to 3 seconds. There are two types of test methods: ACT (Alternating-Cover Test) and CUT (Cover-Uncover Test). In ACT, the total amount of deviation, including overt and latent, is detected by blocking one eye at all times to prevent fusion (distinguishing between strabismus and obliqueness is not performed). CUT discriminates strabismus/oblique/orthotopic by obscuring one eye and then observing the movement of the eye when the obscuration is removed and the movement of the other eye. The difference between the two is that even with the same eyeball movement representing strabismus (oblique), the movement detected by CUT and the movement detected by ACT are larger in ACT (total deviation including latent). detect). In addition, it has been shown that CUT can distinguish not only the presence or absence of strabismus and obliques, but also the degree of binocular vision such as fusion ability. ACT and CUT are described below.

[ACT (Alternating-Cover Test)]
ACT (Alternating-Cover Test) is a test in which one eye is always alternately covered. The total amount of strabismus or latent eye misalignment (oblique position) (total deviation) is determined by always blocking one eye to prevent fusion. Subjects are instructed to gaze at the fixation target and first occlude one eye for 2-3 seconds. The occlusion is then transferred to the other eye, and after 2-3 seconds of occlusion, the occlusion is transferred to the other eye again. Repeat this operation and observe the movement of the unoccluded eye. If the unoccluded eye moves, strabismus or strabismus is present. The deoccluded eye is upright if it does not move. FIG. 4 shows an example of inspection.

For example, in the case of esotropia or esophagus, in either case:
1. The left eye is turned inward due to left eye deprivation (see Fig. 4(A)).
2. When the shield is transferred from the left eye to the right eye, the left eye moves to the right to catch the target, and the right eye also moves to the right (see Fig. 4(B)).
3. When the shield is returned from the right eye to the left eye, the right eye moves to the left to catch the target, and the left eye moves to the left accordingly (see Fig. 4(C)).
4. When the shield is returned to the right eye, the left eye moves to the right to catch the target, and the right eye moves to the right accordingly (see Figure (D)).

The total amount of eye movement of each eye that moves at this time is the total amount of displacement (the sum of the amount of displacement in FIG. 4A and the amount of displacement in FIG. 4B). The total deflection may also be expressed in degrees, as shown in FIG. For example, assume that the left eye is occluded in the case of esotropia (posture). At this time, the angle formed by the original viewing direction and the currently facing direction is the angle α shown in FIG. 5(a). Similarly, if the right eye is shielded, the angle between the original viewing direction and the current viewing direction is the angle β shown in FIG. 5(b). At this time, if the total amount of displacement is represented by an angle, it is "α+β". In practice, this angle is quantified as an amount of prism (the amount of prism will be described later). A test that does this is called the APCT (Alternate-Prism Cover Test).

[CUT (Cover-Uncover Test)]
CUT (Cover-Uncover Test) covers one eye and then observes the movement of the eye when the cover is removed and the movement of the other eye to determine strabismus/oblique/orthoscopic. It is a test to discriminate the rank. Instruct the subject to gaze at the fixation target, block one eye, and observe the movement of the other eye at this time. In the upright or oblique position, the occluded eye does not move regardless of whether the left or right eye is occluded. With strabismus, when the eye fixating on the target is blocked, the other eye moves to fixate on the target. FIG. 6 shows eye movements in the case of oblique position on the CUT.

In the case of internal oblique,
1. There is no misalignment when both eyes are open (see Figure 6(A)).
2.By shielding one eye, the fusion is broken, and under shielding, the shielded eye shows enoblique deviation (see Fig. 6(B)).
3. If the shielding is removed, the fusional force acts again, so the right eye returns to its original position (see Fig. 6(C)).
4. Next, by shielding the other eye, the shielded eye also exhibits encrotic deviation (see Fig. 6(D)).
5. If the shield is removed, the fusional force will act again, so the left eye will return to its original position (see Fig. 6(E)).

Also, FIG. 7 shows eye movements in the case of strabismus in CUT. In the case of right intraocular strabismus,
1. When the right eye is shielded, the right eye remains facing inward (see Fig. 7(B)).
2. Even if the shield is removed, the right eye remains facing inward (see Fig. 7(C)).
3. Next, when the left eye is occluded, the right eye undergoes leftward ocular movement, and the left eye is obliquely deviated (see Fig. 7(D)).
4. When the occlusion is removed, the right eye undergoes enoblique deviation and the left eye undergoes rightward eye movements (see Fig. 7(E)).

In other words, when one eye is shielded, if the shielded eye moves when the shield is removed, it is oblique, and if the other eye that is not shielded moves, it is strabismus. Therefore, it is necessary to observe the state of the other eye when one eye is shielded. A test that quantifies the left and right eye movements when the line of sight is blocked by CUT as a prism amount (described later) is called SPCT (Single Prism-Cover Test).

[Quantitative eye position using Maddox rod]
Quantitative eye position testing using the Maddox minor rod is a method for measuring subjective total deviation in binocular vision, such as micro-angle strabismus with oblique and peripheral fusion, and intermittent strabismus. . As an examination method, the subject wears a Maddox rod in front of either the right or left eye. In this state, when a bright light source is shown at eye level at a distance of 5 m in a semi-dark room, a red line of light can be seen in the eye wearing the Maddox rod. In the upright position, the line of red light seen by the eye wearing the Maddox barb coincides with the image of the light source seen by the opposite eye without the Maddox barb.

On the other hand, in the case of strabismus or oblique position, the red line of light seen by the eye with the Maddox rod and the image of the light source seen by the other eye without the Maddox rod are different. It does not match. If the image of the light source and the red light line do not match, a prism is worn over the eye wearing the Maddox rod, as shown in FIG. The prism is gradually changed from weak power to strong power, and when the image of the light source and the line of red light match, the examinee is asked to answer. The power of the prism at the time of coincidence becomes the subject's subjective total deviation amount. The power (light deflection power) of a prism is expressed in units of prism diopters (△), and 1△ is defined as a deviation angle of 1 cm per 1 m. It is also expressed as 1△=4/7 (approximately 0.57) degrees.

[Current status of eye position examination]
The Cover-Test method seems simple at first glance, but the skill of the examiner is required to identify cover-uncover movements and rapid eye movements during the examination. This examination is performed by a medical technician such as an ophthalmologist or an orthoptist. However, there is a chronic shortage of medical technicians with such national qualifications. For this reason, even health checkups for three-year-old children, which are performed during the sensitive period of visual acuity, are currently being conducted without qualified personnel at more than half of examination venues nationwide. Usually, many of the patients suspected of having strabismus develop symptoms while they are still young children and visit an ophthalmologist with family members and other people around them. Some patients remain without seeing an ophthalmologist or being tested, believing it to be a minor issue. From the above, it is of great significance to develop a simple system that can perform an eye position examination without a qualified person.

Next, digitization (automation) of an eye position test (Cover-Test) using an HMD with an eye-tracking function will be described.
[Previous research]
Non-patent document 1 and non-patent document 2 are a few examples of digitization of eye position examination. In Non-Patent Document 1, adults who have no ocular disease other than refractive error are asked to gaze at visual targets placed at distances of 33 cm and 5 m, and both eyes obtained when one eye is shielded with a plate that transmits near-infrared light. of eyeball data is acquired by a line-of-sight analysis device. We calculated the amount of deviation from this eyeball data and compared it with the eye position of SCPT, demonstrating its usefulness for exotropia.

In Non-Patent Document 2, a device in which a small camera is installed in shutter glasses (NVIDIA 3D Vision2) is developed, and eye open shielding is automatically controlled. With this device, the Cover-Test is actually performed, and eye movement during the examination is acquired as a series of images.

However, these studies have problems such as the need for a large room and the discomfort and burden when wearing them. It is desired that the eye position examination can be performed in a space-saving manner and by a method that places less burden on the examinee.

[Automation by HMD]
In recent years, due to the increase in content such as games using virtual reality (VR), HMDs are becoming popular, and at the same time, devices that perform line-of-sight tracking have also appeared. In this study, we propose to perform an eye position abnormality test in a virtual environment using a head-mounted display (HMD) with eye-tracking function.

The human eye has binocular parallax, which is the difference between the retinal images seen by the left and right eyes caused by viewing an object with both eyes. As shown in FIG. 9, by generating images according to the binocular parallax and presenting these images at the correct positions with respect to the human eyes, it is possible to make the images appear to have depth. . In VR, an environment with depth can be presented by presenting images using this principle.

In the present invention, an examination room is created using VR, a visual target is presented in the room, and an eye position is acquired while an open shielding operation is performed. By using a virtual environment, it is not necessary to prepare a large room in the real world, and it can be implemented in a small space. In addition, the HMD to be used is comfortable to wear, and the burden on the examinee is small. In addition, by constructing the system on VR, it is possible to completely control the HMD attachment, visual target presentation, eye opening and shielding, eye position acquisition, analysis, etc. by the program on the system side. Easy to do.

Next, the configuration and operation of the eye position abnormality detection system 1 for performing eye position abnormality detection by a digitized Cover-Test will be described.
[System configuration]
As described above, the present invention provides an eye position abnormality detection system 1 that performs a quantitative eye position test based on the Cover-Test technique. FIG. 31 is a block diagram showing the functional configuration of the eye position abnormality detection system 1. As shown in FIG. The eye position abnormality detection system 1 includes an HMD 10 and a controller 20 .

In this embodiment, the software and hardware environment used to build the eye position abnormality detection system 1 are shown below. It should be noted that this is only an example implementation, and the invention can also be implemented with other software and hardware. In this example, the eye position abnormality detection system 1 is roughly divided into four pieces of hardware (see FIG. 14).
・A: HMD10 HMD worn by the subject ("VIVE Pro Eye")
・B: Link box Used for exchanging data between the HMD 10 and the controller 20 ・C: Controller 20 A PC for constructing a virtual examination room, recording eyeball data, checking the output state of the HMD, etc.
D: Base station In the virtual examination room, the HMD tracking the HMD 10 and the link box are connected by the E headset cable, and the link box and the PC are connected by the F Mini Display Port and USB (Universal Serial Bus) 3.0, respectively.

[Head mounted display (HMD)]
The HMD 10 is a head-mounted display (HMD) with an eye-tracking function. Typically, the HMD 10 is an eyeglass-type wearable device that can be worn on the head, and includes a display for displaying various information, cameras and sensors for detecting the state of the eyeballs, and communication with external devices. and a control unit for controlling them.

FIG. 32 is a block diagram showing the functional configuration of HMD 10 according to this embodiment. The HMD 10 includes an eyeball data acquisition section 101 and a calibration section 102 .

[Gaze tracking function]
The eyeball data acquisition unit 101 detects the direction of the eyeball (in other words, the line of sight) at any time, and outputs it as eyeball data.

The software and hardware environment used to construct the HMD 10 in this embodiment are shown below. It should be noted that this is only an example implementation, and the invention can also be implemented with other software and hardware. As the HMD 10, an HMD "VIVE Pro Eye" manufactured by HTC was used. Figure 10 shows the appearance of "VIVE Pro Eye". In this case, the eyeball data acquisition unit 101 can be realized by a camera and an infrared sensor built in "VIVE Pro Eye".
The specifications of "VIVE Pro Eye" are as follows.
・Gaze tracking accuracy: 0.5° to 1.1° (accuracy for viewing angles up to 20°)
・Limit of eye-tracking viewing angle: 110°
・Interpupillary distance adjustment (calibration) function (described later)

As shown in FIG. 11, "VIVE Pro Eye" obtains various information about the eye by correcting the position of the image of the eyeball with the built-in camera 1001 (see FIG. 11) and the infrared sensor 1002 (see FIG. 11). (However, the eyeball image itself cannot be obtained.) In addition, a software development kit "SRanipalSDK" (trade name) is provided to use this information. Acquirable information is, for example, as follows.
・Coordinates of the starting point of the line of sight ・The degree of eye opening ・Position of the pupil ・Direction vector of the line of sight ・Convergence distance

In the present embodiment, the eye position abnormality inspection process is constructed using the "line-of-sight starting point coordinates (corresponding to the position of the vertex of the cornea)" as eyeball data. FIG. 12 (quoted from Non-Patent Document 3) shows the positional relationship between the HMD 10 and the eyeball. The line-of-sight origin coordinates obtained by the SRanipal SDK are three-dimensional coordinates of the position of the vertex of the cornea of the eye, with the origin being "System Origin" in FIG. 12, and the unit is mm. The actual position of "System Origin" on the HMD 10 is the position indicated by "System Origin (x,y,z)=(0,0,0)" in FIG. , the vertical upward direction is the y-axis, and the line-of-sight direction is the z-axis. Therefore, the coordinates of the eye position are corrected to the left-hand coordinates by Equation (1).

The eyeball data acquisition unit 101 outputs the acquired eyeball data to the examination execution unit 201 of the controller 2 .

[calibration]
In order to obtain correct eyeball data during examination, the subject needs to be able to wear the HMD 10 correctly. Therefore, it is preferable that the calibration unit 102 performs calibration after the HMD 10 is worn. The calibration unit 102 can be implemented using functions provided by, for example, "SRanipalSDK". In this case, the subject can perform the calibration by operating the HMD 10 by himself/herself.
Calibration consists of the following three steps.
(1) Adjustment of HMD mounting position (2) Adjustment of interpupillary distance (3) Calibration of line-of-sight information

Each step executed by the calibration unit 102 will be described below.
(1) Adjusting the Mounting Position of the HMD 10 In order to perform eye tracking correctly, the camera and infrared sensor shown in FIG. 11 must be able to detect the eyes normally. Therefore, the mounting position of the HMD 10 is adjusted so that detection can be performed normally. FIG. 17 shows an actual instruction screen. First, the calibration unit 102 displays the screen of FIG. 17(A) on the display. The subject raises and lowers the main body of the HMD 10 so that the HMD mark shown in FIG. 17 is within the dotted line and the state shown in FIG.
(2) Adjustment of interpupillary distance In order to perceive the correct size and depth of an object, the presented image must be reproduced with the correct size. For this purpose, it is necessary to match the interpupillary distance with that of the subject. FIG. 18 shows an actual instruction screen. First, the calibration unit 102 displays the screen of FIG. 18(A) on the display. The subject operates the HMD 10 so that the white bar in the bottom of the figure falls within the dotted line, as shown in FIG. 18(B).
(3) Calibration of line-of-sight information In this step, visual targets are presented one after another, and the correspondence between the known positions of the visual targets and the detected line-of-sight direction is checked to calibrate the line-of-sight information such as the direction vector of the line of sight to be calculated. do. After completing the previous step, the calibration unit 102 displays an instruction to follow the eye with a dot on the display. After that, as shown in FIG. 19, points are displayed in the order of the screen center→upper right→lower left→upper left→lower right, and the subject follows them with his/her eyes.

The controller 20 is an information processing device that can communicate with the HMD 10 by wire or wirelessly, and is typically a personal computer, a server computer, or the like. The controller 20 may be a single computer or a computer system in which a plurality of computers perform processing in a distributed manner. Also good. The controller 20 may be installed in a local environment and communicate with the HMD 10, or may be implemented using a so-called cloud environment or the like and communicate with the HMD 10 via communication means such as the Internet. Alternatively, the controller 20 may be built in the HMD 10 or may be realized by hardware common to the control unit of the HMD 10 .

FIG. 33 is a block diagram showing the functional configuration of controller 20 in this embodiment. The controller 20 includes an examination execution unit 201 , an eye position measurement unit 202 and an evaluation unit 203 . Note that the internal configuration of the evaluation unit 203 differs depending on the inspection method. In the case of the CUT examination, the evaluation unit 203 includes a strabismus amount/slantness amount calculation unit 2031 , an eye position deviation amount calculation unit 2032 , and an eye position deviation direction calculation unit 2033 . In the case of the ACT examination, the evaluation unit 203 includes an eye misalignment amount calculator 2032 and an eye misalignment direction calculator 2033 .

The software and hardware environment used to construct the controller 20 in this embodiment are shown below. It should be noted that this is only an example implementation, and the invention can also be implemented with other software and hardware.
・CPU: Intel(R) Core i78700K
・GPU: NVIDIA GeForce RTX2080
・HMD development kit: SRanipalSDKVer1.1.0.1
・3DCG development kit: Unity20193.0f5
・OS: Windows 10
・Implementation language: C#

[Virtual examination environment]
The examination execution unit 201 causes the display of the HMD 10 to display a virtual examination environment. The virtual examination environment is a virtual three-dimensional space including an examination room as shown in FIG. 15, a visual target (a black dot shown in the center of FIG. 15), and shields for covering the left and right eyes. In this embodiment, Unity is used to create a virtual inspection environment. Unity is a game development platform (integrated development environment) provided by Unity Technologies, which can create 3D images by appropriately arranging objects in a 3D space. When the program implemented using Unity is executed, a screen as shown in FIG. 15 is output on the display of the HMD 10 in a binocular stereoscopic view.

The examination environment includes three elements: examination room, target, and shield. As shown in Fig. 15, the examination room was created with a width and depth of 6m and a height of 5m. In this chamber, a red sphere with a diameter of 10 cm (a black dot shown in the center of FIG. 15) is placed at a position 5 m from the subject as a target. For coordinates on Unity, 1 unit corresponds to 1m in the real world. Using this fact, the size of each was set.

The subject is asked to gaze at the optotype on the screen, and the test execution unit 201 performs the open shielding operation based on the ACT and CUT techniques. FIG. 16 shows how the inspection screen is seen by each eye when the shield is opened. First, we start with both eyes unoccluded, and perform occlusion/removal based on the ACT and CUT techniques. Open occluders are achieved by making visible/invisible occluders placed directly in front of each eye. In this embodiment, a black shield that can completely block one's field of vision is placed in front of the screen, and the open shield is performed by switching between visible and invisible. When outputting the inspection screen, there is a possibility that the eyeball data of the eye cannot be obtained by not outputting only the screen of the corresponding eye when it is blocked, so such a method was adopted. In the meantime, the eye position measurement unit 202 continues to acquire eyeball data transmitted from the HMD 10 . The evaluation unit 203 quantifies the degree of strabismus from the obtained magnitude of eye movement. ACT calculates the total amount of deviation of both eyes, and CUT calculates the amount of deviation as strabismus (amount of strabismus) and the amount of deviation (amount of obliques) as oblique.

[Detection of eye misalignment by ACT test]
A method of detecting an eye misalignment amount by an ACT examination will be described with reference to the flowcharts of FIGS. As shown in FIG. 34, the detection of the amount of eye misalignment by the ACT examination broadly includes the following three steps.
(1) HMD mounting and calibration (2) Acquisition of eyeball data during ACT examination (3) Calculation of amount and direction of eye misalignment [HMD mounting and calibration]
The subject wears the HMD 10 and performs calibration by the calibration unit 102 . As shown in FIG. 35, calibration consists of the following three steps. Since the specific processing in each step has already been described, the description is omitted here.
(1) Adjusting the mounting position of the HMD (2) Adjusting the interpupillary distance (3) Calibration of line-of-sight information [acquisition of eyeball data during ACT examination]
As shown in Fig. 36, to acquire eyeball data during an ACT examination, after having the user gaze at the target with both eyes open, the operator must cover one of the left and right eyes while opening the other eye. Perform by alternating left and right. When the examination execution unit 201 executes the ACT examination program with the subject wearing the HMD, an examination environment screen appears on the display of the HMD 10 as shown in FIG. The subject is asked to gaze at the visual target appearing on the screen. In this embodiment, the inspection execution unit 201 starts opening the shield 3 seconds after the screen appears. Each open shield motion is performed every 3 seconds, and the interval between the open shield motions is named Phase0, 1,...,9 from the earliest. During this time, the eyeball data acquisition unit 101 of the HMD 10 acquires eyeball data of the subject at any time, and transmits the eyeball data to the eye position measurement unit 202 of the controller 20 . In the present embodiment, the eyeball data are acquired and transmitted about 90 times per second. Table 1 shows the operation of opening the shield and the state of each eye at that time in this embodiment.

[Calculation of Eye Position Misalignment Amount and Direction]
The amount of prism (amount of eye misalignment) to be acquired in ACT is the total amount of deviation (sum of eye movements shown in FIG. 4). As shown in FIG. 37, the calculation of the prism amount (amount of eye misalignment) and eye misalignment direction in ACT consists of the following eight steps.
(1) Acquisition of eyeball data (2) Removal of invalid frames (3) Correction of the effect of latency (4) Normalization of the number of data (5) Calculation of the average eye position (6) Calculation of the amount of movement ( 7) Calculation of prism amount (8) Judgment of deviation direction

Each step is described below.
[Acquisition of eyeball data]
The eye position measurement unit 202 continues to take eyeball data approximately 90 times per second from the start to the end of the examination. The "line-of-sight starting point coordinates" included in the eyeball data are used as the eye position data. The coordinates of the starting point of the line of sight are obtained as three-dimensional coordinates (x, y, z) (explained in the item [Eye Tracking Function]). Since the eye movement can be regarded as movement on a plane, the z coordinate is not used for determination, and the eye position data is represented by (xy).

[Remove invalid frames]
When acquiring eyeball data using the eyeball data acquiring unit 101 of the HMD 10, data acquisition failures almost always occur several times due to reasons such as blinking. Such data is called an invalid frame. The eyeball data itself is transmitted from the HMD 10 to the controller 20 even in the invalid frame, but in the invalid frame, the coordinates of the starting point of the line of sight are (x, y, z)=(0, 0, 0). The eye position measurement unit 202 excludes invalid frames from subsequent calculations. FIG. 20 shows an example in which invalid frames are excluded.

[Correction of influence of latency]
The human eye experiences a considerable delay between the time the stimulus is applied and the time the eye actually responds. This delay time is called latency. As shown in FIG. 21, there is a delay (latency) between the opening of the shield (T0 in FIG. 21) and the actual eye movement (black dotted line in FIG. 21). If the eye position data from opening of the shield until the eye begins to move is used as it is for calculation of the amount of prism, the result may be adversely affected. Therefore, the eye position measurement unit 202 determines the time from the opening of the shield (T0) to the maximum eye movement within the certain time Dmax as the latency. According to [20][21], the latency is approximately 200 ms for 20-year-olds, 210 ms for 40-year-olds, 230 ms for 50-year-olds, and 290 ms for 65-70-year-olds, so Dmax = 400 ms. The magnitude di of the eye movement in the i-th frame was calculated by Expression 2, with the eye position data in the i-th frame being xi and yi, respectively.

In the upright position, there is almost no eye movement, so the calculated latency is not accurate, but it has little effect on the judgment.
In each Phase, the eye position measuring unit 202 excludes frame data from the start of Phase to latency, and adds frame data from the end of Phase to the next latency. FIG. 22 shows an example of actual correction.

[Normalization of number of data]
Each Phase is 3 seconds long, and approximately 90 eye position data are obtained per second. However, since the number of data in each Phase is not constant due to the effects of processing delays and the processing of the previous two steps, the determination by the evaluation unit 203, which will be described later, is complicated. Therefore, it is preferable that the eye position measurement unit 202 equalizes the number of data used for determination to a constant number by interpolation. Each Phase is equally divided into N, and the data at each time point is obtained by internally dividing the data actually obtained before and after that phase. In the present invention, N=200.

[Calculation of average eye position]
The eye position measurement unit 202 obtains an average value of the eye position data of each Phase in order to obtain the amount of movement of the eye due to open shielding. The average value (xj, yj) of the eye position of Phasej is obtained by Expressions 3 and 4 using the eye position data of frame i included in Phasej.

[Calculation of eye movement]
The eye movement amount mj in Phase j is the difference between the eye position average values of Phase j and Phase j+1 obtained in [Calculation of Eye Position Average Value] above. Formulas 5 and 6 show calculation formulas.

[Calculation of prism amount]
The eye misalignment amount calculation unit 2032 of the evaluation unit 203 calculates a prism amount (amount of eye misalignment) based on the eye movement amount calculated by the eye position measurement unit 202 . As shown in FIG. 23, it is assumed that the eyeball moves from point a to point b due to open shielding. The movement amount mj at this time is obtained by the eye position measurement unit 202 . The eye can be regarded as rotating around the eyeball rotation point. It is known that the distance between the point of rotation of the eyeball and the vertex of the cornea (coordinates of the starting point of the line of sight) is about 13 mm in humans regardless of age and gender. Assuming that the deviation angle is θj (deg), the unit of the prism amount is 1Δ≈0.57 (deg).

This is the amount of prism in one eye. If PRj is the prism amount of the right eye and PLj is the prism amount of the left eye, then the total deviation amount Pall uses only the values of Phases 3 to 6 where the measured values are stable, and is calculated by Equation 9 below.

[Determination of direction of deviation]
The eye misalignment direction calculation unit 2033 of the evaluation unit 203 calculates the direction of eye misalignment. Regarding the determination of the direction of squint deviation, the left eye is used as a reference, the average value (mL) of the movement amount (mLj) of the left eye when the left eye is blocked is obtained, and the sign of the value is used for determination. In addition, the value of Phase 9 is not used because both eyes are open and it is inappropriate as a value to be used in ACT. ML is obtained by Equation 10 below.

In the horizontal direction, when the value of mL becomes negative, it is judged as exotropia (position), and when it becomes positive, it is judged as esotropia (position). In the vertical direction, when the value of mL becomes negative, it is determined that the person is oblique (position), and when the value is positive, it is determined that the person is oblique (position).

The evaluation unit 203 outputs at least one of the calculated prism amount (amount of eye misalignment) and the direction of eye misalignment. For example, these calculation results are stored in a storage area (not shown). Alternatively, these calculation results may be displayed on the display of the HMD 10 or the display (not shown) of the controller 20 or the like.

[Detection of Eye Position Misalignment by CUT Inspection]
A method of detecting an eye misalignment amount by a CUT examination will be described with reference to FIGS. 34 and 38 to 40 flowcharts. As shown in FIG. 38, the detection of eye misalignment by CUT examination broadly includes the following three steps.
(1) HMD wearing and calibration (2) Acquisition of eyeball data during CUT examination (3) Calculation of the amount and direction of misalignment and the amount of strabismus/squint [HMD wearing and calibration]
The subject wears the HMD 10 and performs calibration by the calibration unit 102 . As shown in FIG. 34, calibration consists of the following three steps. Since the specific processing in each step has already been described, the description is omitted here.
(1) Adjusting the mounting position of the HMD (2) Adjusting the interpupillary distance (3) Calibration of line-of-sight information [acquisition of eyeball data during CUT examination]
As shown in FIG. 39, eyeball data acquisition during CUT examination involves gazing at an index with the field of view of both eyes open, followed by the operation of covering either the left or right eye and then opening it alternately. to implement. When the CUT inspection program is executed with the subject wearing the HMD, an inspection environment screen appears on the display of the HMD 10, as shown in FIG. 15, as in the case of ACT. The subject is asked to gaze at the visual target appearing on the screen. In this embodiment, the inspection execution unit 201 starts opening the shield 3 seconds after the screen appears. Each open shield motion is performed every 3 seconds, and the intervals between the open shield motions are named Phase0, 1, ..., 12 in order from the earliest. The operation of open shielding is right eye shielding (Phase1, 5, 9) → right eye removal (Phase2, 6, 10) → left eye shielding (Phase3, 7, 11) → left eye removal (Phase4, 8, 12). 3 sets are implemented as 1 set. During this time, the eyeball data acquisition unit 101 of the HMD 10 acquires eyeball data of the subject at any time, and transmits the eyeball data to the eye position measurement unit 202 of the controller 20 . In the present embodiment, the eyeball data are acquired and transmitted about 90 times per second. Table 2 shows the operation of opening the shield and the state of each eye at that time in this embodiment.

[Calculation of Amount and Direction of Displacement of Eye Position and Amount of Squint/Strabismus]
As shown in FIG. 40, the calculation of the amount of prism (the amount of eye misalignment), the direction of eye misalignment, and the amount of strabismus and the amount of misalignment in CUT consists of the following 10 steps.
(1) Acquisition of eyeball data (2) Removal of invalid frames (3) Correction of the effect of latency (4) Normalization of the number of data (5) Calculation of the representative value of the eye position (6) Calculation of the amount of movement ( 7) Calculation of prism amount (8) Calculation of amount of strabismus and obliqueness (9) Judgment of deviation direction (10) Judgment of strabismus and obliqueness Of these, (1), (2), (4), ( Regarding 7), it is exactly the same as that of ACT, so the explanation is omitted.

[Correction of influence of latency]
As mentioned in [Elimination of Invalid Frames], there is a latency from when the stimulus is applied until the eye actually responds. As with ACT, using this as-is for calculations may adversely affect the results. Therefore, as shown in FIG. 24, the eye position measurement unit 202 determines that the latency is the time from the opening timing (T0) until the eye movement reaches its maximum within a certain period of time Dmax, as in ACT. . However, in the case of CUT, there is an opportunity for both eyes to be opened, and the fusional force works, so the movement of the eye position is more gradual than in ACT. Therefore, with Dmax=1000 ms, the magnitude of eye movement was calculated by Equation 2, and the maximum point was defined as the latency. Furthermore, as in ACT, in each Phase, frame data from the start of Phase to latency is excluded, and frame data from the end of frame to the next latency is added. An example of actual correction is shown in FIG.

[Calculation of representative value of eye position]
The eye position measuring unit 202 obtains a representative value of the eye position in each phase in order to obtain the amount of movement of the eye. In the case of CUT, the movement is slower than in the case of ACT. Therefore, the position of the eye at the peak was used as the representative value. Specifically, the average value of the interval corresponding to the latent time at the end of the corrected Phase (latency interval shown in FIG. 24) was obtained. The representative values v-jx and v-jy of the eye positions are obtained by Equations 11 and 12 below.

ここで、Qjは、図２６に示すように、潜時に相当する区間のデータ数である

Here, Qj is the number of data in the interval corresponding to the latency, as shown in FIG.

[Calculation of movement amount]
Phasej's horizontal/vertical eye movements mjx,mjy are the average value (xj-+1, yj-+1) of Phasej+1 and the representative value of eye position in Phasej (v-jx,v -jy) using the following equations 13 and 14.

[Calculation of strabismus amount and oblique amount]
In CUT, strabismus occurs when one eye is shielded and the opposite eye moves, and when the shielded eye moves when the eye is unshielded, it is oblique. The eye misalignment amount calculation unit 2032 of the evaluation unit 203 calculates a prism amount (amount of eye misalignment) based on the eye movement amount calculated by the eye position measurement unit 202 . The calculation method is the same as that of ACT in [Calculation of prism amount] described above. In CUT, the amount of strabismus/amount of obliqueness calculation unit 2031 of the evaluation unit 203 calculates the amount of obliqueness and the amount of obliqueness, respectively. The strabismus amount of each eye uses the average of the prism amount of the own eye when the other eye is blocked, and the squint amount of each eye uses the average of the prism amount of the Phase when the own eye changes from blocking to opening. Table 3 shows the phase of the amount of prism used for each calculation. The corresponding relationships are shown in FIGS. 27 and 28 (for the right eye, the movement in the direction of the dotted line single arrow is compared, and for the left eye, the movement in the direction of the solid line single arrow is compared). In this way, the amount of strabismus and the amount of squint of the left eye and the amount of squint of the right eye are obtained, and the sum of the amounts of both eyes is taken as the amount of squint and the amount of squint of both eyes.

[Determination of direction of deviation]
The eye misalignment direction calculation unit 2033 of the evaluation unit 203 calculates the direction of eye misalignment. The direction of deviation is based on the left eye. Values DIRx and DIRy used to determine the direction were determined as follows, based on the magnitude of the amount of strabismus and obliqueness obtained in the previous section.
(1) When the amount of strabismus is larger

（２）斜位量の方が大きい場合

(2) When the oblique amount is larger

いずれの場合についても、水平方向については、DIRxの値が負の場合は、外斜視(位)、正の場合は内斜視(位)と判定する。垂直方向については、DIRyの値が正の場合は、上斜視(位)、負の場合は下斜視(位)と判定する。

In any case, in the horizontal direction, if the value of DIRx is negative, it is determined as exotropia (position), and if it is positive, it is determined as esotropia (position). As for the vertical direction, when the value of DIRy is positive, it is determined as an upper squint (position), and when it is negative, it is determined as a lower squint (position).

[Determination of strabismus and oblique position]
CUT is primarily a test to distinguish between strabismus or obliques. Therefore, based on the amount of squint and the amount of squint calculated in the previous section, the amount of squint/slant squint calculation unit 2031 of the evaluation unit 203 calculates squint when the amount of squint is 1Δ or more, If the positional quantity is 1△ or more, it is judged as oblique.

The evaluation unit 203 outputs at least one of the calculated amount of prism (amount of eye misalignment) and direction of eye misalignment, amount of strabismus/horizontal misalignment, and determination result of strabismus/horizontal misalignment. For example, these calculation or determination results are stored in a storage area (not shown). Alternatively, these calculation or determination results may be displayed on the display of the HMD 10 or the display (not shown) of the controller 20 or the like.

(Experimental example)
We evaluated the performance of the implemented eye position inspection system through verification experiments. In the verification experiment, the test results quantified by the eye position abnormality detection system of the present invention and the quantitative eye position test using the Maddox rod (hereinafter abbreviated as Maddox test) are compared.

[Experiment contents]
In the verification experiment, a total of 16 students and employees of the Faculty of Engineering, University of Miyazaki, were used as subjects, using an eye position inspection system in which the inspection method of the present invention was implemented.
The experimental procedure is as follows.
(1) Preliminary explanation to the subject Before the examination, the subject was explained the reason and significance of the study and the safety of the machine to be used. Since the eyeball data is personal information, it was obtained with the consent of the subjects under the conditions that it should be published in a form that cannot identify the individual, and that the data can be discarded at any time at the request of the subject.
(2) Maddox test Next, a Maddox small culm test was performed, and the obtained total displacement amount was regarded as a true value for comparison with the test by the system of the present invention.
(3) ACT inspection by the system of the present invention Next, an ACT inspection was performed by the system of the present invention, and the total amount of displacement was quantified.
(4) CUT inspection by the system of the present invention Finally, a CUT inspection was performed by the system of the present invention to quantify the amount of strabismus and the amount of obliques. The results obtained with the system of the invention were compared with the results obtained with the Maddox test. In CUT, the total amount of deviation was defined as the sum of the amount of strabismus and the amount of oblique.

[Evaluation of system effectiveness]
The effectiveness evaluation result of the eye position abnormality detection system of the present invention will be described.
[ACT]
Table 4 shows the total horizontal (x) deflection in ACT with the Maddox test and the system of the present invention. All units are prism diopters (Δ). Exotropia (position) is denoted as XT/XP, and esotropia (position) is denoted as ET/EP. The quantification result in the proposed system is the value rounded to the third decimal place. In Table 4, those quantified as exotropia (position) of 1 △ or more (deemed as abnormal) are bracketed [ ], and those quantified as esotropia (position) of 1 △ or more are bracketed ( ). is shown.

In order to evaluate the effectiveness of the system of the present invention, as shown in Table 5, the average error between the total deviation amount of the Maddox test and the total deviation amount of the proposed system, the correlation coefficient, and the significant difference. did The error simply evaluates how close the quantification is to the true value. As for the correlation coefficient, since the calculation method of the total deviation amount is different between the Maddox test and the proposed system, quantification is performed by the method of the proposed system to evaluate whether there are any problems. Spearman's rank correlation coefficient was used as the correlation coefficient because it is difficult to imagine a normal distribution. Finally, we examine significant differences to assess whether the correlation is correct.

The average error between the total deviation of the Maddox test and the total deviation of the proposed system was 0.61△, less than 1△, and a value close to the correct answer was obtained. Further, when the Spearman's rank correlation coefficient was obtained between the results of the Maddox test and the system of the present invention, a very strong correlation of 0.956 was obtained as shown in FIG. Furthermore, "null hypothesis H0: the value of the quantification result by the Maddox test has no correlation with the quantification result by the system of the present invention", "alternative hypothesis H1: the value of the quantification result by the Maddox test is the quantification result by the system of the present invention. There is a correlation of ”, and the p-value was calculated to examine the significant difference. As a result, the p-value was 2.11 × 10-4 (<0.01), and there was no significant difference between the prism amount quantified by Maddox and the prism amount quantified by the system of the present invention. The quantification results for the x) direction have a strong correlation with the real clinical test, the Maddox test. From these facts, it is considered that the quantification by ACT in the system of the present invention in the horizontal (x) direction was shown to be useful.

[CUT]
Table 6 shows the total deviation for the Maddox test and the horizontal (x) direction strabismus, oblique and total deviation for CUT by the system of the present invention. In CUT, the sum of the amount of strabismus and the amount of oblique is taken as the total amount of deviation. Ideally, the total amount of deviation should be the amount of strabismus + the amount of obliqueness (amount of strabismus + amount of obliqueness = result of Maddox test), but this is rarely the case depending on the presence or absence of fusion. Therefore, the CUT in the system of the present invention was evaluated by obtaining the correlation between Maddox and the amount of strabismus + the amount of obliques.

Table 7 shows the result of testing between the results of the Maddox test and the results of the system of the present invention based on Table 6 using the Spearman's rank correlation coefficient test. The average absolute value of error for all displacements was 1.50. In addition, the correlation coefficient was 0.810 and the p-value was 1.71×10 −3 (<0.01), and there was no significant difference, so it can be said that there is a strong correlation (see FIG. 40). The results of quantification in the horizontal (x) direction by CUT in the system of the present invention have a large error from the Maddox test, which is an actual clinical test, but it is considered that large errors are quantified properly.

The present invention provides a simple examination support system that helps screen for eye position abnormality.
That is, in the eye position abnormality detection system 1 according to the embodiment of the present invention, an examination room is created using VR, a visual target is presented in the examination room, and the eye position is acquired while performing the open shielding operation. Using a virtual environment eliminates the need to prepare a large room in the real world, making it possible to conduct inspections in a small space. In addition, the HMD 10 to be used does not give a sense of discomfort when worn, and the burden on the subject is small. Furthermore, by constructing the system on VR, it is possible to completely control the presentation of visual targets by the HMD 10, the opening and closing of the eyes, the acquisition of the eye position, the analysis, etc. by the program on the controller 20 side, so that the inspection can be automated and labor-saving. It has a remarkable effect of being able to

1 eye position abnormality detection system 10 HMD
101 Eyeball data acquisition unit 102 Calibration unit 1001 Camera 1002 Infrared sensor 20 Controller 201 Examination execution unit 202 Eye position measurement unit 203 Evaluation unit 2031 Amount of strabismus/horizontal misalignment calculation unit 2032 Displacement amount calculation unit 2033 Displacement direction calculation unit Department

Claims (12)

An eye position abnormality detection system using an HMD with a line-of-sight tracking function,
The eye position abnormality detection system includes the HMD (head-mounted display) with eye-tracking function, and a controller for controlling the inspection operation of the HMD with eye-tracking function,
The controller creates a virtual examination environment including a virtual examination room that is a virtualization of a real examination room, indices displayed in the virtual examination room, and shields for shielding the left and right eyes, respectively, on the HMD. An eye position abnormality detection system characterized by including an examination execution part to be displayed on. The procedure for shielding or removing the left and right eyes with the shield is configured to be performed according to at least either ACT (Alternating-Cover Test) or CUT (Cover-Uncover Test),
2. The eye of claim 1, wherein the act of shielding or removing is configured to be performed by either making the occluder visible or invisible placed directly in front of the left or right eye. position anomaly detection system. The controller is
an eye position measurement unit that acquires eyeball data indicating movement of the left and right eyes accompanying the action of shielding or removing the left and right eyes;
an evaluation unit that quantifies and calculates the degree of strabismus or oblique position based on the eyeball data;
The evaluation unit
In the examination by the ACT, the total amount of deviation of both eyes is calculated,
In the inspection by the CUT, the amount of oblique deviation or the amount of oblique deviation is calculated,
3. The eye position abnormality detection system according to claim 2, wherein a determination of strabismus or oblique is performed based on the total deviation amount, the oblique deviation amount, and the oblique deviation amount. 4. The eye position abnormality detection system according to claim 3, wherein the total amount of deviation, the amount of oblique deviation, and the amount of oblique deviation are calculated by prism amounts. The eye position measurement unit
and removing invalid data from the data obtained from the left and right eye movements, or correcting the effect of latency on the data obtained from the left and right eye movements. The eye position abnormality detection system according to claim 4. The HMD further has a calibration section,
The calibration unit
6. The eye position abnormality detection system according to any one of claims 1 to 5, wherein at least one of adjustment of the mounting position of the HMD, adjustment of the interpupillary distance, and correction of line-of-sight information is executed. . An eye position abnormality detection method using an HMD with a line-of-sight tracking function, comprising:
A controller that controls the inspection operation of the HMD with eye-tracking function,
An examination in which a virtual examination environment comprising a virtual examination room that is a virtualization of a real examination room, indices displayed in the virtual examination room, and shields for shielding the left and right eyes is displayed on the HMD. An eye position abnormality detection method, comprising an execution step. In the inspection performing step, the controller
The procedure of shielding or removing the left and right eyes with the shield is performed according to at least either ACT (Alternating-Cover Test) or CUT (Cover-Uncover Test),
8. The eye position abnormality detection method according to claim 7, wherein the operation of shielding or removing is performed by either making the shield placed in front of the left and right eyes visible or invisible. the controller
an eye position measurement step of acquiring eyeball data indicating the movement of the left and right eyes accompanying the action of shielding or removing the left and right eyes;
an evaluation step of quantifying and calculating the degree of strabismus or oblique position based on the eyeball data;
In the evaluating step, the controller:
In the examination by the ACT, the total amount of deviation of both eyes is calculated,
In the inspection by the CUT, the amount of oblique deviation or the amount of oblique deviation is calculated,
9. The eye position abnormality detection method according to claim 8, wherein a determination of strabismus or oblique is performed based on the total deviation amount, the oblique deviation amount, and the oblique deviation amount. The eye position abnormality detection method according to claim 9, wherein the total deviation amount, the oblique deviation amount, and the oblique deviation amount are calculated by prism amounts. In the eye position measurement step, the controller
and removing invalid data from the data obtained from the left and right eye movements, or correcting the effect of latency on the data obtained from the left and right eye movements. The eye position abnormality detection method according to claim 10. A program for causing a computer to execute the method according to any one of claims 7 to 11.

Images