KR102724897B1 - System for providing augmented realty - Google Patents

Abstract

An augmented reality providing system according to one embodiment of the present invention may include a mobile device including at least two photographing modules arranged to be spaced apart in the longitudinal direction of a main body, and a head mounted display device mounted on the mobile device and including at least two photographing modules arranged to be spaced apart from each of the at least two photographing modules by a critical distance.

Description

{SYSTEM FOR PROVIDING AUGMENTED REALTY}

The present invention relates to an augmented reality providing system.

Augmented reality is a branch of virtual reality that uses computer graphics techniques to synthesize virtual objects or information into the real environment to make them appear as if they were in the original environment.

Augmented reality technology is a technology for interaction between humans and computers. Using augmented reality technology, it is possible to simulate real-world scenes by superimposing virtual objects on them. Through the augmented reality system, users can feel the direct sense of reality they experienced in the objective physical world, and can experience things that cannot be experienced in the real world by overcoming space, time, and other objective constraints. Recently, with the rapid development of augmented reality technology, the application range is becoming increasingly wider, and the content display method is also becoming very diverse.

The task of the present invention is to provide an augmented reality providing system capable of implementing three-dimensional augmented reality.

An augmented reality providing system according to one embodiment of the present invention may include a mobile device including at least two photographing modules arranged to be spaced apart in the longitudinal direction of a main body, and a head mounted display device mounted on the mobile device and including at least two photographing modules arranged to be spaced apart from each of the at least two photographing modules by a critical distance.

According to one embodiment of the present invention, an augmented reality image can be corrected according to distance information between a target object and the left and right eyes, thereby providing a three-dimensional and realistic augmented reality.

FIG. 1 is an exemplary diagram showing an augmented reality providing system according to one embodiment of the present invention.
FIG. 2 is a block diagram showing an augmented reality providing system according to one embodiment of the present invention.

The detailed description of the present invention set forth below refers to the accompanying drawings which illustrate specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention, while different from one another, are not necessarily mutually exclusive. For example, specific shapes, structures, and features described herein may be implemented in other embodiments without departing from the spirit and scope of the invention. It should also be understood that the positions or arrangements of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not intended to be limiting, and the scope of the invention is defined only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if any. Like reference numerals in the drawings designate the same or similar functionality throughout the several aspects.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings so that a person having ordinary skill in the art to which the present invention pertains can easily practice the present invention.

FIG. 1 is an exemplary diagram showing an augmented reality providing system according to one embodiment of the present invention.

An augmented reality providing system according to one embodiment of the present invention may include a mobile device (110) and a head mounted display (HMD) device (120) equipped with the mobile device (110).

A mobile device (110) may include a display unit provided on the front of the main body to provide an augmented reality image and at least two shooting modules (111, 112) provided on the rear of the main body to capture an actual scene including a target object.

At least two shooting modules (111, 112) may include a first shooting module (111) and a second shooting module (112), and each of the first shooting module (111) and the second shooting module (112) may capture a real scene including a subject from different viewpoints. The first shooting module (111) may correspond to the left eye, and the second shooting module (112) may correspond to the right eye, and an image generated from the first shooting module (111) may be referred to as a low left-eye image, and an image generated from the second shooting module (112) may be referred to as a low right-eye image.

The first shooting module (111) and the second shooting module (112) can be arranged to be spaced apart from each other by a reference distance in the longitudinal direction of the main body, and an augmented reality image can be generated from images generated from each of the first shooting module (111) and the second shooting module (112).

For example, each of the first shooting module (111) and the second shooting module (112) may include a high-resolution camera of 12 million pixels or more and may include pixels in the form of RGB (Red/Green/Blue).

The HMD (Head mounted Display) device (120) is an image display device that can be worn on the head like glasses and allows the user to enjoy large images, and can provide a direct sense of reality by outputting images directly in front of the user's eyes. The HMD device (120) can include an accommodation space, and a mobile device (110) can be mounted on the HMD device (120) through the accommodation space.

The HMD device (120) may include at least two lenses that provide an augmented reality image provided on the display unit of the mobile device (110) to the user. The at least two lenses may include three-dimensional lenses, and the augmented reality image may be converted into a three-dimensional image through the three-dimensional lenses and provided to the user. One of the at least two image lenses may provide a left-eye three-dimensional image, and the other may provide a right-eye three-dimensional image.

The HMD device (120) may include at least two photographing modules (121, 122) each provided on the outside of a receiving space for mounting a mobile device (110).

At least two shooting modules (121, 122) may include a third shooting module (121) and a fourth shooting module (122).

Each of the third shooting module (121) and the fourth shooting module (122) may correspond to each of the first shooting module (111) and the second shooting module (112). The corresponding pair of shooting modules may be arranged at a critical distance apart from each other based on the longitudinal direction of the mobile device (110), and for example, the critical distance may be 65 mm.

Distance information of the target object and the left and right eyes can be generated from images generated from each of the corresponding pair of shooting modules.

Referring to FIG. 1, the first shooting module (111) of the mobile device (110) and the third shooting module (121) of the HMD device (120) may correspond to each other, and the second shooting module (112) of the mobile device (110) and the fourth shooting module (122) of the HMD device (120) may correspond to each other.

Distance information between the first photographing module (111) corresponding to the left eye and the target object can be generated from images generated from a corresponding pair of first photographing modules (111) and third photographing modules (121), and distance information between the second photographing module (112) corresponding to the right eye and the target object can be generated from images generated from a corresponding pair of second photographing modules (112) and fourth photographing modules (122).

For example, each of the third shooting module (121) and the fourth shooting module (122) provided in the HMD device (120) may include a VGA (Video Graphic array) camera and may include pixels in the BW (Black/White) format.

The HMD device (120) may include a connection port electrically connected to the mobile device (110). The port of the HMD device (120) is provided at one side of the receiving space, so that when the HMD device is mounted in the receiving space, it can be coupled with the mobile device (110). In order to be coupled with various types of mobile devices (110), the HMD device (120) may additionally include an adapter that mediates the connection port and the mobile device (110). The adapter may be detachably attached to the connection port.

FIG. 2 is a block diagram showing an augmented reality providing system according to one embodiment of the present invention.

An augmented reality providing system (200) according to one embodiment of the present invention may include a photographing unit (210), a detection unit (220), a distance information generating unit (230), an image processing unit (240), and a display unit (250). Among the photographing unit (210) of FIG. 2, the first photographing module (211) and the second photographing module (212) may be provided in the mobile device (110) of FIG. 1, and the third photographing module (213) and the fourth photographing module (214) may be provided in the HMD device (120) of FIG. 1. In addition, the detection unit (220), the distance information generating unit (230), the image processing unit (240), and the display unit (250) may be implemented by a processor, a host, etc. of the mobile device (110) of FIG. 1.

The photographing unit (210) may include a plurality of photographing modules arranged spaced apart from each other, and the plurality of photographing modules may photograph an actual scene including a subject. The plurality of photographing modules of the photographing unit (210) may photograph an actual scene including a subject to generate a plurality of images.

The shooting unit (210) may include first to fourth shooting modules (211, 212, 213, 214). Each of the first to fourth shooting modules (211, 212, 213, 214) of FIG. 2 may correspond to the first to fourth shooting modules (111, 112, 121, 122) of FIG. 1. The first shooting module (211) may correspond to the left eye, the second shooting module (212) may correspond to the right eye, and the image generated from the first shooting module (211) may be referred to as a low left-eye image, and the image generated from the second shooting module (212) may be referred to as a low right-eye image.

The detection unit (220) can detect a target object from images generated from some of the shooting modules of the shooting unit (210). The detection unit (220) can detect a target object from low left-eye images and low right-eye images generated from each of the first shooting module (211) and the second shooting module (212), and obtain status information of the detected target object. The first shooting module (211) and the second shooting module (212) are implemented in a stereo camera manner, so that target detection can be facilitated. The status information of the target object can include at least one of the position, shape, and movement of the target object.

The distance information generation unit (230) can detect distance information from an image generated by the photographing unit (210). The distance information can include first distance information and second distance information.

The distance information generation unit (230) can generate first distance information from images generated from the first shooting module (211) and the third shooting module (213) provided on the left side among the plurality of shooting modules, and as an example, the first distance information can correspond to distance information between the target object and the left eye. In addition, the distance information generation unit (230) can generate second distance information from images generated from the second shooting module (212) and the fourth shooting module (214) provided on the right side among the plurality of shooting modules, and as an example, the second distance information can correspond to distance information between the target object and the right eye.

The image processing unit (240) can generate left-eye and right-eye augmented reality images from the image generated from the photographing unit (210), the state information of the target generated from the detection unit (220), and the distance information generated from the distance information generating unit (230).

The image processing unit (240) can calculate status information of a virtual image corresponding to an actual image captured by the first shooting module (211) and the second shooting module (212) based on the status information of the target object transmitted from the detection unit (220). The virtual image can be pre-programmed by an augmented reality providing program.

The image processing unit (240) can generate an augmented reality image by synthesizing the virtual image and the actual images captured by the first shooting module (211) and the second shooting module (212) according to the status information of the generated virtual image. The image processing unit (240) can generate a transparent model corresponding to the object based on the status information of at least one of the object and the virtual image, and can generate an augmented reality image by overlapping the transparent model with the object to generate a covering effect between the virtual image and the object. In addition, the image processing unit (240) can render the generated augmented reality image. Here, the rendering can be understood as a process of changing the augmented reality image into a three-dimensional image by considering the status information of at least one of the object and the virtual image.

The image processing unit (240) can correct the augmented reality image into a left-eye augmented reality image and a right-eye augmented reality image using the distance information generated by the distance information generating unit (230).

The image processing unit (240) can correct the augmented reality image into a left-eye augmented reality image using the left-eye distance information, and can correct the augmented reality image into a right-eye augmented reality image using the right-eye distance information. The image processing unit (240) can provide a realistic three-dimensional image by correcting the augmented reality image using the left-eye distance information and the right-eye information.

The display unit (250) can display left-eye and right-eye augmented reality images generated by the image processing unit (240). The display unit (250) can include at least two display areas. For example, the at least two display areas can include a first display area (251) and a second display area (252), and each of the first display area (251) and the second display area (252) can display left-eye and right-eye augmented reality images.

Although the present invention has been described above with specific details such as specific components and limited examples and drawings, these have been provided only to help a more general understanding of the present invention, and the present invention is not limited to the above examples, and those with common knowledge in the technical field to which the present invention belongs can make various modifications and variations from this description.

Therefore, the idea of the present invention should not be limited to the embodiments described above, and not only the claims described below but also all modifications equivalent to or equivalent to the claims are included in the scope of the idea of the present invention.

100: Augmented Reality Delivery System
110: Mobile devices
111: 1st shooting module
112: 2nd shooting module
120: Head-mounted display device
121: Third Shooting Module
122: 4th shooting module
200: Augmented Reality Delivery System
210: Filming Department
211: 1st shooting module
212: Second shooting module
213: Third Shooting Module
213: 4th shooting module
220: Detection Unit
230: Distance information generation unit
240; Image Processing Unit
250: Display section

Claims (9)

A mobile device comprising at least two photographing modules spaced apart in the longitudinal direction of the body; and
An augmented reality providing system comprising: a head mounted display device having the mobile device mounted thereon and including at least two photographing modules positioned at a critical distance apart from each of the at least two photographing modules;
In the first paragraph,
An augmented reality providing system that generates an augmented reality image from images captured by at least two shooting modules of the mobile device.
In the second paragraph,
Any one of at least two photographing modules of the mobile device corresponds to any one of at least two photographing modules of the head mounted display device,
An augmented reality providing system that corrects the augmented reality image by generating distance information from images captured by the corresponding pair of shooting modules.
In the third paragraph,
An augmented reality providing system wherein the pair of corresponding photographing modules are spaced apart from each other by a critical distance in the longitudinal direction of the mobile device.
In the third paragraph,
An augmented reality providing system that generates left eye distance information from images captured by one of the corresponding pair of shooting modules and generates right eye distance information from images captured by the other pair of shooting modules.
In paragraph 5,
An augmented reality providing system that corrects the augmented reality image into a left-eye augmented reality image and a right-eye augmented reality image using the left-eye distance information and the right-eye distance information.
A photographing unit including first to fourth photographing modules for photographing a target object;
A detection unit that detects an object from images captured by the first and second shooting modules and obtains status information of the detected object;
A distance information generation unit that generates left-eye distance information and right-eye distance information from images captured by the first to fourth shooting modules; and
An image processing unit that generates an augmented reality image according to the state information of the object and corrects the augmented reality image using the left eye distance information and the right eye distance information;
An augmented reality providing system, wherein some of the first to fourth photographing modules are provided in a mobile device, and others are provided in a head mounted display device equipped with the mobile device.
In Article 7,
An augmented reality providing system, wherein the first and second shooting modules are provided in a mobile device, and the third and fourth shooting modules are provided in a head mounted display device.
In Article 8,
An augmented reality providing system in which the left eye distance information is generated from images captured by the first and third shooting modules arranged on the left among a plurality of shooting modules, and the right eye distance information is generated from images captured by the second and fourth shooting modules arranged on the right among a plurality of shooting modules.

Images