KR101947650B1 - Apparatus and method for generating learning image in game engine-based machine learning - Google Patents

Abstract

The present invention relates to an apparatus for generating a learning image of game engine-based machine learning, which generates various game-related images to increase learning performance of a machine learning process, and a method thereof. According to the present invention, the apparatus comprises: a learning data module outputting CAD-based learning data including at least one game object; a high-speed cluster computing module performing pattern learning and machine learning recognizing the game object with respect to the CAD-based learning data in accordance with a neural network model and neural network operation provided from a neural network model unit and a neural network operation framework; and a game engine application module disposed between the learning data module and the high-speed cluster computing module to detect a plurality of 3D CAD models included in the CAD-based learning data and synthesizing an environment simulation image and a data-enhanced image to the detected 3D CAD models to generate a plurality of 2D learning images used for machine learning.

Description

[0001] APPARATUS AND METHOD FOR GENERATING LEARNING IMAGE IN GAME ENGINE-BASED MACHINE LEARNING [0002]

The present invention relates to an apparatus and method for generating a machine learning learning image based on a game engine capable of enhancing learning performance of a machine learning process by generating various game related images.

In general, machine learning (machine learning) is a technology that studies and constructs a system and its algorithms that perform data learning based on data, perform learning and prediction directly, and improve their performance.

Deep learning techniques for image analysis used in the development of existing machine learning systems require tens of thousands to millions of pieces of learning image data, far exceeding the required amount of learning data required for machine learning.

However, in reality, it is not easy to acquire large-capacity learning data due to a variety of factors other than some standard benchmarking data sets which are partially disclosed. Especially, in the case of learning images related to weapon systems such as trams and airplanes, the source of images is very limited due to the specificity of the field, and there are limitations in collecting / acquiring image data for machine learning learning.

An object of the present invention is to provide an apparatus and method for generating a machine learning learning image based on a game engine capable of variously generating a game related image based on a game engine based technology, .

Another object of the present invention is to provide a machine learning learning image generating apparatus based on a game engine capable of providing a data enhancing function based on a game engine and a data enhancing function based on a virtual scene / Method.

According to an aspect of the present invention, there is provided an apparatus for generating a machine learning learning image based on a game engine, the apparatus comprising: a learning data module for outputting CAD-based learning data including at least one game object; A fast cluster computing module for performing pattern learning and machine learning for recognizing game objects according to a neural network model and a neural network operation provided by the neural network modeling unit and the neural network computing framework; And a plurality of 3D CAD models included in the CAD-based learning data, the plurality of 3D CAD models being provided between the learning data module and the high-speed cluster computing module to synthesize the environment simulated image and the data enhancement image And a game engine application module for generating a plurality of 2D learning images to be used for machine learning.

According to an embodiment of the present invention, the environment simulation image includes an environmental condition simulation image representing an environment surrounding a plurality of detected 3D CAD models, a physical condition simulation image representing physical effects and data enhancement to be applied to the 3D CAD model, .

According to an embodiment of the present invention, the game engine application module simulates the environmental conditions selected according to the type of the 3D CAD model or the input conditions using the environment simulation function in the 3D CAD model, Rain, fog, clouds, and the like.

According to the embodiment of the present invention, the game engine application module divides a game object to be used for machine learning based on the movement, size and position of the game object included in the CAD-based learning data, can do.

According to an embodiment of the present invention, the game engine application module simulates a physical condition selected in accordance with the type of each 3D CAD model or input condition using a physical effect function, into a 3D CAD model, Light amount, and texture.

According to an embodiment of the present invention, the game engine application module performs data enhancement based on a game engine with respect to a combination of a 3D CAD model and environment using a data enhancement function, and performs a 3D CAD model And the environment can simulate the game engine's base perspective and the movement of game objects.

According to an embodiment of the present invention, the game engine application module includes a 3D CAD model selection module for selecting a plurality of 3D CAD models according to the type of each 3D CAD model or the input conditions of the operator included in the CAD-based learning data; An environment simulation module for selecting an environment simulation image corresponding to a plurality of 3D CAD models according to the type of each 3D CAD model or the input condition of the operator from the first storage module; A data enhancement module for selecting an enhancement image based on a virtual scene rendering based on a plurality of 3D CAD models according to a type of each 3D CAD model or an input condition of an operator from a second storage module; And a learning image synthesis module for synthesizing the selected 3D CAD model, environment simulation image, and augmented image to generate a final 2D learning image.

According to the embodiment of the present invention, the 3D CAD model selection module recognizes the type of the 3D CAD model included in the CAD-based learning data, and notifies the environment simulation module and the data enhancement module of the type of each 3D CAD model have.

According to an aspect of the present invention, there is provided a method of generating a machine learning learning image based on a game engine, the method comprising: generating a machine learning learning image based on a type of each 3D CAD model included in CAD- Selecting a plurality of 3D CAD models to be used for running; Selecting an environment simulation image corresponding to a plurality of 3D CAD models from the first storage module according to a type of each 3D CAD model or an input condition of an operator; Selecting an enhancement image corresponding to a plurality of 3D CAD models from the second storage module according to a type of each 3D CAD model or an input condition of an operator; And generating a plurality of 2D learning images by synthesizing the selected 3D CAD model, environment simulation image, and augmented image.

According to an embodiment of the present invention, the environment simulated image includes an environmental condition simulated image representing an environment surrounding a plurality of 3D CAD models, and a physical condition simulated image representing physical effects and data enhancement to be applied to the 3D CAD model .

According to an embodiment of the present invention, the step of generating the plurality of 2D learning images may include the step of simulating the environmental conditions selected according to the type of each 3D CAD model or the input condition using the environment simulation function into the 3D CAD model And the environmental conditions may include a background environment such as snow, rain, fog, and clouds.

According to an embodiment of the present invention, the step of generating the plurality of 2D learning images may include the step of simulating the physical condition selected in accordance with the type of each 3D CAD model or the input condition using the physical effect function into the 3D CAD model And the physical condition may include a light ray, a light amount, and a texture.

According to an embodiment of the present invention, the step of generating the plurality of 2D learning images includes performing data enhancement based on a game engine with respect to a combination of a 3D CAD model and environment using a data enhancement function; And a step of simulating the perspective of the base of the game engine and the movement of the game object in a combination of the 3D CAD model and environment using the perspective data enhancement function.

According to an embodiment of the present invention, the step of selecting the plurality of 3D CAD models includes recognizing the types of the plurality of 3D CAD models included in the CAD-based learning data, And a data enhancement module.

According to the embodiment, the present invention selects a 3D CAD model of interest (eg, a game object) from the CAD-based learning data, and then applies the background environment surrounding the selected 3D CAD model to effects such as snow / rain / And a variety of game-related 2D learning images having a myriad of environments, motions, and perspectives can be generated by simulating perspective or motion based on a game engine together with general data augmenting functions such as cropping, It is possible to solve the problem of insufficient learning data inherent in the running process.

In addition, the present invention provides an interface for creating various game environments and conditions when composing a plurality of game environment images by synthesizing an environment simulated image and a data enhancement image on a 3D CAD model, Can be generated.

1 is a block diagram of an apparatus for generating a machine learning learning image based on a game engine according to an embodiment of the present invention;
2 is a detailed block diagram of the game engine application module 130 shown in FIG.
3 is a flowchart illustrating an operation of a game-engine-based machine learning learning image generation apparatus according to an embodiment of the present invention.
4 is a drawing showing an example of a simulated environment image.
5A and 5B are views showing an example of an image for data enhancement.
6 is a view illustrating an example of generating various game environment images to be used for machine learning in a game engine application module according to an embodiment of the present invention.

It is noted that the technical terms used herein are used only to describe specific embodiments and are not intended to limit the invention. The suffix " module " and " part " for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

And, the singular expressions used in the specification include plural expressions unless the context clearly indicates otherwise. In this specification, " comprises " Or " include. &Quot; Should not be construed to encompass the various components or steps described in the specification, and some of the components or portions may not be included, or may include additional components or steps And the like.

Further, in the description of the technology disclosed in this specification, a detailed description of related arts will be omitted if it is determined that the gist of the technology disclosed in this specification may be obscured.

The present invention provides a learning data mass generating structure that can improve the learning efficiency of machine learning by applying a game engine based technology in order to solve the lack of learning data inherent in the existing machine learning process, We propose a method and apparatus for generating a machine learning learning image that can create conditions.

1 is a block diagram of a machine learning learning image generation apparatus based on a game engine according to an embodiment of the present invention.

1, a learning performance enhancing apparatus for learning a machine learning according to the present invention includes a learning data module 100, a neural network model unit 110, a neural network operation framework 120, a game engine application module 130, A high-speed cluster computing module 140, and a training model predicting unit 140.

The learning data module 100 provides learning data for performing a machine function. The learning data may be three-dimensional (3D) CAD-based data. The three-dimensional (3D) CAD-based data may include at least one game object.

The neural network modeling unit 110 may provide a predefined neural network model and the neural network computing framework 120 may provide neural network operations to perform machine learning and prediction on the training data.

The game engine application module 130 selects a 3D CAD model based on input 3D CAD-based learning data, and performs an environment simulation function for providing a physical environment similar to an actual one to the selected 3D CAD model, A data enhancement function simulating movements is applied to create various game-related images for a machine run.

In one embodiment, the game engine application module 130 uses the environment simulation function, which is one of the functions of the game engine, to determine the environmental condition selected according to the type of each 3D CAD model or the input condition, CAD model. The environmental conditions may include background conditions such as snow, rain, fog, clouds. The game engine application module 130 can simulate the physical condition selected according to the type of the 3D CAD model or the input condition using the physical effect function, which is one of the functions of the game engine, in the 3D CAD model. The physical condition may include light rays, light amount, and texture.

In addition, the game engine application module performs data enhancement based on a game engine on a combination of a 3D CAD model and environment by using a data enhancement function, which is one of the functions of the game engine, The enhancement function can be applied to simulate the perspective of the base of the game engine and the movement of the game object.

The fast cluster computing module 140 applies the learning image generated through the game engine application module 130 to the neural network model provided by the neural network model unit 120 and the neural network operation provided by the neural network operation framework 110, Machine learning can be performed on the learning image.

The training-model predicting unit 150 may determine whether to use the training model for the learning im- ages based on the treadmill and prediction performed in the fast cluster computing module 140. [

2 is a detailed block diagram of the game engine application module 130 shown in FIG.

2, the game engine application module 130 includes a 3D CAD model selection module 10 for selecting a 3D CAD model from CAD-based learning data and a background environment for implementing a physical environment effect A data enhancement module 30 for enhancing data based on a dynamic virtual scene rendering based on various effects of the game engine, such as perspective and motion, a 3D CAD model, a simulated 3D background environment, And a learning image synthesis module 40 for synthesizing the enhanced 3D virtual scene to generate a final 2D learning image.

The 3D CAD model selection unit 10 selects a plurality of 3D CAD models (eg, game objects) from CAD-based learning data according to input conditions (eg, a specific game object) CAD model can be selected. Here, the random selection indicates that an arbitrary game object is selected from a plurality of game objects included in the CAD-based learning object. In some implementations, the randomization may be the most mobile game object or the largest displayed game object. In addition, the term "selection" includes detecting a 3D CAD model (eg, game object) from CAD-based learning data or transforming the angle, direction and size of the detected 3D CAD model (eg, game object) . The parameters associated with this selection may consist of a number of items of setting menus that the user can select on the user interface.

The environment image storage module 20a stores a plurality of simulated environment images surrounding a 3D CAD model (e.g., a game object), and the simulated environment images are arbitrary images such as snow, rain, Such as the weather, rays, and texture of the image.

The enhancement effect storage module 30a may store various effects to be given to a 3D CAD model (e.g., a game object) and dynamic virtual scenes such as perspective and motion.

The environment image storage module 20a and the enhancement effect storage module 30a are separately shown for convenience of explanation, but are not limited thereto. That is, a plurality of environmental images and a plurality of digital noises may be stored in different storage areas, respectively, in one memory. In this case, the memory may not be shown in FIG.

The environment simulating module 20 and the data enhancing module 30 may be configured to select the 3D CAD model based on the conditions inputted by the operator or based on the type of the 3D CAD model The environment image, the enhancing effect, and / or the image corresponding to the environmental image storage module 20a and the enhancement effect storage module 30a can be selected. Here, the selection is performed by accessing the environment image storage module 20a and the enhancement effect storage module 30a to select a desired environment image and a reinforcement effect and / or an image from the stored plurality of environment images and the enhancement effect and / or enhancement image it means.

The learning image synthesis module 40 includes an image synthesis function to generate a final 2D training image by synthesizing a selected 3D CAD model (eg, game object), a simulated texture image, and a virtual enhancement effect and / do.

The operation of the machine learning learning image generating apparatus based on the game engine according to the embodiment of the present invention will be described with reference to the accompanying drawings. Particularly, since the game engine application module 130 is a feature of the present invention, the present invention will be mainly described.

 FIG. 3 is a flowchart illustrating an operation of a game-engine-based machine learning learning image generating apparatus according to an embodiment of the present invention. FIG. 4 is an example of a simulated environment image, .

Referring to FIG. 3, the game engine application module 130 may receive CAD-based learning data to be used for machine learning from the learning data module 100. The CAD-based learning data may be randomly generated through a screen capture on a game engine, and may include at least one 3D CAD model (e.g., a game object) (SlOO).

The game engine application module 130 recognizes / divides at least one or more 3D CAD models (eg, game objects) to be used for a treadmill from CAD-based learning data by using an image dividing function, A plurality of 3D CAD models may be selected by transformation (angle, direction) (S110).

In one embodiment, the 3D CAD model selection module 10 of the game engine application module 130 extracts a plurality of 3D CAD models from CAD-based learning data based on conditions (eg, game objects to be selected) Can be selected. Here, the condition indicates the type of the game object to be selected, but may include additional angles and directions for the 3D CAD model when the number of selected 3D CAD models is small. In this case, the game engine application module 130 may display a separate user interface (UI) for a user selection or provide a menu including a plurality of selection items.

In another embodiment, the 3D CAD model selection module 10 of the game engine application module 130 can select a plurality of 3D CAD models to be used for machine learning from CAD-based learning data through the object recognition technique. For example, the 3D CAD model selection module 10 recognizes game objects (eg, weapons, characters, monsters) from a learning data source (3D CAD model) And a plurality of 2D learning images can be generated by changing the directions differently. For example, the 3D CAD model selection module 10 can select one of the game objects with the highest motion among the CAD-based learning data, the game object with the widest display, or the gain object disposed at the center of the screen as a 3D CAD model However, the present invention is not limited thereto. That is, a predetermined 3D CAD model set in advance may be automatically selected according to the type of game (strategy game, battle game).

The game engine application module 130 selects at least one texture image and a virtual enhancement effect and / or augmentation image to be applied to the plurality of 3D CAD models selected by the 3D CAD model selection section 10 using the image selection function (S120, S130). The selection means selecting at least one image that matches a predetermined 3D CAD model among a plurality of texture images stored in a storage medium (e.g., storage module) and a virtual enhancement effect and / or augmentation image.

In one embodiment, the environment simulation module 20 stores at least one environmental image related to the game object on the basis of the type information (e.g., tram) of the game object inputted from the 3D CAD model selection section 20, Can be selected from the module 20a. The environmental image may include simulated images of environmental conditions (e.g., snow, rain, fog, clouds, etc.) and physical conditions (e.g., light, light, texture), as shown in FIG. 5A and 5B, the data augmenting module 30 may be configured to generate a 3D CAD model based on the type information (eg, tram) of the game object input from the 3D CAD model selecting unit 20, At least one game engine based data augmented image and / or a virtual scene perspective / motion based augmented image may be selected.

Accordingly, the game engine application module 130 can use the image synthesis function to add the selected game engine-based environment image, data enhancement image, and / or virtual scene (s) to the selected plurality of 3D CAD models A variety of 2D game environment images can be generated by synthesizing the perspective / motion based enhancement images (S140). The generated various 2D game environment images are provided to the rear end 140, 150 of FIG. 1 and used for machine learning.

FIG. 6 shows an example of generating various game environment images to be used for machine learning in the game engine application module according to the embodiment of the present invention.

6, the learning image synthesis module 40 of the game engine application module 130 generates a plurality of selected 2D (3D) CAD models (eg, game objects) using the image compositing function The learning image is combined with at least one or more texture simulated images and data enhancement images to generate various 2D training images.

In other words, the learning image synthesis module 40 effects effects on the background environment surrounding the 3D CAD model such as snow / rain / fog through the 3D CAD model and texture simulated image synthesis (combination) In the synthesis of the background, the light / texture and so on are further simulated, and simulated physical environment effects are realized similar to the actual environment. In addition, the learning image synthesis module 40 can simulate a perspective or a motion based on a game engine together with a general data enhancement function such as a crop to enable various situations to be produced. Therefore, it can produce countless situations through myriad environments, movements, and perspectives, and generates a large number of learning images through screen capturing on the game engine. The learning image synthesis module performs the function of capturing a 2D image in a 3D situation and generating a learning image. That is, the learning image synthesis module 40 performs a function of capturing a 2D image in a 3D situation to generate a learning image.

The high speed cluster computing module 140 applies various game learning images generated through the game engine application module 130 to the neural network model provided by the neural network model unit 120 and the neural network operation provided by the neural network operation framework 110 Perform machine learning. That is, the high-speed cluster computing module 140 extracts features necessary for game object recognition from the various game-related 2D learning images generated in the game engine simulation model 130, and generates a neural network model provided from the neural model unit 110 And performs pattern learning on the extracted feature using the neural network operation provided by the neural network operation framework 120. [ Thereafter, the training model predicting unit 150 can determine the game object using the pattern learning result performed by the high-speed cluster computing module 140, and determine whether to apply the training model to the learning image.

As described above, the present invention selects a 3D CAD model of interest (eg, a game object) from the CAD-based learning data, and then displays the background environment surrounding the selected 3D CAD model with effects such as snow / rain / And can generate various game related 2D learning images having countless environments, movements, and perspectives by simulating perspective or motion based on game engine together with general data augmenting function such as cropping, There is an advantage that the problem of lack of learning data inherent in the process can be solved.

In addition, the present invention provides an interface for creating various game environments and conditions when composing a plurality of game environment images by synthesizing an environment simulated image and a data enhancement image on a 3D CAD model, thereby providing a more diverse learning image for machine learning There is an effect that can be generated.

The apparatus and method for generating a machine learning learning image based on the game engine according to the present invention can be applied to the embodiments described above, but the embodiments can be applied to other specific embodiments without changing their technical ideas or essential features It will be understood that the invention may be practiced otherwise than as described. Therefore, it should be understood that the above-described embodiments are to be considered in all respects as illustrative and not restrictive.

10: 3D CAD model selection section 20: Environment simulation module
20a: environment image storage module 30: data enhancement module
30a: Strengthening effect storage module 40: Learning image synthesis module
100: learning data module 110: neural network model part
120: Neural network operation framework 130: Game engine application module
140: high-speed cluster computing module 150: training model prediction unit

Claims (14)

A learning data module for outputting CAD-based learning data including at least one game object;
A fast cluster computing module for performing pattern learning and machine learning for recognizing game objects according to a neural network model and a neural network operation provided by the neural network modeling unit and the neural network computing framework; And
A plurality of 3D CAD models included in the CAD-based learning data are detected between the learning data module and the high-speed cluster computing module, and the environment simulation image and the data enhancement image are synthesized And a game engine application module for generating a plurality of 2D learning images to be used for machine learning,
The game engine application module
A 3D CAD model selecting module for selecting a plurality of 3D CAD models according to the type of each 3D CAD model included in the CAD-based learning data or the input condition of the operator,
The 3D CAD model selection module includes:
And generating the plurality of 2D learning images by differently changing an angle, a perspective and a direction according to the type of the game object after recognizing the game object from the CAD-based learning data,
And selecting a game object having the most movement from the CAD-based learning data as the 3D CAD model. 2. The method of claim 1,
An environment condition simulation image representing an environment surrounding a plurality of detected 3D CAD models, and a physical condition simulation image representing a physical effect and data enhancement to be applied to the 3D CAD model. Image generating device. The method of claim 1, wherein the game engine application module
Using the environmental simulation function, the selected environmental conditions are simulated on the 3D CAD model according to the type of each 3D CAD model or inputted conditions,
Wherein the environmental condition includes a background environment such as an eye, a rain, a mist, and a cloud. 4. The method of claim 3, wherein the game engine application module
Based on the movement, size, and position of the game object included in the CAD-based learning data, a 3D CAD model is selected by dividing an interested game object to be used for machine learning, . The method of claim 1, wherein the game engine application module
The physics effect function is used to simulate the selected physical conditions in 3D CAD model according to the type of each 3D CAD model or input condition,
Wherein the physical condition includes a light ray, a light amount, and a texture. The method of claim 1, wherein the game engine application module
(a) Data enhancement based on the game engine is performed for a combination of the 3D CAD model and environment using the data enhancement function,
(b) a game engine based machine learning learning image generation device characterized by simulating the perspective of the game engine and the movement of the game object in a combination of a 3D CAD model and environment using a perspective data enhancement function. The method of claim 1, wherein the game engine application module
An environment simulation module for selecting an environment simulation image corresponding to a plurality of 3D CAD models according to the type of each 3D CAD model or the input condition of the operator from the first storage module;
A data enhancement module for selecting an enhancement image based on a virtual scene rendering based on a plurality of 3D CAD models according to a type of each 3D CAD model or an input condition of an operator from a second storage module; And
And a learning image synthesis module for synthesizing the selected 3D CAD model, environment simulation image, and augmented image to generate a final 2D learning image. The method of claim 7, wherein the 3D CAD model selection module
Wherein the type of the 3D CAD model included in the CAD-based learning data is recognized and the type of each 3D CAD model is notified to the environment simulation module and the data enhancement module. Selecting a plurality of 3D CAD models to be used for machine learning according to a type of each 3D CAD model included in CAD-based learning data or an input condition of an operator;
Selecting an environment simulation image corresponding to a plurality of 3D CAD models from the first storage module according to a type of each 3D CAD model or an input condition of an operator;
Selecting an enhancement image corresponding to a plurality of 3D CAD models from the second storage module according to a type of each 3D CAD model or an input condition of an operator; And
And generating a plurality of 2D learning images by synthesizing the selected 3D CAD model, environment simulation image, and augmented image,
The step of selecting the 3D CAD model includes:
A plurality of 3D CAD models are selected according to the type of each 3D CAD model included in the CAD-based learning data or the operator's input condition,
Wherein the generating the plurality of 2D learning images comprises:
And generating the plurality of 2D learning images by changing the angle, perspective, and direction differently according to the type of the game object after recognizing the game object from the CAD-based learning data,
And selecting a game object having the most movement in the CAD-based learning data as the 3D CAD model. 10. The method of claim 9, wherein the environment simulation image
An environment condition simulation image representing an environment surrounding a plurality of 3D CAD models, and a physical condition simulation image representing physical effects and data enhancement to be applied to the 3D CAD model. Way. 10. The method of claim 9, wherein generating the plurality of 2D learning images comprises:
And a step of simulating the selected environmental condition according to the type of each 3D CAD model or the input condition using the environment simulation function into the 3D CAD model,
Wherein the environmental condition includes a background environment such as an eye, a rain, a mist, and a cloud. 10. The method of claim 9, wherein generating the plurality of 2D learning images comprises:
And a step of simulating the selected physical condition in the 3D CAD model according to the type of each 3D CAD model or the inputted condition using the physical effect function,
Wherein the physical condition includes a light ray, a light amount, and a texture. 10. The method of claim 9, wherein generating the plurality of 2D learning images comprises:
Performing data enhancement based on a game engine with respect to a combination of a 3D CAD model and environment using a data enhancement function; And
And a step of simulating the perspective of the base of the game engine and the motion of the game object in a combination of the 3D CAD model and environment using the perspective data enhancement function. The method as claimed in claim 9, wherein the step of selecting the plurality of 3D CAD models
Recognizing the types of the plurality of 3D CAD models included in the CAD-based learning data, and notifying the types of the recognized 3D CAD models to the environment simulation module and the data augmentation module. Based learning method.

Images