KR102510051B1 - Control method of electronic device to determine motion matching using by reference position for each time and joint - Google Patents

Abstract

The present invention relates to a control method of an electronic device that determines motion consistency by using time and reference positions for each joint The control method comprises the steps of: acquiring, by an electronic device, joint reference data in which a reference position range for each joint used to recognize a person's motion from an image is set for each of a plurality of first time sections of the image; identifying, by the electronic device, a plurality of jointed objects of a person from the image; matching, by the electronic device, a joint point to each of the identified plurality of joint objects; determining, by the electronic device, whether the point position of the joint point in a second time interval corresponding to any one 1-1 time interval among the plurality of first time intervals is included in a reference position range of the 1-1 time interval; and determining, by the electronic device, whether the motion of the person is the same as the motion indicated by the joint reference data based on the identification result.

Description

Electronic device control method for determining motion matching using time and reference position for each joint

The present invention relates to a control method of an electronic device that determines motion matching using time and reference positions for each joint, and more particularly, identifies each joint of a person included in an image and matches it with a joint point, and a technique for determining whether a motion presented to the user and a motion of the user match according to whether the reference position for each joint corresponds to a joint point.

As various multi-media systems based on image information develop, demand for technologies that can understand information included in images and apply them to various fields is rapidly increasing. In particular, much attention has been focused on constructing a more convenient interface environment by recognizing human gestures, for example, hand gestures, in an image.

However, in the prior art, there is a problem in that the user's motion is not correctly recognized even though the actual user performed the motion accurately by not determining which motion the motion of the person in the image is using a reference point specialized for the user's body structure. .

Accordingly, there is a demand for a technology capable of recognizing a user's motion while changing a reference point specific to the user's body structure in real time even when the image changes over time.

Korea Patent Registration No. 10-1304321, 2013.08.30.

The problem to be solved by the present invention is to identify each joint of a person included in an image, calculate the joint angle of each joint, and determine whether the calculated joint angle is within the range of joint movable angles of the corresponding joint to determine motion recognition. An object of the present invention is to provide a control method of an electronic device that determines whether a motion is matched using time and reference positions for each joint, which can filter motion recognition noise by determining whether or not to perform motion recognition.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

In the control method of an electronic device for determining motion matching using time and reference positions for each joint according to an aspect of the present invention for solving the above problems, the electronic device is used for recognizing a motion of a person from an image. Obtaining joint reference data in which each reference position range is set for each of a plurality of first time sections of an image, identifying, by an electronic device, a plurality of joint objects of a person from an image, by an electronic device, identifying a plurality of joint objects Matching the joint points with each other, the electronic device, the point position of the joint point of the second time interval corresponding to any one of the 1-1st time intervals among the plurality of 1st time intervals Determining whether the motion is included in the reference position range, and determining, by the electronic device, whether the motion of the person is the same as the motion indicated by the joint reference data based on the confirmation result.

At this time, the step of determining whether the motion of the person is the same as the motion indicated by the joint reference data may include, when the point position of the joint point of the second time period is included in the reference position range of the 1-1st time period. , determining that the motion of the person in the second time period is the same as the motion indicated by the joint reference data in the 1-1st time period.

In the control method of an electronic device according to an aspect of the present invention, if the point position of the joint point of the second time interval is not included in the reference position range of the 1-1st time interval, the electronic device of the second time interval and determining that the motion of the person is not the same as the motion indicated by the joint reference data of the 1-1st time interval.

The method for controlling an electronic device according to an aspect of the present invention further includes synchronizing a data start time of joint reference data and a motion start time when a motion of a person starts in an image to the same time point.

In the control method of an electronic device according to an aspect of the present invention, the electronic device divides an image such that the time length of each section between a plurality of first time sections and the time length of each section between a plurality of second time sections are the same. more includes

In the control method of an electronic device according to an aspect of the present invention, the step of selecting, by the electronic device, any two joint points among a plurality of joint points as reference joint points, the electronic device, determining an image body length from the reference joint points The step of calculating and correcting, by the electronic device, the point positions of the plurality of joint points by using a comparison ratio, which is a ratio of the body length of the video to the body length of the reference video corresponding to the reference joint point.

At this time, in the step of correcting the point positions of the plurality of joint points, the electronic device maintains the direction from the reference joint point to each of the plurality of joint points, but the distance from the reference joint point to each of the plurality of joint points is equal to the comparison ratio. The step of correcting by increasing or decreasing is further included.

Other specific details of the invention are included in the detailed description and drawings.

According to the control method of an electronic device for determining motion matching using time and reference positions for each joint of the present invention, each joint of a person included in an image is identified and matched with a joint point, and the reference positions for each joint are identified and matched with each other. It is possible to accurately process motion recognition by determining whether the motion presented to the user and the motion of the user match according to whether or not corresponds to the joint point.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a configuration diagram of an electronic device according to an embodiment of the present disclosure.
2 is a basic flow diagram according to an embodiment of the present disclosure.
3 is an exemplary view of a state in which a user uses an electronic device according to an embodiment of the present disclosure.
4 is an exemplary view of point matching for a joint object according to an embodiment of the present disclosure.
5 is an exemplary view of an example of a process of determining whether a point location is included in a reference location range by an electronic device according to an embodiment of the present disclosure.
6 is an exemplary view of another example of a process of determining whether a point location is included in a reference location range by an electronic device according to an embodiment of the present disclosure.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, only these embodiments are intended to complete the disclosure of the present invention, and are common in the art to which the present invention belongs. It is provided to fully inform the person skilled in the art of the scope of the invention, and the invention is only defined by the scope of the claims.

Terminology used herein is for describing the embodiments and is not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, "comprises" and/or "comprising" does not exclude the presence or addition of one or more other elements other than the recited elements. Like reference numerals throughout the specification refer to like elements, and “and/or” includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various components, these components are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first element mentioned below may also be the second element within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention belongs. In addition, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless explicitly specifically defined.

The term "unit" or "module" used in the specification means a hardware component such as software, FPGA or ASIC, and "unit" or "module" performs certain roles. However, "unit" or "module" is not meant to be limited to software or hardware. A “unit” or “module” may be configured to reside in an addressable storage medium and may be configured to reproduce one or more processors. Thus, as an example, a “unit” or “module” may refer to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. Functions provided within components and "units" or "modules" may be combined into smaller numbers of components and "units" or "modules" or may be combined into additional components and "units" or "modules". can be further separated.

The spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe a component's correlation with other components. Spatially relative terms should be understood as including different orientations of elements in use or operation in addition to the orientations shown in the drawings. For example, if you flip a component that is shown in a drawing, a component described as "below" or "beneath" another component will be placed "above" the other component. can Thus, the exemplary term “below” may include directions of both below and above. Components may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

In this specification, a computer means any kind of hardware device including at least one processor, and may be understood as encompassing a software configuration operating in a corresponding hardware device according to an embodiment. For example, a computer may be understood as including a smartphone, a tablet PC, a desktop computer, a laptop computer, and user clients and applications running on each device, but is not limited thereto.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a configuration diagram of an electronic device according to an embodiment of the present disclosure.

As shown, the electronic device 100 for determining motion matching using time and reference positions for each joint according to an aspect of the present invention includes a memory 110 and a camera for acquiring an image including a user's motion ( 120), a display 130 that outputs images and contents, a memory 110, a camera 120, a processor 140 connected to the display 130 and the communication unit 150, and a communication unit that communicates with other devices. (150).

In an embodiment, the electronic device 100 includes a server, a smartphone, a tablet personal computer (PC), a mobile phone, a video phone, a laptop PC, and a netbook computer. It may include at least one of a notebook computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a wearable device, a head mounted display (HMD), a portable medical device, and a smart TV. .

Meanwhile, the memory 110 may store various programs and data necessary for the operation of the electronic device 100 . The memory 110 may be implemented as a non-volatile memory 110, a volatile memory 110, a flash-memory, a hard disk drive (HDD), or a solid state drive (SSD).

The processor 140 may control overall operations of the electronic device 100 using various programs stored in the memory 110 . The processor 140 may include a RAM, a ROM, a graphic processing unit, a main CPU, first through n interfaces, and a bus. At this time, the RAM, ROM, graphic processing unit, main CPU, first to n interfaces, etc. may be connected to each other through a bus.

RAM stores O/S and application programs. Specifically, when the electronic device 100 is booted, O/S is stored in RAM, and various application data selected by the user may be stored in RAM.

The ROM stores instruction sets for system booting. When a turn-on command is input and power is supplied, the main CPU copies the O/S stored in the memory 110 to the RAM according to the command stored in the ROM, and executes the O/S to boot the system. When booting is completed, the main CPU copies various application programs stored in the memory 110 to RAM, and executes the application programs copied to RAM to perform various operations.

The graphic processing unit uses a calculation unit (not shown) and a rendering unit (not shown) to create a screen including various objects such as items, images, and text. Here, the calculation unit may be configured to calculate attribute values such as coordinate values, shape, size, color, etc. of each object to be displayed according to the layout of the screen by using a control command received from the input unit. And, the rendering unit may be configured to generate screens of various layouts including objects based on the attribute values calculated by the calculation unit. The screen created by the rendering unit may be displayed within the display area of the display 130 .

The main CPU accesses the memory 110 and performs booting using the OS stored in the memory 110 . And, the main CPU performs various operations using various programs, contents, data, etc. stored in the memory 110 .

The first through n interfaces are connected to the various components described above. One of the first through n interfaces may be a network interface connected to an external device through a network.

Meanwhile, furthermore, the processor 130 may control the artificial intelligence model. In this case, of course, the processor 130 may include a graphics-only processor (eg, GPU) for controlling the artificial intelligence model.

Meanwhile, the artificial intelligence model according to the present invention may be a model based on supervised learning or unsupervised learning. Furthermore, the artificial intelligence model according to the present invention may include a support vector machine (SVM), a decision tree, a neural network, and the like, and methodologies to which they are applied.

As an embodiment, the artificial intelligence model according to the present invention may be an artificial intelligence model based on convolutional deep neural networks (CNN) learned by inputting training data. However, it is not limited thereto, and it goes without saying that various artificial intelligence models can be applied to the present invention. For example, models such as a deep neural network (DNN), a recurrent neural network (RNN), and a bidirectional recurrent deep neural network (BRDNN) may be used as an artificial intelligence model, but are not limited thereto.

At this time, convolutional deep neural networks (CNNs) are a type of multilayer perceptrons designed to use a minimum of preprocessing. A convolutional neural network consists of one or several convolutional layers and general artificial neural network layers placed on top of them, and additionally utilizes weights and pooling layers. Thanks to this structure, convolutional neural networks can fully utilize input data with a two-dimensional structure. Also, convolutional neural networks can be trained via standard back-propagation. Convolutional neural networks are easier to train than other feedforward artificial neural network techniques and have the advantage of using fewer parameters.

In addition, deep neural networks (DNNs) are artificial neural networks (ANNs) consisting of a plurality of hidden layers between an input layer and an output layer.

At this time, the structure of the deep neural network may be composed of a perceptron. Perceptron consists of several inputs, one processor, and one output value. The processor multiplies each of a plurality of input values by a weight, and then sums all the input values multiplied by the weight. Then, the processor substitutes the summed value into the activation function and outputs one output value. If a specific value is desired as an output value of the activation function, a weight value multiplied with each input value may be modified, and an output value may be recalculated using the corrected weight value. At this time, each perceptron may use a different activation function. In addition, each perceptron accepts the outputs passed from the previous layer as inputs, and then obtains the outputs using the activation function. The obtained output is passed as input to the next layer. Through the process as described above, several output values can finally be obtained.

Recurrent Neural Network (RNN) refers to a neural network in which the connections between units constituting an artificial neural network constitute a directed cycle. Unlike feed-forward neural networks, recurrent neural networks can utilize the memory inside the neural network to process arbitrary inputs.

Deep Belief Networks (DBN) is a generative graphical model used in machine learning. In deep learning, it means a deep neural network consisting of multiple layers of latent variables. There are connections between layers, but there is no connection between units within a layer.

Due to the characteristics of a generative model, the deep trust neural network can be used for prior learning, and after learning initial weights through prior learning, the weights can be fine-tuned through backpropagation or other discrimination algorithms. This characteristic is very useful when there is little training data, because the smaller the training data, the stronger the influence of the initial weight values on the resulting model. The pre-learned initial value of the weight becomes closer to the optimal weight than the arbitrarily set initial value of the weight, which enables the performance and speed of the fine-tuning step to be improved.

The above-described artificial intelligence and its learning method are described for illustrative purposes, and the artificial intelligence and its learning method used in the above-described embodiments are not limited. For example, all kinds of artificial intelligence technologies and learning methods that can be applied by a person skilled in the art to solve the same problem can be used to implement the system according to the disclosed embodiment.

Meanwhile, the processor 140 may include one or more cores (not shown) and a graphic processor (not shown) and/or a connection path (eg, a bus) for transmitting and receiving signals to and from other components. can

Processor 140 according to one embodiment performs a method described in connection with the present invention by executing one or more instructions stored in memory 110 .

For example, the processor 140 acquires new training data by executing one or more instructions stored in the memory 110, performs a test on the acquired new training data using a learned model, and performs the test As a result, first training data for which the labeled information is obtained with an accuracy equal to or higher than a predetermined first reference value is extracted, the extracted first training data is deleted from the new training data, and the new training data from which the extracted training data is deleted The learned model may be retrained using the training data.

Meanwhile, the processor 140 includes RAM (Random Access Memory, not shown) and ROM (Read-Only Memory) for temporarily and/or permanently storing signals (or data) processed in the processor 140. , not shown) may be further included. In addition, the processor 140 may be implemented in the form of a system on chip (SoC) including at least one of a graphics processing unit, RAM, and ROM.

The memory 110 may store programs (one or more instructions) for processing and control of the processor 140 . Programs stored in the memory 110 may be divided into a plurality of modules according to functions.

The camera 120 is configured to photograph at least one subject, and the processor 140 may recognize at least one object or identify a distance to the object through the camera 120 .

As an embodiment, the camera 120 may be implemented as a depth camera or RGB camera that acquires a 3D image. The depth camera is a stereo camera (stereoscopic camera, stereo camera, binocular camera, two-view camera) that acquires two images at the same time and acquires the subject in three dimensions, and based on the return time and speed of the reflected light on the subject, the background It may be a ToF camera (Time of Flight camera) that obtains 3D image information of the subject.

The display 130 is a configuration for visually outputting various information, and may be implemented as a Liquid Crystal Display (LCD), Plasma Display Panel (PDP), Organic Light Emitting Diodes (OLED), Transparent OLED (TOLED), Micro LED, etc. However, it is not limited thereto, and other types of conventionally known displays may be included.

As an example, the display 130 may be implemented in the form of a touch screen capable of detecting a user's touch manipulation, or may be implemented as a flexible display capable of being folded or bent.

The communication unit 120 may communicate with an external device. In particular, the communication unit 120 may include various communication chips such as a Wi-Fi chip, a Bluetooth chip, a wireless communication chip, an NFC chip, and a low energy Bluetooth chip (BLE chip). At this time, the Wi-Fi chip, the Bluetooth chip, and the NFC chip perform communication in a LAN method, a WiFi method, a Bluetooth method, and an NFC method, respectively. In the case of using a Wi-Fi chip or a Bluetooth chip, various connection information such as an SSID and a session key is first transmitted and received, and various information can be transmitted and received after communication is connected using this. The wireless communication chip refers to a chip that performs communication according to various communication standards such as IEEE, ZigBee, 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), and 5th Generation (5G).

2 is a basic flow diagram according to an embodiment of the present disclosure.

As shown, in the control method of the electronic device 100 for determining the motion match using time and reference positions for each joint according to an aspect of the present invention, the electronic device 100 performs motion recognition of a person from an image. Obtaining joint reference data in which the reference position range for each joint used is set for each of a plurality of first time sections of the image (S1), identifying a plurality of joint objects of the person from the image (S2), and each of the plurality of identified joint objects (S3), and the point position of the joint point of the second time interval corresponding to any one of the 1-1st time intervals among the plurality of 1st time intervals is the reference position range of the 1-1st time interval. (S4), and based on the confirmation result, it is determined whether the motion of the person is the same as the motion indicated by the joint reference data (S5).

Meanwhile, in performing step S1, the electronic device 100 obtains joint reference data in which the reference position range for each joint used for motion recognition of the person is set for each of a plurality of first time sections of the image, and the communication unit ( 150) may be used to receive and obtain joint reference data from an external device.

Here, the joint reference data is data in which a reference position range of each joint previously determined for each motion is set for each of a plurality of first section times. For example, the joint reference data may be data in which a reference position range of each joint corresponding to a specific motion is set for each first time interval, as shown in Tables 1 and 2 below.

time interval 1st time period (1) movement hurrah posture joint joint position range right wrist X1, Y1 left wrist X2, Y2 … …

time interval 1st time period (1) movement Attention posture joint joint position range right wrist X3, Y3 left wrist X4, Y4 … …

In this case, as described above, the joint reference data may be obtained in the form of a table in the electronic device 100 or may be an image (animation, live image) including a state of performing an operation.

When the joint reference data is an image, a motion of the person object may be changed for each first time period and the corresponding motion may be maintained during the first time period. In addition, when the joint reference data is an image, a joint position range may be set by recognizing the position of each joint from the image. The location of each joint of the person object in the image is recognized as point coordinates, and a circle area with the point coordinates as the center of the circle can be set as the joint location range. In addition, the electronic device 100 may output joint reference data, which is an image, to the display 130 and photograph a user performing an operation while viewing the image using the camera 120 .

As described above, the joint location range may mean a region where a corresponding joint should be located.

3 is an exemplary view of point matching for a joint object according to an embodiment of the present disclosure.

Specifically, before performing step S2, the electronic device 100 may acquire an image of the user's appearance.

To this end, the processor 140 may control the camera 120 to capture the user's image, and the camera 120 may capture the user's image and generate an image containing the user's image.

In addition, the display 130 may output an image containing a user's appearance, and may output various contents and information.

In addition, the display 130 may output an image containing a user's appearance, and may output various contents and information.

Thereafter, in performing step S1, at least one object included in the image may be identified, and among the identified objects, a joint object that is an object for a joint of a person may be identified.

In this case, when a joint image for each joint of the user is pre-registered in the memory 110, the electronic device 100 may identify a joint object based on the joint image.

Meanwhile, when there are a plurality of people in the image, the electronic device 100 identifies a person object 1 for each person, acquires a face image for each person object 1, and acquires a face image for each person, based on the face image. Steps S2 to S5 may be performed for each object 1 .

4 is an exemplary view of joint point matching for a joint object according to an embodiment of the present disclosure.

In performing step S3, the electronic device 100 identifies the face of the person object 1, that is, the face image, and based on the face, the neck, left shoulder, left elbow, and left wrist of the person corresponding to the joint object. , right shoulder, right elbow, right wrist, left hip, left knee, left ankle, right hip, right knee and right ankle.

Accordingly, as shown in FIG. 4 , the electronic device 100 has a first joint point 11 on the neck, a second joint point 12 on the left shoulder, a third joint point 13 on the left elbow, and a left joint point 13 on the left shoulder. 4th joint point (14) on the wrist, 5th joint point (15) on the left ankle, 6th joint point (16) on the right shoulder, 7th joint point (17) on the right elbow, 8th joint point on the right wrist (18), the ninth joint point (19) at the right ankle, the tenth joint point (20) at the left hip joint, the eleventh joint point (21) at the left knee, the twelfth joint point (22) at the right hip joint and the right knee Match the thirteenth joint point 23 to each.

At this time, in identifying each joint object (neck, shoulder, elbow, wrist, hip joint, knee, ankle), the electronic device 100 uses not only the face of the person object 1, but also various various items such as the neck, upper body, fingers, and ears. It is possible to identify based on the body part, but is not limited thereto.

5 is an exemplary view of an example of a process of determining whether a point location is included in a reference location range by an electronic device according to an embodiment of the present disclosure, and FIG. 6 is an example of an electronic device according to an embodiment of the present disclosure. is an exemplary view of another example of a process of determining whether a point location is included in a reference location range.

In performing step S4, the electronic device 200 determines that the point position of the joint point of the second time interval corresponding to any one of the 1-1st time intervals among the plurality of first time intervals is It is determined whether it is included in the reference position range.

In this case, a plurality of first time sections are configured, and a reference position range for each joint is set for each first time section. Also, during the first time interval, a reference position range for each joint for the same motion is set.

The electronic device 200 may view the joint points finally generated from the image for each time section.

Specifically, the electronic device 200 divides a plurality of second time intervals based on the playback time of the image, and attaches joint points generated from images whose photographing times are included in each of the plurality of second time intervals to the corresponding second time intervals. classify into sections.

For example, the start time of the 2-1 time interval, which is one of a plurality of second time intervals, is March 1, 2022 at 10:10:05, and the end time is March 1, 2022 at 10:10 In the case of minutes and 10 seconds, the shooting time is March 1, 2022 10:10:05 and the end point is March 1, 2022 10:10:10 Classify by time interval.

In the above-described manner, joint points of the electronic device 200 are classified into each of a plurality of second time intervals.

Thereafter, the electronic device 200 checks the second time section corresponding to the 1-1 time section, and the point position of the joint point for each joint of the second time section in the reference position range for each joint of the 1-1 time section. It is determined whether each is included.

For example, if 10 first time intervals of 5 seconds are continuously configured and 10 second time intervals of 5 seconds are continuously configured, the electronic device 100 may perform the first time interval in chronological order. It is determined whether the point positions of the joint points for each joint in the first second time interval in chronological order are included in the reference position range for each joint in the time interval.

The electronic device 100 sequentially performs the above-described determination process on the reference position range for each joint in all first time intervals and the point positions of joint points for each joint in all second time intervals.

To this end, the electronic device 100 may synchronize the data start time of the joint reference data before step S4 and the motion start time when the motion of the person starts in the image to the same time point.

In addition, before step S4, the electronic device 100 partitions the image captured by the user such that the time length of each section between the plurality of first time sections and the time length of each section between the plurality of second time sections are the same.

Also, before step S4, the electronic device 100 selects any two joint points as reference joint points from among a plurality of joint points, and calculates the video body length from the reference joint points.

Thereafter, the electronic device 100 corrects the point positions of the plurality of joint points by using a comparison ratio, which is a ratio of the reference video body length to the video body length corresponding to the reference joint point.

Here, the reference joint points may be two joint points spaced apart by a distance that can represent the length of the user's body in the image. For example, the reference joint points may be a joint point of the right hip joint and a joint point of the right knee. The distance between the joint point of the right hip joint and the joint point of the right knee on the image is calculated as the image body length.

Here, the reference image body length may be the length between the center points of each reference joint position range of the same joint as the reference joint point in the joint reference data.

In addition, the electronic device 100 maintains the direction from the reference joint point to each of the plurality of joint points, but increases or decreases the distance from the reference joint point to each of the plurality of joint points to compensate for correction.

In addition, the electronic device 100 calculates a distance difference between the point position of the reference joint point and the position of the center point of each joint reference position range of the same joint as the reference joint point among the joint reference data, and calculates a distance difference between a plurality of joint points by the distance difference. You can change the point location.

Through this, the electronic device 100 determines whether the point positions of each joint point for each joint are included in the reference position range for each joint after correcting the difference according to the absolute position and ratio of the joint point from the joint reference data and the image. do.

In performing step S5, the electronic device 100 determines that if the point position of the joint point in the second time interval is included in the reference position range of the 1-1st time interval, the motion of the person in the 2nd time interval is It may be determined that it is the same as the motion indicated by the joint reference data of the 1-hour interval.

As shown in FIG. 5 , the electronic device 100 includes all joint points 11' to 23' of the second time period in the reference position ranges 11 to 23 of the 1-1 time period according to the hurray posture motion. If the point location of is included, it can be determined that the motion performed by the person in the second time period is the same as the hurray posture motion indicated by the joint reference data in the 1-1 time period.

Conversely, as shown in FIG. 6 , the electronic device 100 moves all joint points 11' to 23 of the second time period in the reference position ranges 11 to 23 of the 1-1 time period according to the hurray posture motion. '), if the point position of one or more points (17', 18', 19', 23') is not included, the action performed by the person in the second time interval is indicated by the joint reference data of the 1-1 time interval. It can be judged that it is not the same as the manse posture motion.

The electronic device 100 may display on the display 130 a result of determining whether the operations are the same.

Steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, implemented in a software module executed by hardware, or implemented by a combination thereof. A software module may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any form of computer readable recording medium well known in the art to which the present invention pertains.

Also, different embodiments of the present invention may complement each other or be combined.

Components of the present invention may be implemented as a program (or application) to be executed in combination with a computer, which is hardware, and stored in a medium. Components of the present invention may be implemented as software programming or software elements, and similarly, embodiments may include various algorithms implemented as data structures, processes, routines, or combinations of other programming constructs, such as C, C++ , Java (Java), assembler (assembler), may be implemented in a programming or scripting language such as Python (Python). Functional aspects may be implemented in an algorithm running on one or more processors.

Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features of the present invention. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

1: Person object
100: electronic device
110: memory
120: camera
130: display
140: processor
150: Ministry of Communication

Claims (7)

In the control method of an electronic device,
obtaining, by the electronic device, joint reference data in which a reference position range for each joint used for recognizing a motion of a person is set for each of a plurality of first time intervals from an image;
identifying, by the electronic device, a plurality of joint objects of the person from the image;
matching, by the electronic device, a joint point to each of the identified plurality of joint objects;
selecting, by the electronic device, two joint points among a plurality of joint points as reference joint points;
calculating, by the electronic device, an image body length from the reference joint point;
correcting, by the electronic device, point positions of the plurality of joint points by using a comparison ratio, which is a ratio of a reference video body length corresponding to the reference joint point to the video body length;
dividing, by the electronic device, the image into a plurality of second time sections;
The point position of a joint point matched to a joint object identified by the electronic device from an image during a second time interval corresponding to any one of the 1-1st time intervals among the plurality of first time intervals, the first - Determining whether it is included in the reference position range set in the 1 time interval; and
and determining, by the electronic device, whether the motion of the person is the same as the motion indicated by the joint reference data, based on the determination result.
According to claim 1,
The step of determining whether the motion of the person is the same as the motion indicated by the joint reference data,
In the electronic device, when the point position of the joint point in the second time interval is included in the reference position range of the 1-1 time interval, the motion of the person in the 2nd time interval is performed in the 1-1 A method of controlling an electronic device, comprising: determining that the joint reference data of the time interval is the same as the motion indicated by the motion.
According to claim 1,
In the electronic device, if the point position of the joint point in the second time interval is not included in the reference position range of the 1-1 time interval, the motion of the person in the second time interval A method of controlling an electronic device, comprising: determining that the joint reference data of a 1-hour interval is not the same as the motion indicated.
According to claim 1,
Synchronizing a data start time of the joint reference data and a motion start time when a motion of a person starts in the image to the same time point; further comprising, the control method of the electronic device.
According to claim 1,
The control method of the electronic device further comprising: dividing, by the electronic device, the image such that a time length for each section between the plurality of first time sections and a time length for each section between the plurality of second time sections are the same. .
delete According to claim 1,
The step of correcting the point positions of the plurality of joint points,
Correcting, by the electronic device, maintaining a direction from the reference joint point to each of the plurality of joint points, by increasing or decreasing the distance from the reference joint point to each of the plurality of joint points by a comparison ratio , control methods of electronic devices.

Images