KR20180023302A - Moving robot and control method thereof - Google Patents

Abstract

One aspect of the present invention relates to a moving robot comprising: a driving unit to move a main body; an image acquisition unit to acquire an image around the main body; a sensor unit including one or more sensors to detect an obstacle during moving; a storage unit to store location information of the detected obstacle and location information of the moving robot when the sensor unit detects the obstacle, to register an area with a predetermined size with respect to a location of the detected obstacle as an obstacle area, and to store an image acquired in the obstacle area by the image acquisition unit; and a control unit including an obstacle recognition module to sequentially recognize an attribute of the obstacle with respect to the images acquired in the obstacle area through the image acquisition unit based on data previously trained by machine learning and to determine a final attribute of the obstacle based on a plurality of sequentially recognized recognition results. Accordingly, the attribute of the obstacle is able to be correctly recognized, and the obstacle area is able to be registered and managed.

Description

[0001] Moving robot and control method [0002]

The present invention relates to a mobile robot and a control method thereof, and more particularly, to a mobile robot performing obstacle recognition and avoidance based on machine learning and a control method thereof.

Robots have been developed for industrial use and have been part of factory automation. In recent years, medical robots, aerospace robots, and the like have been developed, and household robots that can be used in ordinary homes are being developed. Among these robots, mobile robots capable of traveling by magnetic force are called mobile robots.

A typical example of a mobile robot used in the home is a robot cleaner. The robot cleaner is a device for cleaning a corresponding area by suctioning dust or foreign matter around the robot while running a certain area by itself.

The mobile robot is capable of moving by itself, is free to move, and is equipped with a plurality of sensors for avoiding obstacles during running, and can travel without obstacles.

In general, an infrared sensor or an ultrasonic sensor is used to detect an obstacle of a mobile robot. The infrared sensor senses the presence and distance of the obstacle through the amount of reflected light or the time of reflected light reflected by the obstacle, and when the ultrasonic sensor emits an ultrasonic wave having a predetermined period and there is an ultrasonic wave reflected by the obstacle, And the distance between the obstacle and the obstacle is determined using the time difference between the moment when the obstacle is reflected and the time when the obstacle is reflected.

On the other hand, obstacle recognition and avoidance have a great influence on the cleaning performance as well as the running performance of the mobile robot, and therefore it is required to secure the reliability of the obstacle recognition capability.

The prior art (Patent Registration No. 10-0669892) discloses a technology for implementing a reliable obstacle recognition technology by combining an infrared sensor and an ultrasonic sensor.

However, the prior art (Patent Registration No. 10-0669892) has a problem that it can not discriminate the attribute of the obstacle.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram for explaining an obstacle detection and avoidance method of a conventional mobile robot.

Referring to FIG. 1, the robot cleaner sucks dust and foreign substances while moving and performs cleaning (S11).

The ultrasonic sensor senses the ultrasonic signal reflected by the obstacle and recognizes the existence of the obstacle (S12). It is determined whether the height of the obstacle exceeds the height of the recognized obstacle (S13).

If it is determined that the robot cleaner is high enough, the robot cleaner moves straight (S14). Otherwise, the robot cleaner rotates the robot cleaner 90 degrees (S15).

For example, if the obstacle is a low threshold. The robot cleaner recognizes the threshold, and if the robot cleaner judges that the recognition result can pass, it moves beyond the threshold.

However, if the obstacle judged to be high enough is a wire, the robot cleaner may pass over the wire and be caught by the wire to be restrained.

In addition, since the pedestal of the fan has a height similar to or lower than the threshold, it can be judged that the robot cleaner is an obstacle to overcome. In this case, the robot cleaner rides on the pedestal of the fan, and the wheels can be restrained while idling.

In addition, if the entire human is not recognized and only a part of the human hair such as hair is detected, it can be determined that the human body is over the human body and straight forward. In this case, however, the robot cleaner can suck human hair, have.

Therefore, it is necessary to grasp the property of the front obstacle and to change the movement pattern according to the property.

On the other hand, interest in machine learning such as artificial intelligence and deep running has greatly increased recently.

Conventional machine learning was centered on statistical based classification, regression, and cluster models. In particular, in the learning of classification and regression model learning, a learning model that distinguishes the characteristics of learning data and new data based on these characteristics was defined by a person in advance. Deep learning, on the other hand, is one in which a computer identifies and identifies its own characteristics.

One of the factors that accelerated the development of deep learning is the deep-running framework that is provided as open source. For example, the deep learning frameworks include Theano at the University of Montreal, Toronto, Torch at New York University, Caffe at the University of California at Berkeley, and TensorFlow at Google.

With the release of deep learning frameworks, in addition to deep learning algorithms, extraction and selection of data to apply to learning processes, learning methods, and learning are becoming more important for effective learning and recognition.

Also, researches are being conducted to utilize artificial intelligence and machine learning in various products and services.

Patent Registration No. 10-0669892 (registered on January 10, 2007)

It is an object of the present invention to provide a mobile robot and its control method capable of performing obstacle recognition and avoidance operations by determining an attribute of an obstacle and adjusting a traveling pattern according to an obstacle property.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a mobile robot capable of improving the stability of the mobile robot itself and the convenience of the user by improving the operation efficiency and cleaning efficiency by performing actions such as forward, And a control method thereof.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a mobile robot and a control method thereof that can accurately recognize an attribute of an obstacle based on machine learning.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a mobile robot and a control method thereof that can efficiently perform machine learning and extract data that can be used for obstacle attribute recognition.

According to one aspect of the present invention, there is provided a mobile robot including a traveling unit for moving a main body, an image acquisition unit for acquiring an image around the main body, and a sensor including at least one sensor for sensing an obstacle The controller detects the obstacle and stores the position information of the detected obstacle and the position information of the mobile robot and registers an area having a predetermined size centered on the detected position of the obstacle as an obstacle area in a map, A storage unit for storing an image acquired by the image acquisition unit in an obstacle area, and a storage unit for storing the attributes of the obstacle in succession to the images acquired through the image acquisition unit in the obstacle region, based on the data learned in the machine learning And an obstacle recognition module for recognizing a final attribute of the obstacle based on a plurality of sequentially recognized recognition results By a control unit, it is possible to accurately recognize the attribute of the obstacle, register, and manage the obstacle region.

In order to achieve the above and other objects, a mobile robot according to an aspect of the present invention includes a traveling control module that controls driving of a traveling part on the basis of a final property of a recognized obstacle, The operation efficiency and the cleaning efficiency can be improved.

According to another aspect of the present invention, there is provided a mobile robot including a traveling unit for moving a main body, an image acquisition unit for acquiring an image around the main body, and at least one sensor for detecting an obstacle in a traveling state When the sensor unit and the sensor unit detect an obstacle, the position information of the detected obstacle and the position information of the mobile robot are stored, and an area having a predetermined size centered on the position of the detected obstacle is registered as an obstacle area in a map A storage unit for storing an image acquired by the image acquisition unit in the obstacle region, and a storage unit for storing the image acquired by the image acquisition unit in the obstacle region on the basis of the data learned in the machine learning And an obstacle recognition module for recognizing the property of the obstacle and discriminating the final property of the obstacle based on the recognized plurality of recognition results By including parts, it is possible to accurately recognize the attributes of obstacles and registration, managing the obstacle region.

According to another aspect of the present invention, there is provided a method of controlling a mobile robot including sensing a moving obstacle through a sensor unit, detecting position information of the detected obstacle when the sensor unit detects the obstacle, Storing a position information of the mobile robot, registering an area having a predetermined size centered on the detected obstacle as an obstacle area in a map, acquiring a plurality of images in an obstacle area during a moving, sequentially recognizing an attribute of an obstacle for images obtained through an image acquisition unit while moving through the obstacle area based on data learned in advance by machine learning; determining, based on a plurality of sequentially recognized recognition results, Determining a final attribute of the obstacle, and controlling the driving of the traveling section based on the final attribute of the recognized obstacle, .

According to at least one of the embodiments of the present invention, the mobile robot can accurately determine the property of the obstacle and adjust the traveling pattern according to the obstacle property, thereby performing highly reliable obstacle recognition and avoidance operations.

According to at least one of the embodiments of the present invention, it is possible to improve the stability of the mobile robot itself and the user's convenience by performing operations such as forward, backward, stop, and bypass according to the recognition result of the obstacle, And a control method thereof can be provided.

In addition, according to at least one embodiment of the present invention, it is possible to provide a mobile robot and a control method thereof that can accurately recognize an attribute of an obstacle based on machine learning.

In addition, according to at least one of the embodiments of the present invention, the mobile robot can efficiently perform machine learning and extract data that can be used for obstacle attribute recognition.

Meanwhile, various other effects will be directly or implicitly disclosed in the detailed description according to the embodiment of the present invention to be described later.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram for explaining an obstacle detection and avoidance method of a conventional mobile robot.
2 is a perspective view illustrating a mobile robot and a charging base for charging the mobile robot according to an embodiment of the present invention.
FIG. 3 is a top view of the mobile robot shown in FIG. 2. FIG.
FIG. 4 is a front view of the mobile robot shown in FIG. 2. FIG.
FIG. 5 is a view showing a bottom portion of the mobile robot shown in FIG. 2. FIG.
FIG. 6 and FIG. 7 are block diagrams showing control relationships among main components of a mobile robot according to an embodiment of the present invention.
Figure 8 is an illustration of a simplified internal block diagram of a server in accordance with an embodiment of the present invention.
9 to 12 are diagrams referred to in explanation of Deep Learning.
13 and 14 are diagrams referred to in explaining the obstacle recognition.
15 is a flowchart illustrating a method of controlling a mobile robot according to an embodiment of the present invention.
16 is a flowchart illustrating a method of controlling a mobile robot according to an embodiment of the present invention.
17 is a flowchart illustrating a method of controlling a mobile robot according to an embodiment of the present invention.
18 to 28 are diagrams referred to in explaining the control method of the mobile robot according to the embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, it is needless to say that the present invention is not limited to these embodiments and can be modified into various forms.

In the drawings, the same reference numerals are used for the same or similar parts throughout the specification.

The suffix "module" and " part "for components used in the following description are given merely for convenience of description and do not give special significance or role in themselves. Accordingly, the terms "module" and "part" may be used interchangeably.

The mobile robot 100 according to an embodiment of the present invention refers to a robot that can move by itself using wheels or the like, and may be a home helper robot and a robot cleaner. Hereinafter, a robot cleaner having a cleaning function of a mobile robot will be described with reference to the drawings, but the present invention is not limited thereto.

2 is a perspective view illustrating a mobile robot and a charging base for charging the mobile robot according to an embodiment of the present invention.

FIG. 3 is a view illustrating a top portion of the mobile robot shown in FIG. 2. FIG. 4 is a front view of the mobile robot shown in FIG. 2. FIG. It is a degree.

FIG. 6 and FIG. 7 are block diagrams showing control relationships among main components of a mobile robot according to an embodiment of the present invention.

3 to 7, the mobile robots 100, 100a, and 100b include a main body 110 and image acquisition units 120, 120a, and 120b that acquire images around the main body 110. FIG.

Hereinafter, in defining each portion of the main body 110, a portion facing the ceiling in the running zone is defined as an upper surface portion (see Fig. 3), and a portion facing the floor in the running zone is defined as a bottom portion And a portion of the portion of the periphery of the main body 110 facing the running direction between the upper surface portion and the bottom surface portion is defined as a front surface portion (see FIG. 4).

The mobile robots 100, 100a and 100b include a traveling unit 160 for moving the main body 110. [ The driving unit 160 includes at least one driving wheel 136 for moving the main body 110. The driving unit 160 includes a driving motor (not shown) connected to the driving wheels 136 to rotate the driving wheels. The driving wheels 136 may be provided on the left and right sides of the main body 110 and will be referred to as the left wheel 136 (L) and the right wheel 136 (R), respectively.

The left wheel 136 (L) and the right wheel 136 (R) may be driven by a single drive motor, but may be driven by a left wheel drive motor and a right wheel 136 (R) And a right wheel drive motor for driving the right wheel drive motor. The running direction of the main body 110 can be switched to the left or right side by making a difference in rotational speed between the left wheel 136 (L) and the right wheel 136 (R).

A suction port 110h for sucking in air may be formed on the bottom surface of the main body 110. A suction device for supplying suction force for sucking air through the suction port 110h is provided in the main body 110 And a dust container (not shown) for dust collecting with the air through the suction port 110h.

The main body 110 may include a case 111 forming a space for accommodating various components constituting the mobile robots 100, 100a, and 100b. An opening for inserting and removing the dust container may be formed in the case 111, and a dust container cover 112 for opening and closing the opening may be rotatably provided with respect to the case 111.

An auxiliary brush 135 having a brush having a plurality of blades extending radially and located on the front side of the bottom surface of the main body 110, May be provided. By the rotation of the brushes 134 and 135, the dusts are separated from the floor in the traveling zone, and the dusts separated from the floor are sucked through the suction port 110h and collected in the dustbin.

The battery 138 supplies not only the drive motor but also the power required for the overall operation of the mobile robots 100, 100a and 100b. When the battery 138 is discharged, the mobile robots 100, 100a, and 100b can travel to return to the charging base 200 for charging. During the return travel, the mobile robots 100, 100a, Can detect the position of the charging stand 200 by itself.

The charging stand 200 may include a signal transmitting unit (not shown) for transmitting a predetermined return signal. The return signal may be an ultrasonic signal or an infrared signal, but is not limited thereto.

The mobile robots 100, 100a and 100b may include a signal sensing unit (not shown) for receiving a return signal. The charging base 200 may transmit an infrared signal through a signal transmitting unit, and the signal sensing unit may include an infrared sensor that senses an infrared signal. The mobile robots 100, 100a and 100b move to the position of the charging base 200 according to the infrared signal transmitted from the charging base 200 and dock the charging base 200. [ The charging is performed between the charging terminal 133 of the mobile robots 100, 100a and 100b and the charging terminal 210 of the charging stand 200 by such docking.

The image acquiring unit 120 photographs a driving area, and may include a camera module. The camera module may include a digital camera. A digital camera includes an image sensor (e.g., a CMOS image sensor) configured with at least one optical lens, and a plurality of photodiodes (e.g., pixels) that are formed by light passing through the optical lens, And a digital signal processor (DSP) that forms an image based on signals output from the photodiodes. The digital signal processor is capable of generating moving images composed of still frames as well as still images.

The image acquisition unit 120 may include a front camera 120a provided to acquire an image of the front of the main body 110 and an image acquisition unit 120b provided on the upper surface of the main body 110 to acquire images of the ceiling And the upper camera 120b, but the position and the photographing range of the image obtaining unit 120 are not necessarily limited thereto.

In the case of this embodiment, a camera is provided at a part (ex, front, rear, bottom) of the mobile robot, and the captured image can be continuously acquired at the time of cleaning. Several cameras may be installed for each part of the camera for photographing efficiency. The image captured by the camera can be used to identify the kind of material such as dust, hair, floor, etc. present in the space, whether it is cleaned, or to confirm the cleaning time.

The front camera 120a can photograph an obstacle present in the traveling direction of the mobile robots 100, 100a and 100b or a situation of the cleaning area.

According to an embodiment of the present invention, the image acquiring unit 120 may acquire a plurality of images by continuously photographing the periphery of the main body 110, and the obtained plurality of images may be stored in the storage unit 150 .

The mobile robots 100, 100a and 100b can improve accuracy of obstacle recognition by using a plurality of images, or by selecting one or more images from a plurality of images and using effective data.

In addition, the mobile robots 100, 100a, and 100b may include a sensor unit 170 including sensors for sensing various data related to the operation and state of the mobile robot.

For example, the sensor unit 170 may include an obstacle detection sensor 131 for sensing an obstacle ahead. The sensor unit 170 may further include a cliff detection sensor 132 for detecting the presence or absence of a cliff on the floor in the driving area and a lower camera sensor 139 for acquiring a bottom image.

 Referring to FIGS. 2 and 4, the obstacle detection sensor 131 may include a plurality of sensors installed on the outer circumferential surface of the mobile robot 100 at regular intervals.

For example, the sensor unit 170 may include a first sensor disposed on the front surface of the main body 110, a second sensor disposed left and right from the first sensor, and a third sensor .

The obstacle detection sensor 131 may include an infrared sensor, an ultrasonic sensor, an RF sensor, a geomagnetic sensor, a position sensitive device (PSD) sensor, and the like.

Meanwhile, the position and type of the sensor included in the obstacle detection sensor 131 may vary depending on the type of the mobile robot, and the obstacle detection sensor 131 may include more various sensors.

The obstacle detection sensor 131 is a sensor for detecting a distance to a wall or an obstacle in a room, and the present invention is not limited to this type, but an ultrasonic sensor will be described below as an example.

The obstacle detection sensor 131 senses an object, particularly an obstacle, existing in a traveling direction (movement direction) of the mobile robot, and transmits the obstacle information to the control unit 140. That is, the obstacle detection sensor 131 can sense the moving path of the mobile robot, the protrusions present on the front or side of the robot, the furniture of the house, the furniture, the wall, the wall edge, and the like and transmit the information to the control unit.

At this time, the controller 140 detects the position of the obstacle based on at least two signals received through the ultrasonic sensor, and controls the movement of the mobile robot 100 according to the position of the detected obstacle.

According to an embodiment, the obstacle detection sensor 131 provided on the outer surface of the case 110 may include a transmitter and a receiver.

For example, the ultrasonic sensor may be provided such that at least one transmitting portion and at least two receiving portions are staggered from each other. Accordingly, it is possible to radiate signals at various angles and to receive signals reflected from the obstacles at various angles.

According to an embodiment, the signal received by the obstacle detection sensor 131 may undergo signal processing such as amplification, filtering, and the like, and then the distance and direction to the obstacle may be calculated.

The sensor unit 170 may further include a motion detection sensor for sensing motion of the mobile robot 100, 100a, and 100b according to driving of the main body 110, and outputting motion information. As the motion detection sensor, a gyro sensor, a wheel sensor, an acceleration sensor, or the like can be used. The data detected based on the data detected by the motion detection sensor or the data detected by the motion detection sensor can constitute odometry information.

The gyro sensor detects the rotation direction and detects the rotation angle when the mobile robots 100, 100a, 100b move according to the operation mode. The gyro sensor detects the angular velocity of the mobile robots 100, 100a and 100b and outputs a voltage value proportional to the angular velocity. The control unit 140 calculates the rotation direction and the rotation angle using the voltage value output from the gyro sensor.

The wheel sensor is connected to the left wheel 136 (L) and the right wheel 136 (R) to sense the number of revolutions of the wheel. Here, the wheel sensor may be a rotary encoder. The rotary encoder senses and outputs the number of rotations of the left wheel 136 (L) and the right wheel 136 (R).

The control unit 140 can calculate the rotational speeds of the left and right wheels using the number of rotations. The control unit 140 can calculate the rotation angle using the difference in the number of revolutions of the left wheel 136 (L) and the right wheel 136 (R).

The acceleration sensor senses a change in the speed of the mobile robots 100, 100a, 100b, for example, changes in the mobile robots 100, 100a, 100b due to start, stop, change of direction or collision with an object. The acceleration sensor is attached to the main wheel or the adjoining positions of the auxiliary wheels, so that the slip or idling of the wheel can be detected.

In addition, the acceleration sensor is embedded in the control unit 140 and can detect a speed change of the mobile robots 100, 100a, and 100b. That is, the acceleration sensor detects the amount of impact according to the speed change and outputs a corresponding voltage value. Thus, the acceleration sensor can perform the function of an electronic bumper.

The control unit 140 can calculate the positional change of the mobile robots 100, 100a, and 100b based on the motion information output from the motion detection sensor. Such a position is a relative position corresponding to the absolute position using the image information. The mobile robot can improve the performance of the position recognition using the image information and the obstacle information through the relative position recognition.

Meanwhile, the mobile robots 100, 100a, and 100b may include a power supply unit (not shown) that includes a rechargeable battery 138 to supply power to the robot cleaner.

The power supply unit supplies driving power and operating power to the respective components of the mobile robots 100, 100a and 100b, and can be charged by receiving a charging current from the charging stand 200 when the remaining power is insufficient.

The mobile robots 100, 100a and 100b may further include a battery sensing unit (not shown) for sensing the state of charge of the battery 138 and transmitting the sensing result to the control unit 140. [ The battery 138 is connected to the battery sensing unit, and the battery remaining amount and charging state are transmitted to the control unit 140. The remaining battery level can be displayed on the screen of the output unit (not shown).

In addition, the mobile robots 100, 100a, and 100b include an operation unit 137 that can turn on / off or input various commands. Various control commands necessary for the overall operation of the mobile robot 100 can be inputted through the operation unit 137. [ In addition, the mobile robots 100, 100a, and 100b can display reservation information, a battery state, an operation mode, an operation state, an error state, and the like, including an output unit (not shown).

Referring to FIGS. 6 and 7, the mobile robots 100a and 100b include a controller 140 for processing and determining various information such as recognizing a current position, and a storage unit 150 for storing various data. In addition, the mobile robots 100, 100a and 100b may further include a communication unit 190 for transmitting and receiving data to / from an external terminal.

The external terminal is provided with an application for controlling the mobile robots 100a and 100b and displays a map of the traveling area to be cleaned by the mobile robots 100a and 100b through execution of the application and cleans a specific area on the map The area can be specified. The external terminal may be, for example, a remote controller, a PDA, a laptop, a smart phone, or a tablet on which an application for setting a map is mounted.

The external terminal can communicate with the mobile robots 100a and 100b to display the current position of the mobile robot together with the map, and information about a plurality of areas can be displayed. Further, the external terminal updates its position and displays it according to the traveling of the mobile robot.

The control unit 140 controls the overall operation of the mobile robot 100 by controlling the image acquisition unit 120, the operation unit 137 and the travel unit 160 constituting the mobile robots 100a and 100b.

The storage unit 150 records various types of information required for controlling the mobile robot 100, and may include a volatile or nonvolatile recording medium. The storage medium stores data that can be read by a microprocessor, and includes a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD- Tape, floppy disk, optical data storage, and the like.

Also, the storage unit 150 may store a map of the driving area. The map may be input by an external terminal, a server or the like capable of exchanging information with the mobile robots 100a and 100b through wired or wireless communication, or may be one generated by self-learning by the mobile robots 100a and 100b have.

The map can display the location of the rooms in the driving area. In addition, the current position of the mobile robots 100a, 100b can be displayed on the map, and the current position of the mobile robots 100a, 100b on the map can be updated in the course of travel. The external terminal stores the same map as the map stored in the storage unit 150.

The storage unit 150 may store cleaning history information. Such cleaning history information can be generated each time cleaning is performed.

The map of the driving area stored in the storage unit 150 may be a navigation map used for traveling during cleaning, a simultaneous localization and mapping (SLAM) map used for position recognition, A learning map for storing information to be used for learning cleaning, a global location map used for global location recognition, an obstacle recognition map for recording information on recognized obstacles, and the like.

Meanwhile, although the maps can be separately stored and managed in the storage unit 150 according to the usage as described above, the maps may not be clearly classified for each use. For example, a plurality of pieces of information may be stored in one map for use in at least two applications.

The control unit 140 may include a travel control module 141, a location recognition module 142, a map generation module 143, and an obstacle recognition module 144.

3 to 7, the travel control module 141 controls traveling of the mobile robots 100, 100a, and 100b, and controls driving of the travel unit 160 according to the travel setting. In addition, the travel control module 141 can grasp the travel paths of the mobile robots 100, 100a, and 100b based on the operation of the travel unit 160. [ For example, the driving control module 141 can grasp the current or past traveling speed and the traveling distance of the mobile robot 100 based on the rotational speed of the driving wheel 136, L), 136 (R)), the current or past direction change process can be grasped. The positions of the mobile robots 100, 100a, and 100b on the map can be updated based on the travel information of the mobile robots 100, 100a, and 100b thus identified.

The map generating module 143 may generate a map of the running area. The map generation module 143 can process the image acquired through the image acquisition unit 120 to generate a map. That is, a cleaning map corresponding to the cleaning area can be created.

Also, the map generation module 143 can process the image acquired through the image acquisition unit 120 at each position and recognize the global position in association with the map.

The position recognition module 142 estimates and recognizes the current position. The position recognition module 142 grasps the position in association with the map generation module 143 using the image information of the image acquisition unit 120 so that even when the positions of the mobile robots 100, 100a, 100b suddenly change The current position can be estimated and recognized.

The mobile robots 100, 100a and 100b are capable of recognizing the position during the continuous running through the position recognition module 142 and are also capable of executing the driving control module 141, the map generation module 143, Through the obstacle recognition module 144, the map can be learned and the current position can be estimated.

The image acquisition unit 120 acquires images of the surroundings of the mobile robot 100 while the mobile robots 100, 100a, and 100b are traveling. Hereinafter, the image acquired by the image acquisition unit 120 is defined as an 'acquired image'.

The acquired image includes various features such as lights, edges, corners, blobs, ridges, etc., located on the ceiling.

The map generation module 143 detects the feature from each of the acquired images. Various methods for detecting features from an image in the field of Computer Vision (Feature Detection) are well known. Several feature detectors suitable for detecting these features are known. For example, Canny, Sobel, Harris & Stephens / Plessey, SUSAN, Shi & Tomasi, Level curve curvature, FAST, Laplacian of Gaussian, Difference of Gaussians, Determinant of Hessian, MSER, PCBR and Gray-level blobs detector.

The map generation module 143 calculates a descriptor based on each minutiae point. The map generation module 143 may convert a feature point into a descriptor using a Scale Invariant Feature Transform (SIFT) technique for feature detection. The descriptor may be denoted by an n-dimensional vector.

The SIFT can detect unchanging features with respect to changes in scale, rotation, and brightness of an object to be photographed. Even if the same region is photographed with a different attitude of the mobile robot 100 (i.e., -invariant) can be detected. Of course, various other techniques (e.g., HOG: Histogram of Oriented Gradient, Haar feature, Fems, Local Binary Pattern (LBP), Modified Census Transform (MCT)) may be applied.

The map generation module 143 classifies at least one descriptor for each acquired image into a plurality of groups according to a predetermined lower classification rule on the basis of the descriptor information obtained through the acquired image of each position, Can be converted into lower representative descriptors, respectively.

As another example, it is also possible to classify all descriptors gathered from acquired images in a predetermined area, such as a room, into a plurality of groups according to a predetermined sub-classification rule, and write descriptors included in the same group according to the predetermined lower representative rule, . ≪ / RTI >

The map generation module 143 can obtain the feature distribution of each position through such a process. Each position feature distribution can be represented as a histogram or an n-dimensional vector. As another example, the map generation module 143 can estimate an unknown current position based on descriptors calculated from each feature point, without going through a predetermined lower classification rule and a predetermined lower representative rule.

In addition, when the current position of the mobile robots 100, 100a, 100b is in an unknown state due to a positional jump or the like, the current position can be estimated based on data such as a pre-stored descriptor or a lower representative descriptor.

The mobile robots 100, 100a, and 100b acquire an acquired image through the image acquisition unit 120 at an unknown current position. Through the image, various features such as lights, edges, corners, blobs, ridges, etc., are found on the ceiling.

The position recognition module 142 detects the features from the acquired image. Various methods of detecting features from an image in the field of computer vision technology and descriptions of various feature detectors suitable for detecting these features are as described above.

The position recognition module 142 calculates the recognition descriptor through the recognition descriptor calculation step based on each recognition feature point. Here, the recognition minutiae and the recognition descriptor are used to describe the process performed by the obstacle recognition module 144, which is to distinguish the terms used to describe the process performed by the map generation module 143. [ However, the characteristics of the outside world of the mobile robots 100, 100a, and 100b are merely defined by different terms.

The location recognition module 142 may convert the recognized minutiae point into the recognition descriptor using the Scale Invariant Feature Transform (SIFT) technique for detecting the feature. The recognition descriptor may be denoted by an n-dimensional vector.

As described above, the SIFT selects characteristic points that are easily distinguishable, such as corner points, from the acquired image, and then determines the distribution characteristics of the brightness gradient of pixels belonging to a certain region around each characteristic point ) Is an image recognition technique that obtains an n-dimensional vector (vector) in which the degree of change in each direction is a numerical value for each dimension.

Based on at least one recognition descriptor information obtained through an acquired image of an unknown current position, the position recognition module 142 generates position information (for example, feature distribution of each position) (Lower recognition feature distribution).

According to a predetermined lower comparison rule, each position feature distribution can be compared with each recognition feature distribution to calculate each similarity. The similarity degree (probability) is calculated for each position corresponding to each position, and the position where the greatest probability is calculated can be determined as the current position.

In this way, the control unit 140 can distinguish the driving zone, generate a map composed of a plurality of areas, or recognize the current position of the main body 110 based on the pre-stored map.

When the map is generated, the control unit 140 can transmit the generated map to an external terminal, a server, or the like through the communication unit 190. [ Also, as described above, the control unit 140 can store the map in the storage unit when the map is received from an external terminal, a server, or the like.

In addition, when the map is updated during travel, the controller 140 transmits the updated information to the external terminal so that the map stored in the external terminal and the mobile robots 100, 100a, and 100b is the same. As the map stored in the external terminal and the mobile robots 100, 100a, and 100b are kept the same, the mobile robot 100, 100a, and 100b can clean the designated area with respect to the cleaning command from the mobile terminal, So that the current position of the mobile robot can be displayed on the terminal.

At this time, the map is divided into a plurality of areas for the cleaning area, and includes a connection path for connecting the plurality of areas, and may include information about the obstacles in the area.

When the cleaning command is inputted, the control unit 140 determines whether or not the position on the map matches the current position of the mobile robot. The cleaning command can be input from the remote controller, the operation unit, or an external terminal.

If the current position does not coincide with the current position on the map or the current position can not be confirmed, the controller 140 recognizes the current position and restores the current position of the mobile robot 100, The traveling section 160 can be controlled to move to the designated area.

If the current position does not coincide with the position on the map or the current position can not be confirmed, the position recognition module 142 analyzes the acquired image input from the image obtaining unit 120 and estimates the current position based on the map can do. In addition, the obstacle recognition module 144 or the map generation module 143 may also recognize the current position in the same manner.

After the current position of the mobile robots 100, 100a, 100b is recognized and the driving control module 141 calculates the traveling route from the current position to the designated area and controls the traveling unit 160 to be the designated area Move.

In the case of receiving the cleaning pattern information from the server, the travel control module 141 can divide the entire traveling area into a plurality of areas and set one or more areas as designated areas according to the received cleaning pattern information.

Also, the travel control module 141 may calculate the travel route in accordance with the received cleaning pattern information, travel along the travel route, and perform cleaning.

The control unit 140 may store the cleaning history in the storage unit 150 when the cleaning of the designated area is completed.

The control unit 140 may transmit the operation state or the cleaning state of the mobile robot 100 to the external terminal and the server through the communication unit 190 at predetermined intervals.

Based on the received data, the external terminal displays the position of the mobile robot together with the map on the screen of the application being executed, and also outputs information on the cleaning state.

The mobile robots 100, 100a and 100b according to the embodiment of the present invention move until an obstacle or a wall surface is detected in one direction. When the obstacle recognition module 144 recognizes an obstacle, , And the like can be determined.

For example, if the recognized obstacle is an obstacle of a kind that can be passed over, the mobile robots 100, 100a, 100b can continue straight ahead. Alternatively, if the recognized obstacle is an obstacle of a kind that can not be surpassed, the mobile robots 100, 100a and 100b rotate and move a certain distance, move to a distance in which the obstacle is sensed in the opposite direction of the initial moving direction, Can travel in the form of

The mobile robots 100, 100a and 100b according to the embodiment of the present invention can perform obstacle recognition and avoidance based on machine learning.

The control unit 140 includes an obstacle recognition module 144 for recognizing an obstacle learned through machine learning in the input image and a driving unit for driving the driving unit 160 based on the recognized obstacle characteristics And a travel control module 141 for controlling the vehicle.

The mobile robots 100, 100a and 100b according to the embodiment of the present invention may include an obstacle recognition module 144 in which the attributes of obstacles are learned by machine learning.

Machine learning means that the computer learns from the data through the computer, and the computer takes care of the problem, even though the computer does not instruct the person directly with the logic.

Deep Learning (Deep Learning). It is based on Artificial Neural Networks (ANN) for constructing artificial intelligence. It is an artificial intelligence technology that allows a computer to learn like a human being without learning it as a way of teaching people how to think.

The ANN may be implemented in a software form or a hardware form such as a chip.

The obstacle recognition module 144 may include an artificial neural network (ANN) in the form of software or hardware in which the property of the obstacle is learned.

For example, the obstacle recognition module 144 may include a Deep Neural Network (DNN) such as CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), DBN (Deep Belief Network) . ≪ / RTI >

Deep learning will be described later in detail with reference to Figs. 9 to 12 and the like.

The obstacle recognition module 144 may determine an attribute of the obstacle included in the input image data based on the weights among the nodes included in the DNN.

If the sensor unit 170 senses an obstacle while the mobile robots 100, 100a and 100b are moving, the control unit 140 controls the image acquisition unit (not shown) based on the moving direction and the moving speed of the main body 110 120 may select a specific view image before the obstacle detection point of the sensor unit 170 among the plurality of consecutive images acquired by the sensor unit 170.

When the image acquisition unit 120 acquires an image using the obstacle detection point of the sensor unit 170 as a trigger signal, the obstacle is not included in the acquired image or is included small .

Therefore, an embodiment of the present invention can detect obstacles of the sensor unit 170 among a plurality of consecutive images acquired by the image acquisition unit 120, based on the moving direction and the moving speed of the main body 110 A point-in-time image before the point-in-time can be selected and used as data for obstacle recognition.

On the other hand, the obstacle recognition module 144 can recognize the attributes of the obstacle included in the selected point-in-time image based on the data learned through machine learning.

If the sensor unit 170 senses an obstacle while the mobile robots 100, 100a and 100b are moving, the control unit 140 controls the image acquisition unit (not shown) corresponding to the direction of the obstacle detected by the sensor unit 170 120 to extract a part of the image to be acquired.

The image acquisition unit 120, particularly the front camera 120a, can acquire images within a predetermined angle range in the moving direction of the mobile robots 100, 100a, and 100b.

The control unit 140 may not use the entire image acquired by the image acquisition unit 120, in particular, the front camera 120a, but may determine an attribute of an obstacle existing in a moving direction using only a part of the image.

6, the controller 140 may perform image processing for extracting a partial area of an image acquired by the image acquiring unit 120 corresponding to the direction of an obstacle sensed by the sensor unit 170 Module 145 may be further included.

7, the mobile robot 100b extracts a portion of the image acquired by the image acquisition unit 120 corresponding to the direction of the obstacle detected by the sensor unit 170 And may further include a separate image processing unit 125.

The mobile robot 100a according to the embodiment of FIG. 6 and the mobile robot 100b according to the embodiment of FIG. 7 have the same configuration except for the image processing module 145 and the image processing unit 125. FIG.

Alternatively, in some embodiments, the image acquisition unit 120 may directly extract a portion of the image, unlike the embodiments of FIGS.

The obstacle recognition module 144 learned by the machine learning has a high recognition rate as the learned object occupies a large portion of the input image data.

Accordingly, the present invention can enhance the recognition rate by extracting another region of the image acquired by the image acquisition unit 120 according to the direction of an obstacle detected by the sensor unit 170, such as an ultrasonic sensor, and using the extracted region as recognition data.

The obstacle recognition module 144 can recognize the obstacle based on the data learned by the machine learning in the extracted image.

The driving control module 141 may control driving of the driving unit 160 based on the recognized attributes of the obstacle.

If the obstacle is sensed in the right direction of the front surface of the main body, the control unit 140 extracts a lower right area of the image acquired by the image acquiring unit, detects the obstacle in the left direction of the front surface of the main body A lower left region of the image obtained by the image obtaining unit is extracted and a center lower region of the image obtained by the image obtaining unit is extracted when the obstacle is detected in the front direction of the main body can do.

In addition, the control unit 140 may control the extraction of the extraction target region in the image acquired by the image acquisition unit so as to correspond to the direction of the detected obstacle.

Meanwhile, the storage unit 150 may store input data for determining an obstacle attribute and data for learning the DNN.

The storage unit 150 may store an original image acquired by the image acquisition unit 120 and extracted images from which a predetermined region is extracted.

In addition, according to the embodiment, the weight and the biases constituting the DNN structure may be stored in the storage unit 150. [

Alternatively, according to an embodiment, the weight and the biases constituting the depth-based neural network structure may be stored in an embedded memory of the obstacle recognition module 144.

The obstacle recognizing module 144 may perform a learning process by using the extracted image as training data every time a part of an image acquired by the image acquiring unit 120 is extracted, After the extracted image is acquired, the learning process can be performed.

That is, the obstacle recognition module 144 updates the DNN structure such as a weight by adding a recognition result every time an obstacle is recognized, or acquires a predetermined number of training data The learning process can be performed with the training data to update the DNN structure such as the weight.

Alternatively, the mobile robots 100, 100a, and 100b may transmit the original image or the extracted image acquired by the image acquisition unit 120 to the predetermined server through the communication unit 190, and may receive data related to machine learning from the predetermined server Lt; / RTI >

In this case, the mobile robots 100, 100a, and 100b may update the obstacle recognition module 141 based on the data related to the machine learning received from the predetermined server.

Figure 8 is an illustration of a simplified internal block diagram of a server in accordance with an embodiment of the present invention.

8, the server 70 may include a communication unit 820, a storage unit 830, a learning module 840, and a processor 810.

The processor 810 can control the overall operation of the server 70. [

The server 70 may be a server operated by a home appliance manufacturer such as the mobile robots 100, 100a, 100b or a server operated by a service provider, or may be a kind of a cloud server.

The communication unit 820 can receive various data such as state information, operation information, operation information, and the like from a home appliance such as a mobile terminal, a mobile robot 100, 100a, and 100b, and a gateway.

The communication unit 820 can transmit data corresponding to the received various information to a home appliance such as a portable terminal, a mobile robot 100, 100a, or 100b, a gateway, and the like.

To this end, the communication unit 820 may include one or more communication modules such as an Internet module, a mobile communication module, and the like.

The storage unit 830 may store the received information and may include data for generating the corresponding result information.

In addition, the storage unit 830 may store data used for machine learning, result data, and the like.

The learning module 840 may serve as a learning appliance of the home appliances such as the mobile robots 100, 100a, and 100b.

The learning module 840 may include an artificial neural network such as a CNN (Convolutional Neural Network), a RNN (Recurrent Neural Network), a Deep Belief Network (DBN), and a Deep Neural Network (DNN) Neural networks can be learned.

As the learning method of the learning module 840, both unsupervised learning and supervised learning can be used.

Meanwhile, the controller 810 may control the artificial neural network structure of the home appliances such as the mobile robots 100, 100a, and 100b to be updated to the learned neural network structure after learning according to the setting.

9 to 12 are diagrams referred to in explanation of Deep Learning.

Deep Learning, a type of machine learning, is a multi-level, deep learning process based on data.

Deep learning can represent a set of machine learning algorithms that extract key data from multiple sets of data as they step up.

The deep learning structure may include an artificial neural network (ANN). For example, the deep learning structure may include a deep neural network (DNN) such as CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), DBN Be able to

Referring to FIG. 9, the ANN may include an input layer, a hidden layer, and an output layer. Each layer contains a plurality of nodes, and each layer is associated with the next layer. The nodes between adjacent layers can be connected to each other with a weight.

Referring to FIG. 10, a computer (machine) finds a certain pattern from inputted input data 1010 and forms a feature map. The computer (machine) can extract the low-level features 1020, the intermediate-level features 1030, and the high-level features 1040, recognize the objects, and output the results 1050.

The artificial neural network can be abstracted into higher-level features as it goes to the next layer.

9 and 10, each node may operate based on an activation model, and an output value corresponding to an input value may be determined according to an activation model.

The output value of any node, e.g., lower level feature 1020, may be input to the next layer associated with that node, e.g., a node of the intermediate level feature 1030. A node of the next layer, for example, a node of the intermediate level feature 1030, may receive values output from a plurality of nodes of the lower level feature 1020.

In this case, the input value of each node may be a value to which a weight is applied to the output value of the node of the previous layer. The weight can mean the link strength between nodes.

Also, the deep running process can be seen as a process of finding an appropriate weight.

On the other hand, the output value of any node, e.g., the intermediate level feature 1030, may be input to the next layer, e.g., the node of the higher level feature 1040, associated with that node. A node of the next layer, e.g., a node of the higher level feature 1040, may receive values output from a plurality of nodes of the intermediate level feature 1030.

The artificial neural network can extract feature information corresponding to each level using a learned layer corresponding to each level. The artificial neural network is sequentially abstracted, and a predetermined object can be recognized using the feature information of the highest level.

For example, if we look at the face recognition process by deep learning, the computer distinguishes between bright pixels and dark pixels according to the brightness of a pixel from the input image, distinguishes simple forms such as a border and an edge, You can distinguish things. Finally, the computer can grasp the form that defines the human face.

The deep running structure according to the present invention can utilize various known structures. For example, the deep learning structure according to the present invention may be a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), a Deep Belief Network (DBN), or the like.

Recurrent Neural Network (RNN) is widely used in natural language processing, and it is possible to construct an artificial neural network structure by stacking layers at each moment with an effective structure for time-series data processing that varies with time .

Deep Belief Network (DBN) is a deep-run structure consisting of multiple layers of deep-running RBM (Restricted Boltzman Machine). The Restricted Boltzman Machine (RBM) learning is repeated, and a DBN (Deep Belief Network) having a corresponding number of layers can be constituted by a certain number of layers.

CNN (Convolutional Neural Network) is a structure widely used in the field of object recognition, and will be described with reference to FIGS. 11 and 12. FIG.

CNN (Convolutional Neural Network) is a model that simulates a person's brain function based on the assumption that when a person recognizes an object, it extracts the basic features of the object, then undergoes complicated calculations in the brain and recognizes the object based on the result to be.

11 is a view showing a CNN (Convolutional Neural Network) structure.

The CNN (Convolutional Neural Network) may also include an input layer, a hidden layer, and an output layer.

A predetermined image 1100 is input to an input layer.

Referring to FIG. 11, a hidden layer is composed of a plurality of layers, and may include a convolution layer and a sub-sampling layer.

In the CNN (Convolutional Neural Network), various filters for extracting features of images through convolution operations and pooling or non-linear activation functions for adding nonlinear characteristics Is used.

Convolution is mainly used in filter processing in image processing and is used to implement a filter for extracting features from an image.

For example, as shown in FIG. 12, when the convolution operation is repeatedly performed on the whole image while moving the 3 × 3 window, a proper result can be obtained according to the weight value of the window.

Referring to FIG. 12A, when a convolution operation is performed on a predetermined area 1210 of a whole image using a 3 × 3 window, a result 1201 is displayed.

Referring to FIG. 12 (b), when the area 1220 in which the 3X3 window is shifted to the right by 1 is obtained again, a predetermined result 1202 is obtained.

That is, as shown in (c) of FIG. 12, the final result can be obtained by performing an operation on the entire image while moving a predetermined window.

The convolution layer can be used to perform convolutional filtering to filter information extracted from the previous layer using a filter of a predetermined size (e.g., the 3X3 window illustrated in FIG. 12).

The convolution layer performs a convolution operation on the input image data 1100 and 1102 using a convolution filter and generates feature maps 1101 and 1103 in which features of the input image 1100 are expressed do.

As a result of the convolutional filtering, filtering images corresponding to the number of filters can be generated according to the number of filters included in the convolution layer. The convolution layer may consist of the nodes included in the filtered images.

In addition, a sub-sampling layer paired with a convolution layer may include the same number of feature maps as a convolution layer that forms a pair.

The sub-sampling layer reduces the dimension of feature maps 1101 and 1103 through sampling or pooling.

The output layer recognizes the input image 1100 by combining various features expressed in the feature map 1104.

The obstacle recognizing module of the mobile robot according to the present invention can use the various deep learning structures described above. For example, although the present invention is not limited, a CNN (Convolutional Neural Network) structure widely used in object recognition in an image can be used.

On the other hand, the learning of the artificial neural network can be achieved by adjusting the weight of the inter-node connection line so that the desired output is obtained for a given input. Also, the artificial neural network can continuously update the weight value by learning. Back propagation can be used for artificial neural network learning.

Figs. 13 and 14 are diagrams referred to in explaining the obstacle recognition of the obstacle recognition module 144. Fig.

Referring to FIG. 13, the obstacle recognition module 144 can classify obstacles classified into classes such as an electric fan, a home theater, a multi-tap, a lamp base, a human body, and a barrier.

In addition, the obstacle recognition module 144 classifies and classifies the classes of the fan, the home theater, the multi-tap, the lamp base, and the humanoid as a dangerous obstacle super-class in a superordinate concept.

Also, the obstacle recognition module 144 can classify obstacle obstacles that can run straight, such as a barrier, into non-hazardous obstacle super-class, and recognize them.

Referring to FIG. 14A, the obstacle recognition module 144 recognizes the input image, and the recognition result that the fan has a confidence value of 0.95 and the home theater has a confidence value of 0.7 can be obtained . In this case, the obstacle recognition module 144 can output the fan, which is the recognition result having the higher confidence value, as the recognition result on the input image.

On the other hand, the confidence may be normalized to a range of 0.0 to 1.0.

Referring to FIG. 14B, the obstacle recognition module 144 recognizes the input image, and the recognition result that the fan has a confidence value of 0.35 and the home theater has a confidence value of 0.4 can be obtained.

For example, when the confidence value of 0.6 or less is set to not be recognized, the obstacle recognition module 144 can determine that the unknown data is not selected because the confidence values of the two recognition results are lower than the reference value have.

Referring to FIG. 14 (c), the obstacle recognition module 144 recognizes the input image, and the recognition result that the fan has a confidence value of 0.95 and the home theater has a confidence value of 0.9 can be obtained.

For example, when the recognition result having the confidence value of 0.9 or more is set to be selected as the final recognition result, the obstacle recognition module 144 does not select a specific recognition result because the trust values of the two recognition results are higher than the reference value , It can be judged as a dangerous obstacle, which is an upper concept.

Alternatively, even when the recognition result of the highest confidence value is set to be recognized when the difference in the confidence value is 0.15 or more, it can be determined that the dangerous obstacle is an upper concept.

On the other hand, the driving control module 141 can also control the driving unit 160 to avoid the dangerous obstacle and move the dangerous obstacle.

15 is a flowchart illustrating a method of controlling a mobile robot according to an embodiment of the present invention.

Referring to FIGS. 2 to 7 and 15, first, the mobile robots 100, 100a and 100b move according to commands or settings and perform cleaning (S1510).

The sensor unit 170 includes an obstacle detection sensor 131 and can detect an obstacle within a predetermined range in the moving direction (S1520).

If an obstacle is sensed through the sensor unit 170 during movement (S1520), the position information of the detected obstacle and the position information of the mobile robots 100, 100a and 100b may be stored in the storage unit 150 (S1530) .

In addition, a region having a predetermined size centered on the detected position of the obstacle may be registered as an obstacle region in a map stored in the storage unit 150, and may be stored and managed (S1530).

The position information of the mobile robot can be calculated from odometry information, and the odometry information is configured based on data sensed by the motion sensor of the sensor unit 170 .

Meanwhile, the image acquisition unit 120 may acquire a plurality of images in the obstacle region during movement (S1540).

The image acquisition unit 120 may register the obstacle area and continuously photograph the obstacle area in the obstacle area before the mobile robot 100, 100a, or 100b leaves the obstacle area to acquire a plurality of images .

In addition, the mobile robots 100, 100a, and 100b may acquire images of the entire obstacle region while avoiding the detected obstacle.

On the other hand, photographing the obstacle area while traveling in the obstacle area by a series of motions for searching for the entire obstacle area may significantly change the traveling path of the mobile robots 100, 100a, 100b.

Therefore, it is preferable that the moving robot 100, 100a, 100b normally travels and takes an image of the obstacle area and acquires images of the obstacle every time the robot 100 passes the obstacle area.

For example, assuming that the mobile robots 100, 100a, and 100b travel in a basic zigzag pattern that moves to a distance at which an obstacle is sensed in the direction opposite to the initial movement direction after straight forward, A plurality of images can be obtained by capturing a predetermined number of images while passing through the region, moving to the opposite direction, and then taking a picture while passing through the obstacle region again to acquire a predetermined number of images.

On the other hand, the obstacle recognizing module 144 sequentially recognizes obstacles to be obtained through the image acquiring unit 120 while moving the obstacle area based on the data learned by the machine learning Attribute (S1550).

The obstacle recognition module 144 may include an artificial neural network that is learned to recognize attributes such as the type of obstacle by machine learning, and may recognize the attributes of the obstacle based on the learned data (S1550).

For example, CNN (Convolutional Neural Network), which is one of the deep learning structures, is installed in the obstacle recognition module 144, and CNN (Convolutional Neural Network) learned the attributes of the obstacles included in the input data, Can be output.

The obstacle recognition module 144 performs obstacle recognition on one image obtained in the obstacle region and performs obstacle recognition on another image and then selects a more accurate recognition result from the two recognition results on the two images Can be stored.

Meanwhile, the obstacle recognition module 144 may compare the confidence values included in the two recognition results to select a more accurate recognition result among the recognition results. The current obstacle recognition result may be the current obstacle recognition result and the current obstacle recognition result may be the previous obstacle recognition result if the obstacle recognition is performed on the next image, Can be the current obstacle recognition result.

On the other hand, when the obstacle recognition module 144 recognizes the attributes of the obstacles sequentially with respect to the images obtained through the image acquisition unit 120 (S1550), the current obstacle recognition result is compared with the previous obstacle recognition result Can be repeatedly performed.

Whenever the mobile robots 100, 100a, 100b are traveling while passing through the obstacle area, the obstacle area can be photographed and images can be acquired.

The obstacle recognition module 144 may sequentially perform an image recognition process on the acquired images, and then compare the recognition results with previous obstacle recognition results to select a recognition result having a higher confidence value.

Preferably, the obstacle recognition module 144 compares the current obstacle recognition result with the previous obstacle recognition result, and if the current obstacle recognition result and the previous obstacle recognition result are the same, And the confidence may be raised to reflect a predetermined weight.

That is, if the current obstacle recognition result and the previous obstacle recognition result are the same, there is no need to change the obstacle recognition result. Therefore, while maintaining the result, the confidence value corresponding to the recognition result reflects the predetermined weight Can be increased.

Therefore, each time the same recognition result is obtained, the obstacle recognizing module 144 can control to store a confidence value by reflecting a predetermined weight.

If the current obstacle recognition result and the previous obstacle recognition result are not the same, the obstacle recognition module 144 may output a recognition result having a higher confidence value out of the current obstacle recognition result and the previous obstacle recognition result, The new obstacle recognition result can be registered.

That is, the obstacle recognition module 144 compares the two recognition results, and compares the recognition result with a later recognition result, leaving a recognition result with a higher confidence value.

Meanwhile, the obstacle recognition module 144 may determine the final property of the obstacle based on the sequentially recognized recognition results (S1560).

The obstacle recognition module 144 recognizes the obstacles sequentially and can determine the final property of the obstacle when the predetermined criterion is satisfied. That is, the obstacle recognition module 144 can output the current recognition result at the moment when the predetermined criterion is satisfied, as the final recognition result.

Since the obstacle recognition process is no longer performed after the predetermined criterion is satisfied, the current recognition result at the moment when the predetermined criterion is satisfied may be the last current recognition result.

For example, at the moment when the obstacle recognition is completed for all the areas of the obstacle area, that is, when the coverage of the obstacle area is 100%, the final current recognition result is determined as the final attribute of the obstacle .

According to an embodiment, the obstacle recognition process may be set to end when the recognition coverage reaches a predetermined reference value such as 90% or 95%.

Alternatively, the obstacle recognition module 144 compares the current obstacle recognition result with the previous obstacle recognition result, and if the current obstacle recognition result and the previous obstacle recognition result are the same, And the confidence value may be changed to a mean value of the confidence value of the current obstacle recognition result and the confidence value of the previous obstacle recognition result.

In this embodiment, when the current obstacle recognition result is identical to the previous obstacle recognition result, the confidence value for the same recognition result is determined as an average value of the existing confidence values of the two recognition results.

In the present embodiment, if the current obstacle recognition result and the previous obstacle recognition result are not the same, the obstacle recognition module 144 may determine that the current obstacle recognition result and the previous obstacle recognition result And to register the recognition result as the new current obstacle recognition result.

On the other hand, the driving control module 141 can control driving of the driving unit 160 based on the final attribute of the recognized obstacle (S1580).

For example, the travel control module 141 can control to travel by bypassing the obstacle when the recognized obstacle is an obstacle of a height not exceedable.

In addition, the travel control module 141 can be controlled so as to continuously travel straight ahead in the case of an obstacle of a height such that the recognized obstacle can pass, such as a low-level barrier.

In addition, the travel control module 141 can control the obstacle to travel while bypassing obstacles that are likely to be restrained even if the obstacle is a low-level obstacle such as a pedestal of a fan, human hair, a multi-tap, or a wire.

In step S1570, the obstacle recognition module 144 registers the obstacle as a dangerous obstacle or a non-hazardous obstacle in the map according to the final attribute of the determined obstacle.

For example, if the recognized obstacle is an obstacle of a height that can not be exceeded by the recognized obstacle, or if the obstacle is a low-height obstacle such as a pedestal, a human hair, a multi-tap, Recognized obstacles can be registered on the map as dangerous obstacles.

In addition, obstacles such as barriers that can be crossed even when driving straight ahead can be registered on the map as non-hazardous obstacles.

Thereafter, the mobile robot 100 can travel on the basis of the dangerous obstacles and the non-hazardous obstacle information registered in the map to avoid the non-hazardous obstacles.

16 is a flowchart illustrating a method of controlling a mobile robot according to an embodiment of the present invention.

16, the mobile robots 100, 100a, and 100b move according to an instruction or a setting (S1610). When an obstacle is detected through the sensor unit 170 during movement The location information of the detected obstacle and the location information of the mobile robots 100, 100a, and 100b may be stored in the storage unit 150 (S1630).

In operation S1630, an area having a predetermined size centered on the detected location of the obstacle may be registered as an obstacle area in a map stored in the storage unit 150, and may be stored and managed.

In addition, the image acquisition unit 120 may acquire a plurality of images in the obstacle area during movement (S1640).

On the other hand, the obstacle recognition module 144 determines the obstacle property of each of the plurality of images obtained through the image acquisition unit 120 in the obstacle region, based on the data learned by the machine learning (S1650), and the final attribute of the obstacle can be determined based on the recognized plurality of recognition results (S1660).

That is, in the present embodiment, the obstacle recognition module 144 does not perform the process of recognizing the obstacle to the image sequentially acquired by the obstacle recognition module 144 and comparing the result to the previous recognition result, After recognizing the obstacle for all of the acquired images, the final attribute of the obstacle can be finally determined based on the plurality of recognition results.

The obstacle recognition module 144 can determine the final attribute of the obstacle based on at least one of the frequency of the recognition result, the confidence value, and the average value of the confidence values, from the recognized plurality of recognition results.

For example, the obstacle recognition module 144 can determine, as the final attribute of the obstacle, the recognition result having the highest frequency from the recognized plurality of recognition results. That is, the obstacle attribute that is determined the most among the plurality of recognition results can be selected as the final attribute.

 Or the obstacle recognition module 144 can determine the recognition result with the highest confidence value as the final attribute of the obstacle. That is, the obstacle attribute determined in the recognition result having the highest confidence value among the plurality of recognition results can be selected as the final attribute.

Or the obstacle recognition module 144 may determine the recognition result corresponding to the highest average value among the average values of the confidence values corresponding to the same recognition results as the final attribute of the obstacle. That is, the obstacle recognition module 144 may group the plurality of recognition results among those having the same recognition result, average the trust values, and then determine the recognition result of the group having the highest average value as the final property have.

Also in this embodiment, the obstacle recognition module 144 may register and manage the obstacle as a dangerous obstacle or a non-hazardous obstacle according to the final attribute of the discriminated obstacle (S1670).

In addition, the driving control module 141 may control the driving of the driving unit 160 based on the final attribute of the recognized obstacle (S1680).

17 is a flowchart illustrating a method of controlling a mobile robot according to an embodiment of the present invention.

Referring to FIGS. 2 to 7 and 17, first, the mobile robots 100, 100a and 100b can move according to an instruction or a setting (S1710).

When the sensor unit 170 includes the ultrasonic sensor, the obstacle is recognized by sensing the reflected ultrasonic signal in step S1720. The mobile robots 100, 100a, and 100b detect the obstacle by the motion sensor of the sensor unit 170 The location information can be obtained based on the data (S1730).

The control unit 140 can recognize an obstacle through a process described with reference to FIGS. 15 and 16 on a plurality of images acquired in a predetermined area including a position where the obstacle is detected through the image obtaining unit 120 have.

The controller 140 can determine the attribute of the obstacle detected in the plurality of images acquired in the predetermined area including the position where the obstacle is sensed, based on the data learned by the machine learning.

In addition, the controller 140 can determine whether the detected obstacle is at a height that can be exceeded (S1740).

If the recognized obstacle is an obstacle of a height that can not be exceeded (S1740), the control unit 140 may control to turn around the obstacle after traveling 90 degrees (S1775).

Meanwhile, if the recognized obstacle is an obstacle of a height that can be exceeded (S1740), the fisher 140 can determine the attribute information of the detected obstacle (S1750). That is, the control unit 140 can determine whether the recognized obstacle is an obstacle that may be proceeded because the possibility of restraint is small.

If it is determined that the obstacle can be proceeded, the control unit 140 may control to continue to move straight (S1770).

Conventionally, it is determined whether or not the detected obstacle is a height that can be exceeded, and the vehicle runs straight in the case of a low-height obstacle.

However, in the case of an obstacle such as a wire, the mobile robot is wound around the wire and constrained.

In addition, the mobile robot attempts to escape from the restrained state by applying motion, which is shaken to the left or to the right, when it is restrained, but a safety accident such as peeling of the covering of the wire may occur.

However, the present invention can improve the reliability by recognizing the obstacle attribute information using the machine learning and the image information, and determining the driving pattern according to the recognized obstacle attribute.

In addition, the obstacle recognition module 144 may register the obstacle as a non-hazardous obstacle or a dangerous obstacle in the map according to the final attribute of the determined obstacle (S1760, S1765).

For example, if the recognized obstacle is an obstacle of a height that can not be exceeded by the recognized obstacle, or if the obstacle is a low-height obstacle such as a pedestal, a human hair, a multi-tap, The recognized obstacle can be registered in the map as a dangerous obstacle (S1765).

In addition, an obstacle such as a barrier that can be traversed even when traveling straight ahead can be registered in the map as a non-hazardous obstacle (S1760).

Thereafter, the mobile robot 100 can travel on the basis of the dangerous obstacles and the non-hazardous obstacle information registered in the map to avoid the non-hazardous obstacles.

18 to 28 are diagrams referred to in explaining the control method of the mobile robot according to the embodiment of the present invention.

Referring to FIG. 18, the mobile robot 100 can sense the obstacle 1800 while traveling. When the mobile robot 100 detects the obstacle 1800, the obstacle region can be set and registered so as to include the recognized obstacle vicinity.

19, the obstacle recognizing module 144 detects an obstacle area 1910 having a predetermined size (aXb) around the obstacle 1800 in the entire area 1900 on a map stored in the storage unit 150 It can be controlled to register.

For example, the obstacle area 1910 may be set to a rectangular shape having a total size of 1m x 1m with the front and rear angles of 50cm and the left and right angles of 50m around the obstacle 1800.

Meanwhile, the obstacle recognizing module 144 may control to store the position information of the mobile robot 100 at the time of recognizing the obstacle 1800 together with the registration of the obstacle area 1910. [ On the other hand, the position information of the mobile robot 100 may be information based on odometry information.

For example, the obstacle recognition module 144 may control to store the location of the obstacle and the location information of the mobile robot in the following storage format.

Storage format: (Op, Rp),

Op (X, Y) = Object Position,

Rp (X, Y) = Robot Position

Meanwhile, the image acquisition unit 120 may acquire a plurality of images in the obstacle region 1910 during movement.

The image obtaining unit 120 may continuously photograph the obstacle region 1910 before the mobile robot 100 leaves the obstacle region 1910 to acquire a plurality of images.

FIG. 20 shows that a predetermined region of the obstacle region 1910 is recognized by two photographs. The circles 2010 and 2020 shown in FIG. 20 represent recognition coverage, respectively, and their sizes and shapes may be changed according to the configuration and performance of the image acquisition unit 120. [

Referring to FIG. 21, the mobile robot 100 may photograph the obstacle 1800 at another position of the obstacle region 1910.

According to the present invention, it is possible to improve the accuracy of obstacle recognition by acquiring an image of the obstacle 1800 taken at various positions while moving the obstacle region 1910 and performing an obstacle recognition process for each of the acquired images .

Meanwhile, the obstacle recognizing module 144 can manage the inner area of the obstacle area 1910 by dividing the inner area of the obstacle area 1910 into a grid shape at regular intervals.

22 illustrates an obstacle region 2200 constructed in the form of a 4X4 grid.

The obstacle recognizing module 144 can distinguish and store an area in which an image within the obstacle region 2200 is captured and an obstacle is recognized.

For example, as shown in FIG. 22 (a), it is possible to distinguish the recognized area from the unrecognized area in the entire obstacle area 2200 by changing the value of the recognized predetermined area 2210, ) To a certain extent.

On the other hand, the recognized areas 2210, 2220, 2230, and 2240 may increase as the mobile robot 100 travels and recognizes an obstacle, as shown in FIG. 22 (b).

22C, when the recognition coverage reaches 100%, that is, when the obstacle recognition in the entire obstacle region 2200 is completed, the obstacle recognition process can be terminated.

Or, according to the setting, when the recognized coverage becomes 90% or 95%, the obstacle recognition process may be terminated.

23A and 23B illustrate an example in which the obstacle region 1910 is recognized up to 100% by the recognition coverage areas 2310 and 2320 of different sizes and the size of the recognition coverage areas 2310 and 2320 The shape may be changed according to the configuration and performance of the image acquiring unit 120.

In addition, the present invention can be set to end the obstacle recognition process when the obstacle is not related to the order in which the obstacle is photographed and the obstacle is recognized in the obstacle area 1910 and the predetermined coverage value reaches 90%, 95%, 100% have.

As described with reference to FIGS. 15 and 16, the obstacle recognition module 144 may be configured to determine, based on data learned through machine learning, The obstacle recognizing process can be performed on the plurality of acquired images.

According to an embodiment, it is possible to sequentially recognize recognition of an obstacle, compare a current recognition result with an obstacle result recognized in the past, and if two recognition results are the same, increase a confidence value of the recognition result, A higher recognition value can be registered as a current obstacle.

The obstacle can be registered with the recognition value with the highest confidence value at the moment when the recognition coverage around the obstacle becomes full by repeating this process.

After the final registration of the obstacle, the obstacle recognition is not proceeded while traveling in the same area, so that the ultrasonic sensor based travel can be performed.

The mobile robot can travel while recognizing the presence of an obstacle by the sensor unit 170 such as an ultrasonic sensor. At this time, if an obstacle is detected at the same position as the registered obstacle, it is possible to travel without proceeding to recognize additional obstacle. In addition, when an obstacle is detected or an unregistered obstacle is detected at a position different from the registered obstacle, the obstacle recognition process described above can be performed.

The control method of the mobile robot according to the present invention can be applied not only to the current obstacle situation detected by the sensor unit 170 but also to the same obstacle by using the obstacle information previously recognized and registered on the map of the storage unit 150 It prevents repeated collision and collision.

In addition, the control method of the mobile robot according to the present invention can prevent unnecessary additional obstacle recognition from being performed for previously registered obstacles.

24 and 25 illustrate some of the sequential obstacle recognition processes for the same obstacle according to the embodiment of the present invention.

Referring to FIGS. 24 and 25, the obstacle recognition module 144 recognizes the obstacle as a fan by recognizing the image acquired by the mobile robot in the first position Rp1. For example, the confidence value of the current recognition result recognized as a fan may be 0.8.

On the other hand, since the recognition result is the first recognition result, there is no comparison target, and the registration result corresponding to the current recognition result can also be a fan.

On the other hand, the obstacle recognition module 144 can obtain the recognition result determined by the fan having the confidence value of 0.7 based on the image acquired by the mobile robot at the second position Rp2.

The obstacle recognition module 144 can maintain the fan with the registration result corresponding to the current recognition result because the first recognition result and the second recognition result are the same.

In this case, the trust value corresponding to the current recognition result can use one of the trust values of the two recognition results as it is.

Alternatively, as shown in FIG. 24, an average value 0.75 obtained by averaging the first recognition result and the second recognition result using the confidence value corresponding to the current recognition result may be used.

More preferably, since the first recognition result and the second recognition result are the same, a confidence value of 0.835 upwards reflecting the predetermined weight value can be used as the confidence value corresponding to the current recognition result as shown in FIG.

Referring to FIGS. 24 and 25, the obstacle recognition module 144 determines that the obstacle recognition module 144 is determined as a lamp support having a reliability value of 0.7 based on the image acquired by the mobile robot in the third position Rp3. A recognition result can be obtained.

Referring to FIGS. 24 and 25, the obstacle recognition module 144 may compare the third recognition result with the previous recognition result.

Since the confidence value of the third recognition result is 0.7 and is lower than the previous recognition result 0.75 or 0.835, the obstacle recognition module 144 can select the fan having the higher confidence value as the registration result corresponding to the current recognition result.

23 and 24, the process of recognizing the obstacles and comparing the recognition results sequentially has been described. However, the present invention is not limited to the processes of collecting a plurality of recognition results as described with reference to FIG. 16, You may.

26 to 28 are diagrams referred to in explaining a process of acquiring an image of an obstacle while the mobile robot is traveling in an obstacle area and filling the recognition coverage.

Referring to FIG. 26, the image acquiring unit 120 may acquire a plurality of images by photographing an obstacle region 2600 registered at a predetermined size around a detected position (op) of the obstacle.

Meanwhile, the image acquisition unit 120 may acquire one or more images even during one travel in which the mobile robot 100 enters and leaves the obstacle region 2600.

For example, the travel until the mobile robot 100 enters the obstacle area 2600 (2611) and then departs (2615) can be divided into one travel.

FIG. 26 illustrates two travels 2610 and 2620 in which the mobile robot 100 enters and leaves the obstacle area 2600. FIG.

26, the mobile robots 100, 100a, and 100b can acquire images of the obstacle region 2600 while avoiding the detected obstacles and performing bypass travels 2612, 2613, and 2614 .

27 illustrates four traveling (2710, 2720, 2730, 2740) in which the mobile robot 100 enters and leaves an obstacle area 2700 in which the mobile robot 100 is registered.

The image acquiring unit 120 acquires a plurality of images of the obstacle region 2700 until the recognized coverage of the obstacle region 2700 is equal to or greater than a reference value set at 90%, 95%, 100%, etc. in the four travels 2710, 2720, 2730, Images can be acquired.

28 shows a state in which the obstacle area 2800 of the mobile robot 100 is registered in the obstacle area 2800 while avoiding the obstacle detected during one travel while leaving the obstacle area 2800 and performing detour travels 2812, 2813, And an image for the region 2800 is obtained.

Photographing the obstacle area while traveling in the obstacle area by a series of motions for searching for the entire obstacle area may significantly change the traveling path of the mobile robot 100. [

Therefore, it is preferable to photograph the obstacle area and obtain images of the obstacle every time the mobile robot 100 normally travels and passes through the obstacle area.

For example, assuming that the mobile robot 100 travels in a basic zigzag pattern in which the obstacle is detected in a direction opposite to the first movement direction after the straight ahead, the direction change, and the like, A plurality of images can be acquired by capturing a predetermined number of images, traveling to the opposite direction, and then returning to the obstacle area and capturing a predetermined number of images.

According to at least one of the embodiments of the present invention, the mobile robot can accurately determine the property of the obstacle and adjust the traveling pattern according to the obstacle property, thereby performing highly reliable obstacle recognition and avoidance operations.

According to at least one of the embodiments of the present invention, it is possible to improve the stability of the mobile robot itself and the user's convenience by performing operations such as forward, backward, stop, and bypass according to the recognition result of the obstacle, And a control method thereof can be provided.

In addition, according to at least one embodiment of the present invention, it is possible to provide a mobile robot and a control method thereof that can accurately recognize an attribute of an obstacle based on machine learning.

In addition, according to at least one of the embodiments of the present invention, the mobile robot can efficiently perform machine learning and extract data that can be used for obstacle attribute recognition.

The mobile robot according to the present invention is not limited to the configuration and method of the embodiments described above, but the embodiments may be modified such that all or some of the embodiments are selectively combined .

Meanwhile, the control method of the mobile robot according to the embodiment of the present invention can be implemented as a code that can be read by a processor on a recording medium readable by the processor. The processor-readable recording medium includes all kinds of recording apparatuses in which data that can be read by the processor is stored. Examples of the recording medium that can be read by the processor include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet . In addition, the processor-readable recording medium may be distributed over network-connected computer systems so that code readable by the processor in a distributed fashion can be stored and executed.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It should be understood that various modifications may be made by those skilled in the art without departing from the spirit and scope of the present invention.

Body: 110
Image acquisition unit: 120
Front camera: 120a
Upper camera: 120b
Control panel: 137
Control section: 140
Storage: 150
Driving section: 160
Sensor part: 170
Communication department: 190

Claims (19)

A traveling part for moving the main body;
An image acquiring unit acquiring an image around the main body;
A sensor unit including at least one sensor for detecting an obstacle during the movement;
When the sensor unit detects an obstacle, the position information of the detected obstacle and the position information of the mobile robot are stored, and a region having a predetermined size centered on the detected position of the obstacle is registered as an obstacle region in a map A storage unit for storing an image acquired by the image acquisition unit in the obstacle area; And
An apparatus for recognizing an obstacle, comprising: an obstacle recognition unit for recognizing an obstacle property sequentially for images obtained through the image acquisition unit in the obstacle area based on data learned by machine learning; And an obstacle recognition module for determining a final property of the obstacle based on the obstacle recognition result. The method according to claim 1,
The obstacle recognition module includes:
And repeatedly performing a process of comparing the current obstacle recognition result and the previous obstacle recognition result when the attributes of the obstacle are sequentially recognized for the images obtained through the image acquisition unit. 3. The method of claim 2,
The obstacle recognition module includes:
By comparing the current obstacle recognition result with the previous obstacle recognition result,
Wherein if the current obstacle recognition result and the previous obstacle recognition result are the same, the current obstacle recognition result is maintained, and the confidence value is increased by reflecting a predetermined weight,
If the current obstacle recognition result and the previous obstacle recognition result are not the same, control to register a recognition result having a higher confidence value in the current obstacle recognition result and the previous obstacle recognition result as the new current obstacle recognition result Wherein the mobile robot is a mobile robot. 3. The method of claim 2,
The obstacle recognition module includes:
And the final current recognition result is determined as the final property of the obstacle when the obstacle recognition is completed for all the areas of the obstacle area. 3. The method of claim 2,
The obstacle recognition module includes:
By comparing the current obstacle recognition result with the previous obstacle recognition result,
Wherein if the current obstacle recognition result and the previous obstacle recognition result are the same, the current obstacle recognition result is maintained, and the confidence value is the confidence value of the current obstacle recognition result and the previous obstacle recognition result To an average value of the trust values,
If the current obstacle recognition result and the previous obstacle recognition result are not the same, control to register a recognition result having a higher confidence value in the current obstacle recognition result and the previous obstacle recognition result as the new current obstacle recognition result Wherein the mobile robot is a mobile robot. The method according to claim 1,
The obstacle recognition module includes:
And registers the obstacle as a dangerous obstacle or a non-hazardous obstacle in the map according to a final attribute of the discriminated obstacle. The method according to claim 1,
Wherein,
And a traveling control module for controlling the driving of the traveling part according to a final property of the determined obstacle. The method according to claim 1,
The location information of the mobile robot may include:
And the odometry information is calculated from the odometry information. A traveling part for moving the main body;
An image acquiring unit acquiring an image around the main body;
A sensor unit including at least one sensor for detecting an obstacle during the movement;
A sensor unit including at least one sensor for detecting an obstacle during the movement;
When the sensor unit detects an obstacle, the position information of the detected obstacle and the position information of the mobile robot are stored, and a region having a predetermined size centered on the detected position of the obstacle is registered as an obstacle region in a map A storage unit for storing an image acquired by the image acquisition unit in the obstacle area; And
An obstacle detection apparatus for recognizing an obstacle attribute for each of a plurality of images acquired through the image acquisition unit in the obstacle region based on data learned by machine learning, And an obstacle recognition module for determining a final property of the obstacle. 10. The method of claim 9,
The obstacle recognition module includes:
And judges a final attribute of the obstacle based on at least one of the frequency of the recognition result, the confidence value, and the average value of the confidence values, from the recognized plurality of recognition results. 10. The method of claim 9,
The obstacle recognition module includes:
A recognition result corresponding to the highest average value among the recognition results having the highest frequency or the average value of the confidence values corresponding to the same recognition results with the highest confidence value or the same recognition results is obtained from the recognized plurality of recognition results, And judges it as a final attribute. 10. The method of claim 9,
The obstacle recognition module includes:
And registers the obstacle as a dangerous obstacle or a non-hazardous obstacle in the map according to a final attribute of the discriminated obstacle. 10. The method of claim 9,
Wherein,
And a traveling control module for controlling the driving of the traveling part according to a final property of the determined obstacle. 10. The method of claim 9,
The location information of the mobile robot may include:
And the odometry information is calculated from the odometry information. Sensing an obstacle while moving through the sensor unit;
When the sensor unit detects an obstacle, it stores position information of the detected obstacle and position information of the mobile robot, and registers an area having a predetermined size centering on the detected position of the obstacle as an obstacle area in a map step;
Obtaining a plurality of images in the obstacle area during movement;
Recognizing an attribute of an obstacle sequentially for images obtained through the image acquiring unit while moving in the obstacle area based on data learned by machine learning;
Determining a final attribute of the obstacle based on the sequentially recognized plurality of recognition results; And
And controlling driving of the traveling section based on the final attribute of the recognized obstacle. 16. The method of claim 15,
The step of recognizing the attributes of the obstacles sequentially includes:
And a step of repeatedly performing a process of comparing the current obstacle recognition result and the previous obstacle recognition result when the attributes of the obstacle are sequentially recognized for the images obtained through the image acquisition unit . 17. The method of claim 16,
The step of recognizing the attributes of the obstacles sequentially includes:
By comparing the current obstacle recognition result with the previous obstacle recognition result,
Wherein if the current obstacle recognition result and the previous obstacle recognition result are the same, the current obstacle recognition result is maintained, and the confidence value is increased by reflecting a predetermined weight,
And a recognition result having a higher confidence value than the current obstacle recognition result and the previous obstacle recognition result is registered as the new current obstacle recognition result if the current obstacle recognition result and the previous obstacle recognition result are not the same Of the mobile robot. 17. The method of claim 16,
Wherein the step of determining a final attribute of the obstacle comprises:
And the final current recognition result is determined as the final property of the obstacle when the obstacle recognition is completed for all the areas of the obstacle area. 16. The method of claim 15,
And registering the obstacle as a dangerous obstacle or a non-dangerous obstacle in the map according to a final attribute of the discriminated obstacle.

Images