JP2017196678A - Robot motion control device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a robot motion control device which makes a robot execute motion of a user without making the user executing the motion feel uncomfortable.SOLUTION: A robot motion control system RS includes a robot RT, a robot motion input device TM, and a robot motion control device 100. A CPU of the robot motion control device 100 executes determination processing of a candidate object of a robot motion target, decision processing of an object of the robot motion target, calculation processing of user motion track, calculation processing of robot motion track, and generation processing of robot motion control information. Thereby, the motion of the user can be securely executed by the robot without making the user executing the motion feel uncomfortable.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a robot operation control apparatus that controls the operation of a robot, and more particularly to an apparatus that causes a user's operation to be executed as a robot operation.

  The operation of the conventional two-dimensional CCD image pickup apparatus will be described using the interaction operation system shown in FIG. In the interaction operation system, sensor information is detected on the robot 100 side in order to provide an unconstrained and intuitively interlocking robot operation system in which the action of the operator and sensory information detected by the robot 100 influence each other. A sensory information detection unit 121 that transmits sensory information, a sensory information output unit 130 that transmits sensory information, and a robot drive device 140 that drives the robot. The operator gives the sensory information transmitted from the robot 100 to the operator. The sensory information imparting unit 300 and the robot teaching device 400 including the multi-joint structure 410, the control unit 420, and the teaching information output unit 430 are mounted. The multi-joint structure 410 has a potentiometer 413 in the movable unit 412. And a CPU 414 that processes the sensor signal of the potentiometer 413. And the teaching device output unit 430 that transmits the processed sensor signal, so that the operation of the robot can be performed regardless of the physical condition of the operator without restricting the operation of the operator. (See Patent Document 1 above).

JP 2013-91114 A

  The above interaction operation system has the following points to be improved. In the interaction device, a control delay due to the time required for communication between the robot 100 and the robot teaching device 400 occurs, the operator feels uncomfortable during the operation, and the operation speed of the operator becomes slow. There is. Further, since there is a difference in mechanical structure between the robot 100 and the robot teaching device 400, an error occurs in the operation position and the operation posture, and the robot 100 can execute the operation intended by the user. There is a point that should be improved. Furthermore, since the sensory presentation provided to the user from the robot 100 is insufficient, the operator cannot accurately grasp the operation environment of the robot 100, for example, the distance and position between the operation target and the robot 100, and the robot 100 There is a point to be improved that an operation intended by the user cannot be executed. That is, these problems have a point that should be improved, which prevents the operator from intuitively operating the robot 100 and, as a result, reduces work efficiency.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a robot operation control device that allows a user who performs an operation to perform the operation of the user without causing any uncomfortable feeling.

  Means for solving the problems in the present invention and the effects of the invention will be described below.

  A robot motion control device according to the present invention is a robot motion control device that causes a robot to perform a robot motion that is a motion corresponding to user motion information indicating a user motion that is a user motion, the robot in the robot An operation target candidate object determining unit that determines an operation target candidate object that can be a target of the robot operation from robot image information indicating an image of the direction of the robot operation, and a target object of the robot operation from the operation target candidate object A robot motion target object determining means for determining a motion target object, a user motion trajectory calculating means for calculating a user motion trajectory corresponding to the user motion information using the user motion information, Robot motion trajectory calculation that calculates a robot motion trajectory that is the trajectory of the robot motion using the user motion trajectory Stage, using the robot operation track, having a robot operation control information generating means for generating a robot motion control information for controlling the operation of the robot.

  Thereby, the user's operation can be surely executed by the robot without a sense of incongruity to the user who executes the operation.

  In the robot motion control device according to the present invention, the robot motion target object determining means determines the motion target object from the image using line-of-sight information indicating a user's line of sight with respect to the image of the robot image information. It is characterized by.

  Thus, the user can easily determine the motion target object simply by directing his / her line of sight toward the motion target object. Therefore, the user's work efficiency is increased.

  In the robot motion control device according to the present invention, the robot motion target object determining means calculates the direction of the user's motion with respect to the motion target candidate object as a probability distribution using a probability filter. The operation target object is determined.

  As a result, it is possible to determine the movement target object while looking at the movement direction of the user. That is, even if the motion target object is determined by mistake, the correct motion target object can be determined in a short time.

  In the robot motion control apparatus according to the present invention, the robot motion trajectory calculating means changes a user motion vector that is a user motion vector in the user motion trajectory to obtain a robot motion vector that is a robot motion vector. It is characterized by calculating.

  Thereby, the user's operation can be surely executed by the robot without a sense of incongruity to the user who executes the operation.

  The robot operation control program according to the present invention is a robot operation that causes a computer to function as a robot operation control device that causes a robot to execute a robot operation that is an operation corresponding to user operation information indicating a user operation that is a user operation. An operation target for determining an operation target candidate object that can be a target of the robot operation from robot image information indicating an image of a direction of the robot operation in the robot. Candidate object determining means, robot action target object determining means for determining an action target object to be the action object of the robot from the action target candidate object, and corresponding to the user action information using the user action information User motion to calculate a user motion trajectory that is the trajectory of the motion Trajectory calculating means, robot motion trajectory calculating means for calculating a robot motion trajectory that is the trajectory of the robot motion using the user motion trajectory, and robot motion control for controlling the motion of the robot using the robot motion trajectory It functions as a robot motion control information generating means for generating information.

Thereby, the user's operation can be surely executed by the robot without a sense of incongruity to the user who executes the operation.

1 is a diagram showing a robot motion control apparatus 100 that is an embodiment of a robot motion control apparatus according to the present invention. FIG. It is a figure which shows the hardware constitutions of robot RT. It is a figure which shows the hardware constitutions of the robot motion input apparatus TM. It is a figure which shows the hardware constitutions of the multi joint structure TM10. It is a hardware configuration of the line-of-sight information acquisition device TM30. It is a figure which shows the hardware constitutions of the robot operation control apparatus. It is a flowchart which shows a robot operation | movement control process. It is a flowchart which shows a robot operation | movement control process. It is a flowchart which shows a robot operation | movement control process. It is a flowchart which shows a robot operation | movement control process. It is a flowchart which shows a robot operation | movement control process. It is a flowchart which shows a robot operation | movement control process. It is a figure which shows the conventional robot operation control apparatus.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  A robot motion control apparatus according to the present invention will be described using the robot motion control apparatus 100 shown in FIG. 1 as an example.

First Hardware Configuration The robot motion control device 100 functions as a part of the robot motion control system RS. The robot motion control system RS includes a robot RT, a robot motion input device TM, and a robot motion control device 100. When the user inputs an operation to be performed by the robot RT via the robot motion input device TM, the robot motion control system RS receives the robot motion control device 100 via the robot motion input device TM. The operation is adjusted so that the robot RT can operate, and is output to the robot RT. The robot RT executes the adjusted operation. Thereby, it is possible to cause the robot RT to surely execute the user's operation without feeling uncomfortable for the user who performs the operation using the robot operation input device TM.

1. Hardware Configuration of Robot RT A hardware configuration of the robot RT will be described with reference to FIG. The robot RT includes a robot body RT10 and a robot visual image acquisition device RT30. The robot main body RT10 has a trunk portion RT11, a head portion RT12, a neck portion RT13, a shoulder portion RT14, an arm portion RT15, an elbow portion RT16, a wrist portion RT17, a finger portion RT18, and a base portion RT19. The robot body RT10 has a drive unit that moves each part in a predetermined direction, such as a drive unit that rotates the head RT12 in the vertical direction and a drive unit that rotates the neck part RT13 in the left-right direction. The robot body RT10 is fixed at a predetermined position through the base portion RT19. In the robot body RT10, an operation reference point PT0 corresponding to an operation reference point PT1 (described later) of the multi-joint structure TM10 is set. When the robot main body RT10 acquires the robot operation control information (described later), the robot main body RT10 operates the torso RT11 to the finger RT18 to execute an operation corresponding to the robot operation control information.

  The robot visual image acquisition device RT30 has a stereo camera and acquires a visual image with respect to human eyes. The robot visual image acquisition device RT30 is disposed at a position corresponding to the human eye in the head RT12 of the robot main body RT10. The robot visual image acquisition device RT30 acquires an image in the direction of operation of the robot body RT10. The robot visual image acquisition device RT30 transmits the acquired visual image as robot visual image information.

2. Hardware Configuration of Robot Motion Input Device TM The hardware configuration of the robot motion input device TM will be described with reference to FIG. The robot motion input device TM includes a multi-joint structure TM10 and a robot line-of-sight information acquisition device TM30. The multi-joint structure TM10 is mounted in close contact with the back, back of the head, arms, and fingers of the operator, and is used to input an operation to the robot body RT10 by changing the shape in conjunction with the operation of the operator. Device.

  The multi-joint structure TM10 will be described with reference to FIG. The multi-joint structure TM10 includes a large number of joint bodies TM11. The joint body TM11 has a cylindrical casing. The joint body TM11 has a movable part TM12 and a potentiometer TM13 inside the housing. Movable part TM12 is arrange | positioned at the both ends of joint body TM11, ie, a connection part with adjacent joint body TM11. The movable part TM12 rotates about any one of the X-axis direction, the Y-axis direction, and the Z-axis direction as a central axis. The potentiometer TM13 is arranged corresponding to the movable part TM12. The potentiometer TM13 measures the amount of change (rotation amount) of the angle of the movable part TM12. Further, a part of the joint body TM11 constituting the multi-joint structure TM10 has a CPUTM 14 with a CAN communication function. In the multi-joint structure TM10, an operation reference point PT1 that is a reference for the position of the joint body TM11 is set.

  In the multi-joint structure TM10, the shape of the multi-joint structure TM10 changes in conjunction with the operation of the operator, and the angle of the movable part TM12 changes. For each joint body TM11, the initial position with respect to the motion reference point PT1 and the position after the shape change are detected as an angle change value of the potentiometer TM13. The angle change value of the potentiometer TM13 detected in each joint body TM11 is transmitted and accumulated in the adjacent CPU TM14. The CPU TM 14 transmits the transmitted angle change value of the potentiometer TM 13 to the robot motion control apparatus 100 as multi-joint structure state information as user motion information.

  The line-of-sight information acquisition device TM30 will be described with reference to FIG. In FIG. 5, A shows a state when the line-of-sight information acquisition device TM30 is viewed from the side surface, and B shows a state when the line-of-sight information acquisition device TM30 is viewed from the side surface. As shown in FIG. 5A, the line-of-sight information acquisition device TM30 is mounted on the user's head as a so-called head-mounted display. As illustrated in FIG. 5B, the line-of-sight information acquisition device TM30 includes a display TM31 and a line-of-sight detection unit TM33. The display TM31 displays a predetermined image. The line-of-sight detection unit TM33 detects the line of sight of the user with respect to the image displayed on the display TM31 and acquires the line-of-sight information. The line-of-sight information acquisition device TM30, when acquiring the robot visual recognition image information from the robot visual recognition image acquisition device RT30 of the robot RT via the robot motion control device 100, displays an image corresponding to the acquired robot visual recognition image information on the display. To do.

3. Hardware Configuration of Robot Motion Control Device 100 A hardware configuration of the robot motion control device 100 will be described with reference to FIG. The robot operation control device 100 includes a CPU 100a, a memory 100b, a hard disk drive 100c (hereinafter referred to as HDD 100c), a keyboard 100d, a mouse 100e, a display 100f, an optical drive 100g, and a communication circuit 100h.

  The CPU 100a performs processing based on other applications such as an operating system (OS) and a robot operation control program recorded in the HDD 100c. The memory 100b provides a work area for the CPU 100a. The HDD 100c records and holds programs of other applications such as an operating system (OS) and a robot operation control program, and various data.

The keyboard 100d and the mouse 100e accept external commands. The display 100f displays an image such as a user interface. The optical drive 100g reads data from the optical medium, such as reading the robot movement control program from the optical medium 100p in which the robot movement control program is recorded, and reading other application programs from other optical media. Read. The communication circuit 100h transmits / receives information to / from an external communication device such as the robot RT or the robot motion input device TM.

Operation of Second Robot Operation Control Device 100 The operation of the robot operation control device 100 will be described with reference to the flowchart shown in FIG. The CPU 100a of the robot motion control apparatus 100 includes a robot motion target candidate object determination process (S701), a robot motion target object determination process (S703), a user motion trajectory calculation process (S705), a robot motion trajectory calculation process (S707), and Then, the robot operation control information generation process (S709) is executed. Each process will be described below.

1. Robot Motion Target Candidate Object Determination Process The robot motion target candidate object determination process will be described with reference to the flowchart shown in FIG. The robot operation target candidate object determination process refers to a process of determining an operation target candidate object (described later) that can be an operation target of the robot body RT10 from the robot visual image information acquired from the robot visual image acquisition device RT30.

  First, the robot visual image acquisition device RT30 of the robot RT transmits the acquired robot visual image information to the robot motion control device 100.

  When the CPU 100a of the robot motion control apparatus 100 obtains the robot viewing image information via the communication circuit 100h (S801), the CPU 100a determines the position and shape of the object displayed in the robot viewing image using an image processing technique. (S803). The CPU 100a generates motion target candidate object information using the object whose position and shape have been determined as motion target candidate objects, and stores and holds the information in the memory 100b (S805). The CPU 100a transmits the acquired robot visual image information to the line-of-sight information acquisition device TM30 worn by the user via the communication circuit 100h (S807).

2. Robot Motion Target Object Determination Process The motion target object determination process will be described with reference to FIG. The motion target object determination process is a process of determining a motion target object from motion target candidate objects.

  When the line-of-sight information acquisition device TM30 acquires the robot visual recognition image information, the visual line information acquisition device TM30 generates a robot visual recognition image and displays it on the display. The user determines an object to be operated (hereinafter referred to as an operation target object) from the robot visual image displayed on the display, and moves the line of sight to the operation target object. When the user's head moves as the user's line of sight moves, the shape of the multi-joint structure TM10 changes in conjunction. The multi-joint structure TM10 transmits the angle change value of the potentiometer TM13 indicating the change in position of the movable portion TM12 in each joint body TM11 to the robot motion control apparatus 100 via the CPU TM14 as multi-joint structure state information. At the same time, the line-of-sight information acquisition apparatus TM30 transmits the acquired line-of-sight information of the user to the robot motion control apparatus 100.

When the CPU 100a of the robot motion control apparatus 100 acquires the multi-joint structure state information and the line-of-sight information via the communication circuit 100h (S901), whether or not the user starts to operate, for example, the user's hand operates. It is determined whether or not it has started (S903). Note that whether or not the user has started to operate is determined based on whether or not the state of the multi-joint structure TM10 has changed from the multi-joint structure state information acquired so far. When the CPU 100a determines that the user has started to move, the CPU 100a approximates it as a motion obtained by adding an acceleration correction term to the uniform acceleration motion, and calculates the motion of the user (S905). Specifically, the movement of the user's hand is calculated using the following equation.

  Next, the CPU 100a predicts a future user's motion direction using the calculated user's motion, and calculates a motion direction prediction probability distribution (S907). When calculating the probability distribution of the motion direction prediction, a particle filter which is a generally used probability filter is used.

The CPU 100a calculates the motion target possibility index for each motion target candidate object in consideration of the visual line direction of the operator indicated by the visual line information (S909). Here, the motion target possibility index is an index indicating the possibility that the user will be the motion target for the motion target candidate object included in the motion target candidate object information. Specifically, the CPU 100a calculates a motion target possibility index for each motion target candidate object using the following equation.

  In calculating the motion target possibility index for each motion target candidate object, the CPU 100a uses line-of-sight information, motion target candidate object information, multi-joint structure state information, and the like stored in the memory 100b.

  The CPU 100a determines that the motion target candidate object corresponding to the largest motion target possibility index is the motion target object (S911). If the CPU 100a determines in step S903 that the user is not operating, the CPU 100a proceeds to step S1203, and generates robot motion control information based on the currently acquired multi-joint structure state information (see FIG. 12). ).

3. User Motion Trajectory Calculation Process The user motion trajectory calculation process will be described with reference to FIG. The user motion trajectory calculation processing refers to processing for calculating a user motion trajectory that is a motion trajectory corresponding to the multi-joint structure state information using the multi-joint structure state information.

The CPU 100a acquires multi-joint structure state information from the memory 100b (S1001). The CPU 100a calculates the motion trajectory of the user with respect to the motion target object determined in step S911 using the acquired multi-joint structure state information (S1003). When calculating the user's motion trajectory, the measured human user's motion trajectory is based on the minimum jerk model using the minimum jerk model known as the human motion model (see the following equation). To fit and predict the user's future motion trajectory.

Specifically, x0, t0, xf, and tf that minimize the evaluation value E are calculated using the following equation.

4). Robot Motion Trajectory Calculation Processing Robot motion trajectory calculation processing will be described with reference to FIG. The robot motion trajectory calculation processing refers to processing for converting a user motion represented by the user motion trajectory into a robot motion trajectory representing motion in the robot body RT10.

The CPU 100a calculates a robot motion vector that is a motion vector of the robot in which the user motion vector (operation speed and motion direction) in the user motion trajectory is slightly changed (S1101). Specifically, the robot motion vector is calculated using the following formula:

Further, the CPU 100a optimizes the evaluation value J of the following equation in real time so as to minimize the difference between the user motion vector and the robot motion vector (S1103). Note that the CPU 100a optimizes using a calculation method of model predictive control, and uses u (t) obtained as a result of the optimization in each control cycle.

  This minimizes the uncomfortable feeling between the user motion and the motion of the robot body RT10 caused by the difference between the user motion vector and the robot motion vector.

5. Robot Operation Control Information Generation Processing Robot operation control information generation processing will be described with reference to FIG. The robot operation control information generation process is a process for generating information for causing the robot body RT10 to execute an operation represented by a robot operation trajectory.

  The CPU 100a generates robot motion control information for actually operating the robot body RT10 using the robot motion vector calculated in step S1101 (S1201). The CPU 100a transmits the generated robot control information to the robot body RT10 via the communication circuit 100h (S1203).

When the robot body RT10 acquires the robot operation control information, the robot body RT10 operates each unit in accordance with the acquired robot operation control information.

[Other Embodiments]
(1) Robot visual recognition image information: In the above-described first embodiment, the robot visual recognition image acquisition device RT30 arranged on the head RT12 of the robot main body RT10 is used as robot visual recognition image information as an image in the direction in which the robot main body RT10 operates. However, the present invention is not limited to the example. For example, a separate image from the robot body RT10 may be used to acquire an image in the direction in which the robot body RT10 operates using a camera arranged at a predetermined position.

  (2) Acquisition of line-of-sight information: In the above-described first embodiment, the movement target object is determined from the movement target candidate objects using the line-of-sight information regarding the user's line of sight acquired using the line-of-sight information acquisition device TM30. If information that can identify the motion target object such as detailed multi-joint structure state information can be acquired as user motion information, the motion target object may be determined without using the line-of-sight information.

  (3) Input of user's motion: In the above-described first embodiment, the user's motion is input using the multi-joint structure TM10. However, the user's motion is acquired and the robot body RT10 is operated. Anything that can be input is not limited to the example. For example, the motion of the user may be detected using a motion sensor, or the motion of the user may be acquired from an image, and the motion of the user may be detected using image analysis.

  (4) Hardware configuration of the multi-joint structure TM10: In the above-described first embodiment, the joint body TM11 of the multi-joint structure TM10 is assumed to have one movable part TM12, that is, one degree of freedom. Two or more movable parts TM12 may be included and two or more degrees of freedom may be provided. Moreover, you may make it arrange | position arbitrary movable parts, without arrange | positioning a movable part in order centering on each axis | shaft of an X-axis, a Y-axis, and Z-axis. Furthermore, not all the joint bodies TM11 have the movable part TM12, and some joint bodies TM11 may have the movable part TM12.

  (5) Hardware configuration of the robot motion control apparatus 100: In the above-described first embodiment, the robot motion control apparatus 100 is formed using the CPU 100a or the like. It is not limited to things. For example, a predetermined program may be executed by the robot motion control apparatus 100 using a dedicated logic circuit.

(6) Robot operation control program: In the first embodiment, the robot operation control program realizes the function shown in FIG. 7 by the processing according to the flowcharts shown in FIGS. The present invention is not limited to the examples as long as the functions shown are realized.

  The robot motion control device according to the present invention can be used, for example, in a robot motion control system that controls the motion of a humanoid robot by human motion.

100 Robot motion control device 100a CPU
100b Memory 100c Hard disk drive 100d Keyboard 100e Mouse 100f Display 100g Optical drive 100h Communication circuit 100p Optical media RT Robot RT10 Robot main body RT11 Body RT12 Head RT13 Neck RT14 Shoulder RT15 Arm RT16 Elbow RT17 Wrist RT18 RT19 Base part RT30 Robot visual image acquisition device TM Robot motion input device TM10 Multi-joint structure TM11 Joint body TM12 Movable part TM13 Potentiometer TM14 CPU
TM30 Robot Gaze Information Acquisition Device TM30 Gaze Information Acquisition Device TM31 Display TM33 Gaze Detection Unit RS Robot Motion Control System

Claims (5)

A robot motion control device that causes a robot to execute a robot motion that is a motion corresponding to user motion information indicating a user motion that is a user motion,
Action target candidate object judging means for judging an action target candidate object that can be a target of the robot action from robot image information indicating an image of the direction of the robot action in the robot;
A robot motion target object determining means for determining a motion target object to be a motion target object of the robot from the motion target candidate object;
A user motion trajectory calculating means for calculating a user motion trajectory that is a trajectory of the motion corresponding to the user motion information using the user motion information;
Robot motion trajectory calculating means for calculating a robot motion trajectory that is the trajectory of the robot motion using the user motion trajectory,
Robot operation control information generating means for generating robot operation control information for controlling the operation of the robot using the robot operation trajectory,
A robot motion control device having In the robot motion control device according to claim 1,
The robot motion target object determining means includes
Using the line-of-sight information indicating the line of sight of the user with respect to the image of the robot image information, determining the operation target object from the image;
A robot motion control device characterized by the above. In the robot motion control device according to claim 1 or 2,
The robot motion target object determining means includes
Determining the motion target object from the image by calculating a direction of the motion of the user with respect to the motion target candidate object as a probability distribution using a probability filter;
A robot motion control device characterized by the above. In any one of the robot motion control apparatuses according to claims 1 to 3,
The robot motion trajectory calculating means includes
Changing a user motion vector that is a user motion vector in the user motion trajectory to calculate a robot motion vector that is a robot motion vector;
A robot motion control device characterized by the above. A robot operation control program that causes a computer to function as a robot operation control device that causes a robot to execute a robot operation that is an operation corresponding to user operation information indicating a user operation that is a user operation,
The robot motion control program is
The computer,
Action target candidate object judging means for judging an action target candidate object that can be a target of the robot action from robot image information indicating an image of the direction of the robot action in the robot;
A robot motion target object determining means for determining a motion target object to be a motion target object of the robot from the motion target candidate object;
A user motion trajectory calculating means for calculating a user motion trajectory that is a trajectory of the motion corresponding to the user motion information using the user motion information;
Robot motion trajectory calculating means for calculating a robot motion trajectory that is the trajectory of the robot motion using the user motion trajectory,
Robot operation control information generating means for generating robot operation control information for controlling the operation of the robot using the robot operation trajectory,
To function as a
A robot motion control program characterized by

Images