JP6849262B1 - Information processing method, information processing system, program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily and quickly detect a contour shape of an object from three-dimensional point cloud data, and to be able to control a robot arm and a tool by using the contour shape. SOLUTION: A step of acquiring three-dimensional point cloud data from an object by a sensor, a step of designating contour point cloud data constituting an object contour from the three-dimensional point cloud data, and a step of designating the contour point cloud data. , A step of acquiring tool control information including tool position information and tool attitude information for specifying a tool trajectory of a tool connected to an arm of a working robot, and a step of controlling the tool based on the tool control information. It is an information processing method characterized by including and. [Selection diagram] Fig. 6

Description

The present invention relates to an information processing method, an information processing system, and a program.

Conventionally, there has been a method of generating arm trajectory data of a working robot based on CAD data of an object (see, for example, Patent Document 1).

JP-A-2002-46087

However, when CAD drawing data is used as CAD data, the CA of the object is used.
There may be cases where the shape of the D data and the shape of the actual object do not match. Instead of this, when creating CAD data from scan data of an actual object (for example, 3D point cloud data), it is necessary to create CAD data from the scan data, although the error between the two is reduced. However, it was not useful for the user because the number of steps would increase accordingly.

Further, even if an attempt is made to directly generate arm trajectory data from the 3D point cloud data, it is difficult to determine the contour shape of the object from the point cloud data because the 3D point cloud data is merely a collection of point data. Therefore, it may be necessary for the user to manually teach the arm trajectory.

The present invention has been made in view of such a background, and is a technique capable of easily and quickly detecting the contour shape of an object from three-dimensional point cloud data and using the same to control a robot arm and a tool. The purpose is to provide.

The main invention of the present invention for solving the above-mentioned problems is to specify a step of acquiring three-dimensional point cloud data from an object by a sensor and contour point cloud data constituting the outline of the object from the three-dimensional point cloud data. Steps to be performed, a step to acquire tool control information including tool position information and tool attitude information for specifying the tool trajectory of the tool connected to the arm of the working robot from the contour point cloud data, and the tool control. It is an information processing method including a step of controlling the tool based on information.

Other problems disclosed in the present application and solutions thereof will be clarified in the columns and drawings of the embodiments of the invention.

According to the present invention, the contour shape of an object can be easily and quickly detected from the three-dimensional point cloud data, and the robot arm and the tool can be controlled using the contour shape.

It is a figure which shows the whole configuration example of one information processing system 100 of this embodiment. It is a figure which shows the hardware configuration example of the terminal 1 which concerns on this embodiment. It is a figure which shows the functional structure example of the terminal 1 which concerns on this embodiment. The figure which shows the display example of the 3D point cloud data which concerns on this embodiment is shown. It is a figure which shows the display example of the contour point cloud data which concerns on this embodiment. It is a figure which shows the flowchart example of the information processing method which concerns on this embodiment. It is a figure which shows the flowchart example of the boundary processing method which concerns on this embodiment. It is a figure explaining an example of the boundary processing method which concerns on this Embodiment. It is a figure explaining an example of the tool control information calculation method which concerns on this Embodiment. It is a figure explaining another example of the tool control information calculation method which concerns on this embodiment. It is a figure explaining another example of the tool control information calculation method which concerns on this embodiment. It is a figure explaining an example of the tool control information calculation method which concerns on this Embodiment. It is a schematic diagram explaining an example of the fitting method which concerns on this Embodiment. It is a detailed figure explaining an example of the fitting method which concerns on this Embodiment. It is a detailed figure explaining an example of the fitting method which concerns on this Embodiment.

The contents of the embodiments of the present invention will be described in a list. The present invention includes, for example, the following configuration.

[Item 1]
Information processing method
Steps to acquire 3D point cloud data from an object with a sensor,
A step of designating the contour point cloud data that constitutes the contour of the object from the three-dimensional point cloud data, and
From the contour point cloud data, a step of acquiring tool control information including tool position information and tool posture information for specifying the tool trajectory of the tool connected to the arm of the working robot, and
Steps to control the tool based on the tool control information,
An information processing method characterized by including.
[Item 2]
The information processing method according to item 1.
The step of acquiring the tool control information is
Steps to set the tool movement range and
Includes a step of designating contour point cloud data within the tool movement range as a tool position at regular intervals and acquiring tool position information.
An information processing method characterized by the fact that.
[Item 3]
The information processing method according to item 1.
The step of acquiring the tool control information is
Steps to set the tool movement range and
The contour point cloud data within the tool movement range includes a step of sparsely designating a straight line portion and densely designating a curved portion as a tool position to acquire tool position information.
An information processing method characterized by the fact that.
[Item 4]
The information processing method according to item 2 or 3.
The step of acquiring the tool control information is
The step of calculating the surface normal vector of the point data specified as the tool position, and
Further including a step of acquiring the direction of the surface normal vector as tool posture information in the point data.
An information processing method characterized by the fact that.
[Item 5]
The information processing method according to items 1 to 4.
Further including a step in which the user edits the tool position information or the tool posture information.
An information processing method characterized by the fact that.
[Item 6]
The information processing method according to items 1 to 5.
A step of storing feature point data in the contour point cloud data, and
The object is a first object, and an object different from the first object is a second object.
A step of acquiring three-dimensional point cloud data from the second object, designating contour point cloud data, and storing feature point data.
The amount of deviation is calculated by comparing the feature point data of the first object with the feature point data of the second object, and the positional deviation of the coordinate data relating to the first object is corrected based on the amount of deviation. To do,
An information processing method characterized by the fact that.
[Item 7]
The information processing method according to item 6.
As the correction of the position deviation, the position deviation of the tool position information is corrected.
An information processing method characterized by the fact that.
[Item 8]
The information processing method according to item 6 or 7.
The step of storing the feature point data is
A step of designating a predetermined point cloud of the contour point cloud data of the first object as point data of the center point of the comparison target setting range that defines a predetermined range including at least the center point.
A step of storing the contour point cloud data of the first object belonging to the predetermined range as the feature point data of the first object, and
An information processing method including a step of storing point cloud data of the second object belonging to the predetermined range as feature point data of the second object.
[Item 9]
It is an information processing system
A 3D point cloud data acquisition unit that acquires 3D point cloud data from an object using a sensor,
A boundary processing unit that specifies contour point cloud data that constitutes the contour of the object from the three-dimensional point cloud data, and
From the contour point cloud data, tool control information including tool position information and tool posture information for specifying the tool trajectory of the tool connected to the arm of the working robot is acquired, and the tool is based on the tool control information. Tool control unit that controls
An information processing system characterized by including.
[Item 10]
A program that allows a computer to execute an information processing method.
The program is used as the information processing method.
Steps to acquire 3D point cloud data from an object with a sensor,
A step of designating the contour point cloud data that constitutes the contour of the object from the three-dimensional point cloud data, and
From the contour point cloud data, a step of acquiring tool control information including tool position information and tool posture information for specifying the tool trajectory of the tool connected to the arm of the working robot, and
Steps to control the tool based on the tool control information,
A program characterized by having a computer execute.
[Item 11]
Information processing method
Steps to acquire 3D point cloud data from an object with a sensor,
A step of designating the contour point cloud data that constitutes the contour of the object from the three-dimensional point cloud data, and
An information processing method characterized by including.

<Details of the embodiment>
A specific example of the information processing system 100 according to the embodiment of the present invention will be described below with reference to the drawings. It should be noted that the present invention is not limited to these examples, and is indicated by the scope of claims, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims. In the following description, the same or similar elements are given the same or similar reference numerals and names in the accompanying drawings, and duplicate description of the same or similar elements may be omitted in the description of each embodiment. In addition, the features shown in each embodiment can be applied to other embodiments as long as they do not contradict each other.

FIG. 1 is a diagram showing an example of the information processing system 100 of the present embodiment. As shown in FIG. 1, the information processing system 100 of the present embodiment has a terminal 1 and a working robot 2. The working robot 2 has at least an arm 21, a tool 22, and a sensor 23. The terminal 1 and the working robot 2 are connected to each other so as to be able to communicate with each other by wire or wirelessly.

<Terminal 1>
FIG. 2 is a diagram showing a hardware configuration of the terminal 1. The terminal 1 may be a general-purpose computer such as a personal computer, or may be logically realized by cloud computing. The illustrated configuration is an example, and may have other configurations.

The terminal 1 has at least a processor 10, a memory 11, a storage 12, and a transmitter / receiver 13.
, Input / output units 14, etc., which are electrically connected to each other through the bus 15.

The processor 10 is an arithmetic unit that controls the operation of the entire terminal 1, at least controls the transmission and reception of data and the like with the working robot 2, and performs information processing and the like necessary for executing an application and performing authentication processing. For example, the processor 10 is a CPU (Central Process Unit).
ng Unit) and / or GPU (Graphics Processing U)
Nit), and each information processing is performed by executing a program or the like for this system stored in the storage 12 and expanded in the memory 11.

The memory 11 is a DRAM (Dynamic Random Access Memory).
Main memory composed of volatile storage devices such as y), flash memory and HDD (Hard)
Includes auxiliary storage configured with a non-volatile storage device such as Disk Drive). The memory 11 is used as a work area or the like of the processor 10, and also stores a BIOS (Basic Input / Output System) executed when the terminal 1 is started, various setting information, and the like.

The storage 12 stores various programs such as application programs. A database storing data used for each process may be built in the storage 12.

The transmission / reception unit 13 connects the terminal 1 to at least the work robot 2 and transmits / receives data and the like according to the instructions of the processor. The transmission / reception unit 13 is configured by wire or wireless, and in the case of wireless, for example, WiFi, Bluetooth (registered trademark) and B.
It may be configured by a short-range communication interface of LE (Bluetooth Low Energy).

The input / output unit 14 is an information input device such as a keyboard and a mouse, and an output device such as a display.

The bus 15 is commonly connected to each of the above elements and transmits, for example, an address signal, a data signal, and various control signals.

<Working robot 2>
Returning to FIG. 1, the working robot 2 according to the present embodiment will be described.

As described above, the working robot 2 has an arm 21, a tool 22, and a sensor 23. The illustrated configuration is an example, and may have other configurations.

The operation of the arm 21 is controlled by the terminal 1 based on the three-dimensional robot coordinate system. Further, the arm 21 may further include a controller (not shown) connected to the working robot 2 by wire or wirelessly, thereby controlling its operation.

The operation of the tool 22 is controlled by the terminal 1 based on the three-dimensional tool coordinate system. Further, the configuration of the tool 22 may include any tool according to the application, and may be, for example, a welding torch, a paint injection device for painting, a gripping device, an excavating device, a polishing device, or the like.

The sensor 23 senses an object based on a three-dimensional sensor coordinate system. Sensor 2
Reference numeral 3 denotes a laser sensor that operates as, for example, a three-dimensional scanner, and acquires three-dimensional point cloud data of an object by sensing. The three-dimensional point cloud data is as shown in FIG. 4, for example, and each point data has sensor coordinates, and the shape of the object can be grasped by the point cloud.

More specifically, the arm is subjected to a predetermined calibration before the work, the robot coordinate system, the tool coordinate system, and the sensor coordinate system are associated with each other, and the user specifies a position based on the sensor coordinate system, for example. The operation of the 21 and the tool 22 may be controlled based on the corresponding positions.

<Function of terminal 1>
FIG. 3 is a block diagram illustrating the functions implemented in the terminal 1. In the present embodiment, the processor 10 of the terminal 1 has a three-dimensional point cloud data acquisition unit 101, a three-dimensional point cloud data display unit 102, a boundary processing unit 103, and a tool control unit 104. Further, the storage 12 of the terminal 1 has a three-dimensional point cloud data storage unit 121, a contour point cloud data storage unit 122, and a tool control information storage unit 123.

The three-dimensional point cloud data acquisition unit 101 controls the working robot 2 according to an instruction from the input / output unit 14 of the terminal 1, and acquires the three-dimensional point cloud data of the object by the sensor 23. The acquired three-dimensional point cloud data is, for example, three-dimensional coordinate information data based on the sensor coordinate system, and is stored in the three-dimensional point cloud data storage unit 121.

The three-dimensional point cloud data display unit 102 displays the three-dimensional point cloud data acquired by the three-dimensional point cloud data acquisition unit 101 on the input / output unit 14 of the terminal 1 as illustrated in FIG. 4, for example. The user can visually recognize the displayed three-dimensional point cloud data from any direction, and can specify the arbitrary direction with an information input device such as a keyboard or mouse connected to the input / output unit 14, for example. is there.

The boundary processing unit 103 leaves only the contour point cloud data 51 that constitutes the contour of the three-dimensional point cloud data as illustrated in FIG. 5 based on the three-dimensional point cloud data 41 as illustrated in FIG. Process as follows. Like the three-dimensional point cloud data, the contour point cloud data is, for example, three-dimensional coordinate information data based on the sensor coordinate system, and is stored in the contour point cloud data storage unit 122. Further, the contour point cloud data is the terminal 1 by the contour point cloud data display unit (not shown) like the three-dimensional point cloud data.
It is displayed in the input / output unit 14 of. An example of a specific processing method of the boundary processing unit 103 will be described later.

The tool control unit 104 calculates tool control information including one or more position information and posture information of the tool 22 with respect to the object based on the tool movement range, the tool movement interval, and the contour point cloud data, and the calculated tool The tool 22 is moved based on the control information. That is, in the tool control unit 104 of the processor 10 or other configurations (not shown), arm trajectory data for operating the arm 21 according to the tool trajectory data is generated based on the tool control information corresponding to the so-called tool trajectory data. An example of a specific calculation method of the tool control unit 104 will be described later.

<Flowchart of information processing method>
FIG. 6 is an example of a flowchart of an information processing method in the information processing system 100 of the present embodiment.

First, the user operates the work robot 2 by the terminal 1 or the controller (not shown), and based on the control by the three-dimensional point cloud data acquisition unit 101 of the terminal 1, the object located on the work table, for example, by the sensor 23. Acquire three-dimensional point cloud data of an object (SQ101).

Next, the three-dimensional point cloud data acquired by the three-dimensional point cloud data display unit 102 is displayed on the terminal 1 (SQ102).

Next, the boundary processing unit 103 processes the contour point cloud data so as to generate it from the three-dimensional point cloud data, and displays it on the terminal 1 (SQ103).

Next, the tool control unit 104 calculates tool control information including one or more position information and posture information of the tool 22 with respect to the object based on the tool movement range, the tool movement interval, and the contour point cloud data, and calculates the calculation. Move the tool 22 based on the tool control information (SQ104).
..

Therefore, the information processing system 100 of the present embodiment can easily and quickly detect the contour shape of the object from the three-dimensional point cloud data, and control the robot arm using the contour shape.

<Flowchart of boundary processing method>
FIG. 7 shows the boundary processing method SQ103 in the information processing system 100 of the present embodiment.
It is an example of a more detailed flowchart of.

First, the boundary processing unit 103 selects one point data from the three-dimensional point cloud data (SQ201).

Next, the boundary processing unit 103 draws a line segment from the selected point data for each of the points included in the circle having a predetermined radius centered on the selected point data (SQ202).

Next, among the plurality of the line segments, for example, the angle formed by two adjacent line segments in a predetermined rotation direction centering on the selected point data is calculated (SQ203).

Next, it is determined whether or not any of the calculated angles is smaller than the set value set by the user (SQ204).

More specifically, in FIG. 8, for example, when the user sets the set value to 100 °, in the circle 71, the angle formed by the two adjacent line segments is smaller than the set value like the angle A. It is determined that the value (A <100 °) is (Yes), and in the circle 72, it is determined that there is an angle having a value (B> 100 °) larger than the set value, such as the angle B (No).
).

Here, when it is determined as No, the selected point data is stored in the contour point cloud data storage unit 122 as the contour point cloud data (SQ205).

Then, if it is determined to be Yes, or after the end of SQ205, it is determined whether or not all the point data has been selected, if it is determined to be No, the process returns to SQ201, and if it is determined to be Yes, the process ends. (SQ206).

Therefore, the information processing system 100 of the present embodiment can easily and quickly detect the contour point cloud data constituting the contour shape of the object from the three-dimensional point cloud data by, for example, the boundary processing method described above. is there.

In addition to the flowchart of FIG. 7, for example, after SQ206, the input / output unit 14 of the terminal 1
Only the contour point cloud data may be displayed in. Further, for example, at the timing before or after SQ206, the point cloud data that was not regarded as the contour point cloud data is deleted from the three-dimensional point cloud data, and the three-dimensional point cloud data display unit 102 corresponds to the contour point cloud data. The three-dimensional point cloud data to be used may be displayed on the input / output unit 14 of the terminal 1. By doing so, for example, as shown in FIG. 5, only the outline of the three-dimensional point cloud data can be displayed.

<Calculation method of tool control information>
FIG. 9 shows a calculation method SQ of tool control information in the information processing system 100 of the present embodiment.
This is an example of 104.

First, regarding the tool position information, for example, as shown in FIG. 9, the tool movement range (for example, defined by the tool movement start position and the tool movement end position) is set, and the tool control unit 104 sets the contour within the tool movement range. By designating the point cloud data as the tool position at regular intervals, the tool position information at equal intervals along the contour can be obtained. Information about a part of the contour point cloud data designated as the tool position is stored in the tool control information storage unit 123 as the tool position information. As a result, the tool position information can be specified by a simple process. Although the outline (dotted line) of the virtual object is displayed for easy understanding, it does not have to be displayed when it is displayed on the actual terminal 1. Further, although the tool position is specified at the tool movement start position and the tool movement end position, the contour point group data in the vicinity may be specified.

Further, for example, as shown in FIG. 10, the tool movement range is set and the tool control unit 10 is set.
According to No. 4, the contour point cloud data within the tool movement range may be roughly specified as the tool position in the straight line portion and the curved portion of the object contour. This is because the tool 22 moves linearly between the tool positions, so that the tool position spacing may be wide in the straight portion, but in the curved portion, the tool position spacing is narrow in order to draw a tool trajectory similar to the curve. Is more preferable.

Regarding the above-mentioned rough designation, for example, it may be determined whether or not the angle formed by the two virtual center lines connecting the reference point data and the two point data in the contour point cloud data is equal to or larger than the reference value. For example, as shown in FIG. 11, when the point data n of the contour point group data is used as the reference point data, a line connecting the center of the point data n and the center of the point data n + 1 adjacent to the point data n is drawn. The virtual center line A is defined as the virtual center line A, the line connecting the center of the point data n and the center of the predetermined point data m is defined as the virtual center line B, and the line connecting the center of the point data n and the center of the predetermined point data m + 1 is defined as the virtual center line B. Let it be the virtual center line C. Then, for example, when the angle formed by the virtual center line A and the virtual center line B is less than the reference value, the point data m is not stored in the tool control information storage unit 123 as the tool position.
On the other hand, for example, when the angle formed by the virtual center line A and the virtual center line C is equal to or larger than the reference value, the point data m + 1 is stored in the tool control information storage unit 123 as the tool position. Since the position of the point data n + 1 is usually close to the position of the point data n, it is not necessary to adopt the point data n + 1 as the tool position.

Then, next, the point data m + 1 is used as the reference point data, and the point data is sequentially determined in the same manner, and this is repeated to specify the tool position. As a result, the tool control unit 104 can easily draw a tool trajectory closer to the contour shape of the object.

The point data stored as the tool position may be displayed on the terminal 1 so that the user can confirm it. At that time, for example, the point data stored as the tool position and the other point data are displayed in a distinctive manner by color-coding, and the user deletes the point data stored as the tool position, for example. Or, the tool position may be specified from other point data, and the editing may be performed arbitrarily. This enables more accurate arm control.

Next, regarding the tool posture information, for example, as shown in FIG. 12, in the point data C stored in the tool control information storage unit 123 as the tool position, the peripheral contour point cloud data (
For example, the surface normal vector is calculated from the contour point cloud data adjacent to the point data C), and the direction along the surface normal vector is stored in the tool control information storage unit 123 as the tool posture information. This makes it possible to align the tool 22 perpendicular to the surface of the object at the tool position.

The tool posture may also be recognizable and displayed on the terminal 1. At that time, the tool posture may be arbitrarily changed by the user. This enables more accurate arm control.

<Example of fitting>
In the information processing system 100 of the present embodiment, a tool including tool trajectory information (tool position information and tool posture information) for a predetermined object (hereinafter, referred to as "first object") stored by the tool control unit 104. When performing work associated with the type of tool 22 on a plurality of objects having the same or similar shape (hereinafter referred to as "second object") by using (control information), the position shifts. In consideration of the above, sensing may be performed once by the sensor 23 before the operation of the tool.

At this time, in the second object, for example, the contour point cloud data of the first object when the tool control information is acquired is compared with the point cloud data of the second object acquired before the work. Then, an operation (fitting) for correcting the misalignment may be executed. The point cloud data of the second object may be three-dimensional point cloud data, point cloud data obtained by thinning the three-dimensional point cloud data by a certain ratio, or after boundary processing. It may be contour point cloud data.

For example, each point cloud data as shown in FIG. 13 is obtained from the first object and the second object. More specifically, the comparison method for correcting the positional deviation between the two may be, for example, comparing the feature point data of each object.

The designation method for storing the feature point data in each object is as follows as an example. First, the processor 10 uses, for example, the predetermined point data of the contour point cloud data (white point data in FIG. 14) of the first object in the comparison target setting range that defines a predetermined range including at least the center point. It is specified as point data of the center point. In FIG. 14, two ranges composed of spheres having a radius Rs from the point data of the center point are shown as comparison target setting ranges S, but what kind of shape is the shape of the comparison target setting range S? However, the number of comparison target setting ranges S may be arbitrary (for example, points, cubes, rectangular parallelepipeds, prisms, cylinders, ellipsoids, etc.).

Then, as illustrated in FIG. 15, for example, the processor 10 and the storage 12 use the contour point cloud data of the first object belonging to the range of the comparison target setting range S as the feature point data of the first object. It is designated and stored, and the point cloud data of the second object belonging to the range of the comparison target setting range S is designated and stored as the feature point data of the second object.

The method of comparing the feature point data of each object is as follows as an example. As illustrated in FIG. 15, for example, the processor 10 has a th-order included in the circle from the center point of the circle consisting of the search range radius R of a predetermined range centered on each of the feature point data of the first object. The distance L to each feature point data of the two objects is calculated, and for example, the average value of the shortest distance L from each of the feature point data of the first object, or the average value of all the distances L is calculated. calculate. Then, for example, the processor 10 moves the point cloud data of the second object and calculates the average value again. This is executed a plurality of times, and for example, the processor 10 confirms or calculates the amount of movement of the point cloud data of the second object when the average value is the smallest. Finally, for example, storage 1
2 stores the movement amount as the displacement amount of the positional deviation. The number of times to be executed a plurality of times may be set to a predetermined number of times by the user in advance, or may be set to end when the average value becomes equal to or less than the predetermined value set by the user in advance. You may.

Finally, based on the amount of deviation, for example, the processor 10 corrects the tool trajectory information for the first object to the position of the second object by using the amount of deviation to match the second object. It is possible to use a tool trajectory.

<Other examples of fitting>
In addition to the above-mentioned fitting method, for example, among the contour point cloud data shown in FIG. 5, the point data of the feature portion (for example, the corner portion) constituting the object is used as the feature point data in the contour point cloud data storage unit 122. Further, it may be stored and the deviation in the sensor coordinate system may be determined by comparing the feature point data of both. Based on this, the positional deviation of the tool control information is corrected. As a result, as in the above specific example, it is possible to draw a tool trajectory that matches the position of the actual object, but it is not necessary to execute the tool control information acquisition process (SQ104) again, so that it is easy. It is possible to correct the misalignment.

After the deviation correction by comparing the feature point data, at least the contour point cloud data of the first object stored as the tool position and the point cloud data of the second object acquired before the corresponding work. If there is point data in which the position shift occurs by a predetermined distance or more even after the correction, the tool control information calculation SQ104 may be executed again. Alternatively, the tool control information calculation SQ104 may be executed again only within the range of the tool control information corresponding to the point data in which the position shift occurs by a predetermined distance or more. This makes it possible to draw a tool trajectory along the actual shape, especially when working on an object having a similar shape, but it is not necessary to execute the tool control information calculation SQ104 every time. , Processing load and work time are further reduced.

Although the present embodiment has been described above, the above embodiment is for facilitating the understanding of the present invention, and is not for limiting and interpreting the present invention. The present invention can be modified and improved without departing from the spirit thereof, and the present invention also includes an equivalent thereof.

1 Terminal 2 Working robot 21 Arm 22 Tool 23 Sensor

Claims (5)

Information processing method
The step of acquiring the first 3D point cloud data from the first object by the sensor,
First contour point cloud data that constitutes the contour of the entire first object from the first three-dimensional point cloud data based on the mutual positional relationship of the point data in the first three-dimensional point cloud data. Steps to specify and
A step of designating a predetermined point data of the first contour point cloud data as point data of the reference point in a comparison target setting range that defines a predetermined range including at least a reference point.
A step of storing the first contour point cloud data belonging to the predetermined range as feature point data of the first object, and a step of storing the data.
The step of acquiring the second 3D point cloud data from the second object by the sensor,
A step of storing the second three-dimensional point cloud data belonging to the predetermined range as feature point data of the second object, and
A step of comparing the feature point data of the first object with the feature point data of the second object to calculate the deviation amount, and
A step of acquiring tool control information including tool position information and tool posture information for specifying a tool trajectory of a tool connected to an arm of a working robot based on at least the first contour point cloud data.
Steps to control the tool based on the tool control information,
Including
The reference point is the center point of the comparison target setting range, and is
The comparison target setting range, Ri range der composed point data of the central point at a given radius of the sphere,
The step of acquiring the tool control information includes a step of setting a tool movement range.
An information processing method characterized by the fact that. The information processing method according to claim 1.
Obtaining a pre-SL tool control information, the first contour point group data before Symbol Tool movement range, the linear part is sparse, acquires tool position information curved portion is designated as densely tool position Steps to do , including
An information processing method characterized by the fact that. The information processing method according to claim 1 or 2.
The step of designating the first contour point cloud data is
A step of selecting one point data from the first three-dimensional point cloud data, and
A step of drawing a line segment from the selected point data for each of the points contained inside a circle having a predetermined radius centered on the selected point data.
A step of calculating the angle formed by two adjacent line segments in a predetermined rotation direction centering on the selected point data among a plurality of the line segments.
Steps to compare the calculated angle with the set value,
When it is determined that the calculated angle is larger than the set value, the step of determining the selected point data as the first contour point cloud data is included.
An information processing method characterized by the fact that. It is an information processing system
The first three-dimensional point cloud data is acquired from the first object by a sensor, and
A 3D point cloud data acquisition unit that acquires 2D point cloud data from a second object using a sensor, and a 3D point cloud data acquisition unit.
First contour point cloud data that constitutes the contour of the entire first object from the first three-dimensional point cloud data based on the mutual positional relationship of the point data in the first three-dimensional point cloud data. Boundary processing unit that specifies
A predetermined point data among the first contour point cloud data is designated as the point data of the reference point in the comparison target setting range that defines at least a predetermined range including the reference point.
The first contour point cloud data belonging to the predetermined range is stored as the feature point data of the first object, and the data is stored.
The second three-dimensional point cloud data belonging to the predetermined range is stored as the feature point data of the second object, and the data is stored.
A correction control unit that calculates the amount of deviation by comparing the feature point data of the first object with the feature point data of the second object , and
Tool control information acquisition unit that acquires tool control information including tool position information and tool posture information for specifying the tool trajectory of the tool connected to the arm of the working robot based on at least the first contour point cloud data. When,
A tool control unit that controls the tool based on the tool control information is provided.
The reference point is the center point of the comparison target setting range, and is
The comparison target setting range, Ri range der composed point data of the central point at a given radius of the sphere,
The tool control information acquisition unit sets the tool movement range.
An information processing system characterized by this. A program that allows a computer to execute an information processing method.
The program is used as the information processing method.
The step of acquiring the first 3D point cloud data from the first object by the sensor,
First contour point cloud data that constitutes the contour of the entire first object from the first three-dimensional point cloud data based on the mutual positional relationship of the point data in the first three-dimensional point cloud data. Steps to specify and
A step of designating a predetermined point data of the first contour point cloud data as point data of the reference point in a comparison target setting range that defines a predetermined range including at least a reference point.
A step of storing the first contour point cloud data belonging to the predetermined range as feature point data of the first object, and a step of storing the data.
The step of acquiring the second 3D point cloud data from the second object by the sensor,
A step of storing the second three-dimensional point cloud data belonging to the predetermined range as feature point data of the second object, and
A step of comparing the feature point data of the first object with the feature point data of the second object to calculate the deviation amount, and
A step of acquiring tool control information including tool position information and tool posture information for specifying a tool trajectory of a tool connected to an arm of a working robot based on at least the first contour point cloud data.
Steps to control the tool based on the tool control information,
Is a program that causes a computer to execute
The reference point is the center point of the comparison target setting range, and is
The comparison target setting range, Ri range der composed point data of the central point at a given radius of the sphere,
The step of acquiring the tool control information includes a step of setting a tool movement range.
A program characterized by that.

Images