CN112017186A

CN112017186A - Material increase and residual height prediction method based on molten pool image and depth residual error network

Info

Publication number: CN112017186A
Application number: CN202011200767.0A
Authority: CN
Inventors: 陆骏; 赵壮; 韩静; 张毅
Original assignee: Nanjing Zhipu Photoelectric Technology Co ltd
Current assignee: Nanjing Zhipu Photoelectric Technology Co ltd
Priority date: 2020-11-02
Filing date: 2020-11-02
Publication date: 2020-12-01

Abstract

The invention relates to a material increase and residual height prediction method based on a molten pool image and a depth residual error network, belongs to the technical field of image processing, and aims to realize regulation and control of welding quality. The method accurately predicts the additive surplus height based on the molten pool image and the depth residual error network, the change of the cladding layer surplus height reflects the change of the melting depth to a certain extent during additive manufacturing, quality control of the additive manufacturing process is realized by monitoring the surplus height, the future surplus height development trend of the welding line is accurately predicted, and the welding quality is conveniently regulated and controlled in real time.

Description

Material increase and residual height prediction method based on molten pool image and depth residual error network

Technical Field

The invention relates to a material increase residual height prediction method based on a molten pool image and a depth residual error network, and belongs to the technical field of image processing.

Background

The general welding process is mainly characterized by welding penetration and good penetration, but due to the influence of a remelted region of a cladding layer, the penetration in the material adding process is difficult to monitor. And the change of the surplus height of the cladding layer reflects the change of the melting depth to a certain extent during the material increase, so that the quality control of the material increase manufacturing process is realized by monitoring the surplus height. In fact, the accumulation of parts is made up of a series of single-layer single-pass accumulations, so the forming size and quality of these single-layer single-pass directly determine the forming quality of the accumulated parts. The welding process is a local rapid heating and cooling process, and the shape of a molten pool, the distribution of a temperature field and the intensity of a substance spectrum are also changed continuously in the process. Therefore, it is important to monitor the changes of these physical quantities to reflect the current welding conditions.

With the development of computer technology and the rise of big data, deep learning has been widely applied to various industrial fields including welding fields. A depth residual error-based network monitors the increment of the residual height of each cladding layer in the electric arc additive manufacturing process by using visual information of a molten pool. The method comprises the steps of taking a molten pool visual information map as an input value, carrying out convolution neural network operation on the molten pool visual information map to estimate the residual height of a cladding layer, and meanwhile calculating a normalized relative Error through a normalized Mean square Error (Mean Squared Error, abbreviated as MSE) and a loss function so as to be capable of training all samples equally.

Disclosure of Invention

In order to solve the technical problems, the invention provides a material increase residual height prediction method based on a molten pool image and a depth residual error network, which has the following specific technical scheme:

the additive residual height prediction method based on the molten pool image and the depth residual error network comprises the following steps:

the method comprises the following steps: building monitoring equipment: the method comprises the following steps of building a welding device and a visual sensing device, wherein the welding device and the visual sensing device are both installed on a welding robot, and the welding device comprises a welding power supply, a wire feeder and a cooling system; the vision sensing device is provided with a color CDD camera;

step two: image processing: the deep characteristics of the molten pool image are learned through a network, and the relation between the molten pool image and the residual height of each layer is established;

step three: simplifying the difficulty of network learning: the network learning adopts a residual error network structure, and the residual error network structure directly bypasses the input information to output, so that the network does not directly fit the original mapping, thereby fitting the residual error mapping;

step four: integrating local information: acquiring a characteristic diagram output by using a residual error network structure, performing pooling, and performing first full connection to integrate local information with category distinctiveness in a previous pooling layer;

step five: the network performance is improved: the excitation function of each neuron of the full connection layer adopts a ReLU function, and the full connection output value of the last layer is transmitted to be the residual height;

step six: and (3) prediction calculation: and predicting a specific numerical value by using the mean square error, wherein the calculation formula is as follows:

（1）

in the formula

Is a predicted value of the number of the frames,

is the true value and n is the number of samples.

Furthermore, the residual error network structure adopts a residual error block as a basic composition unit, and the residual error network structure extracts image characteristics through the basic structure of the residual error block.

Further, the basic network structure of the characteristic extraction part of the molten pool image is Resnet-34.

Further, the puddle image size was chosen to be 640 x 350 and scaled to 200 x 100.

The invention has the beneficial effects that:

the method accurately predicts the additive surplus height based on the molten pool image and the depth residual error network, the change of the cladding layer surplus height reflects the change of the melting depth to a certain extent during additive manufacturing, quality control of the additive manufacturing process is realized by monitoring the surplus height, the future surplus height development trend of the welding line is accurately predicted, and the welding quality is conveniently regulated and controlled in real time.

Drawings

FIG. 1 is a diagram of a molten pool image monitoring device of the present invention,

FIG. 2 is a flow chart of molten pool image monitoring of the present invention,

FIG. 3 is a characteristic map of a weld puddle image of the present invention,

figure 4 is a schematic electrical signal diagram of a weld of the present invention,

FIG. 5 is a schematic diagram of the image pile-up forming profile of the weld puddle of the present invention,

FIG. 6 is a diagram showing the multi-physical-quantity characterization of the second cladding layer according to the present invention,

FIG. 7 is a physical representation of the cladding layers of the present invention,

FIG. 8 is a graph showing the regression result and error of the second layer of the cladding layer according to the present invention,

FIG. 9 is a schematic diagram of the regression result and error of the fifth layer of the cladding layer according to the present invention,

FIG. 10 is a graph showing the regression result and error of the eighth layer of the cladding layer according to the present invention,

FIG. 11 is a diagram illustrating the regression result and error of the tenth layer of the cladding layer according to the present invention,

FIG. 12 is a graph showing the regression results of the residual heights of the layers at 1min in the cold zone of the present invention,

FIG. 13 is a graph showing the regression results of the residual heights of the layers in the cold zone of the present invention at a time of 2min,

FIG. 14 is a graph showing the regression results of the residual heights of the layers at 3min in the cold zone of the present invention,

FIG. 15 is a graph showing the regression results of the residual heights of the layers at 4min in the cold zone of the present invention,

in the figure: 1-mother board, 2-thin-wall part, 3-color camera, 4-welding gun, 5-laser, 6-black and white camera, and 7-workbench.

Detailed Description

The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic views illustrating only the basic structure of the present invention in a schematic manner, and thus show only the constitution related to the present invention.

（1）

in the formula

Is a predicted value of the number of the frames,

is the true value and n is the number of samples.

As shown in fig. 1 and 2, the apparatus for monitoring the residual amount of the cladding layer in the arc additive manufacturing process of the present invention includes two parts: a welding device and a vision sensor device. The welding device includes a welding power source, a wire feeder, and a cooling system. The vision sensor device comprises a colour camera 3 fixed to the robot. A mother plate 1 is placed on the welding workbench 7 in advance, the laminated thin-wall parts 2 are placed on the surface of the mother plate 1, and in order to enable the collected molten pool image to correspond to the actual position of a welding seam, a laser 5 is used for auxiliary positioning. A laser 5 with a center wavelength of 450nm is used to irradiate the upper edge part of the welding wire, and a black and white camera 6 is arranged to capture a laser point. In the data acquisition process, the FPGA system sends two paths of synchronous signals to control the color camera 3 and the black-and-white camera 6 to acquire images at the same time. Meanwhile, due to the influence of the remelting region, in order to accurately measure the extra height increment of each layer, after welding of each layer is finished, the height of the current layer is scanned by a Wiiboox three-dimensional scanner, the change relation between the position of the welding seam and the height is obtained, and the height difference of the same welding seam positions of two adjacent layers is the extra height increment of the current welding seam position. Thereby unifying the weld pool image, the weld seam position and the corresponding residual height increment. In the image processing process, deep features of the molten pool image are learned through a network, and the relation between the molten pool image and the residual height of each layer is established. The traditional neural network or the full-connection network has the problems of information loss and loss during information transmission, and simultaneously, the problem of gradient disappearance or gradient explosion can also occur along with the increase of the network depth. The invention adopts a residual error network structure, adopts a residual error block as a basic composition unit, wherein the input isxFor a stacked layer structure, the learned characteristics are set as

And the residual structure is expected to learn the characteristics of

Then, then

It shows that the original learning features are

It can be seen from the above that learning original features directly is more complicated than learning residuals. The residual error structure directly bypasses the input information to output, so that the network does not directly fit the original mapping, but fits the residual error mapping, and the learning difficulty is simplified. When residual error

In the process, the accumulation layer can also achieve the effect of the identity mapping, so that the problem of network degradation caused by the increase of the depth of the convolutional neural network is solved to a great extent. In the general case of the above-mentioned,

therefore, the network will always learn new features based on the input features, thereby having better network performance. The invention provides a method for monitoring the residual height of a cladding layer based on a depth residual error network, which extracts image characteristics through the basic structure of a residual error module. The size of the molten pool after shearing alignment is 640 multiplied by 350, and the original image is scaled to 200 multiplied by 100 by comprehensively considering the training speed and the precision of the network. The following table shows specific information of the network part,

the basic block comprises two convolution layers, the convolution kernel size is 3 x 3, Resnet-34 is used as a basic network of a characteristic extraction part of the molten pool image, and after output characteristic diagrams are pooled, the first full connection is carried out to integrate local information with category distinctiveness in the previous pooling layer. In order to improve the network performance, a ReLU function is adopted for the excitation function of each neuron of the full-connection layer, and the output value of the full-connection of the last layer is transmitted, namely the residual height. FIG. 3 demonstrates the process of feature map shrinkage due to convolution and average pooling.

Example one

The embodiment is based on a cold metal transition method, namely, CMT (CMT) based stainless steel single-channel multi-layer additive manufacturing as a background, and the change condition of the residual quantity of the cladding layer in the arc additive manufacturing process is monitored. Wherein the process is single filament CMT, the welding current is 130A, the welding speed is 5mm/s, and the shielding gas is argon-oxygen mixed gas (98.5% Ar)₂+1.5%O₂) The gas flow is 25L/min, the welding length is 80mm, the number of welding layers is 10, the lifting height of the welding gun 4 is 1mm, the welding wire number is ER316L, the base material is 304 stainless steel, the acquisition frequency of a camera is 1000Hz, and the exposure time is 100 us. The CMT is used as a special MIG/MAG welding, when the molten drop short circuit is transited, the welding machine obtains a short circuit signal, a welding power supply is cut off, a welding wire is drawn back, the molten drop is helped to fall off, and the cold transition of the molten drop is realized. The heating mode of the heat source with the heat-cold-heat alternation greatly reduces the heat input and can realize the additive manufacturing and forming of the ultrathin part. As shown in fig. 4, the CMT welding current is a function of time. Generally, a period of hot-cold alternation is 70Hz, namely about 14ms, because the camera of the embodiment collects the frequency of 1000Hz, the number of the molten pool images collected in one period is 14, and the condition that the molten pool images collected in one period change along with time is shown. It can be seen from FIG. 4 that from 9ms onwards, the welder was in a short-circuit transient state and the acquired weld puddle image was completely unaffected by the arc light. Therefore, in order to reduce the influence of interference, the 9ms molten pool image is adopted uniformly.

(1) Single pass multi-layer weld pool quality analysis

The stacking manner of this example is the same direction stacking, and the height of each layer formed is as shown in fig. 5 (a), wherein the interlayer cooling time is 3 min. The height of the arcing end is higher and the height of the arcing end is lower than that of the middle section of the cladding layer. As shown in fig. 5 (b) and (c), the acquired molten pool images also have obvious differences, but the problems can be solved by adjusting welding parameters, so the molten pool images at the positions 20-70 mm away from the arc end of the cladding layer are analyzed in the embodiment. Due to the change of the heat dissipation condition around the molten pool and the influence of other factors, the molten pool shape is changed dynamically all the time, and the molten pool image at the position 20-70 mm away from the arc end of a certain layer of cladding layer can be discussed in four stages according to the change of the length of the molten pool. As shown in fig. 6 (a), the change in the molten pool state will be described by taking the molten pool of the second layer cladding layer as an example. The first stage is 20 mm-40.5 mm. Although the front part of the molten pool leaves the arc striking area at this stage, the rear part of the molten pool does not leave the arc striking area, and the area can be observed to be in a red hot state due to the fact that the amount of cladding in the arc striking area is large and the heat dissipation condition is poor. The second stage is 40.5mm to 47 mm. The rear part of the molten pool leaves the arc striking area at the stage, the molten pool shape begins to shrink, and the length and the width of the molten pool at the stage are minimum. The third stage is 47 mm-66 mm. In the stage, due to the influence of heat accumulation, the heat dissipation condition is poor, the molten pool is in a high-temperature state for a long time, and the flowing of the molten pool is facilitated, so that the length of the molten pool is lengthened, and the width of the molten pool is widened. The fourth stage is 66 mm-70 mm. In the stage, because the cladding layer is inclined, the molten pool flows downwards under the influence of gravity, the position of the molten pool away from the welding gun 4 is lengthened, the elongation of the rod is lengthened, and the length of the molten pool is changed. Fig. 6 (c) shows a region at a fixed distance from the welding gun 4, that is, a red square region in fig. 6 (b), and the average temperature of the weld bead changes with the welding time. It is known that the average temperature change of this area undergoes a law of plateau-fall-rise-fall, corresponding to the four phases of the previous analysis.

In fact, the difference between the molten pool morphologies of different cladding layers is greater. The cladding layer 1 is in direct contact with the base material, and therefore, the heat radiation condition is the best, so that the difference in the length and area of the molten pool from the other layers is the largest, as shown in fig. 7 (c). Therefore, if the subsequent molten pool quality analysis of the single-pass multilayer is not particularly described, the molten pool of the cladding layer after the first layer is taken as a research object. As shown in fig. 7 (a), the layers 2 to 10 all start to be tilted after undergoing a plateau, and the position where the tilting occurs is closer to the starting arc end as the number of layers increases, the dotted line portion in fig. 7 (a) being the boundary line of the four stages of the molten pool of each cladding layer, where the last boundary line is the position on the cladding layer when each layer starts to be tilted, and fig. 7 (b) being the image of the molten pool when each layer starts to be tilted. As the number of layers increases, the inclination angle of each layer increases, and the width, length, area of the molten pool and the elongation of the wire rod at the beginning of the inclination are completely different along with the influence of heat dissipation conditions and other factors. FIG. 7 (c) is an image of the weld pool at 50mm from the start of arc for each layer, and the weld pool formation from layer 2 to layer 6 is in the third stage, where the weld layer has not yet tilted and there is little change in wire rod elongation. And the molten pool form from the 7 th layer to the 10 th layer is in the fourth stage, at the moment, the molten pool flows downwards due to the gravity action caused by the inclined cladding layer, the elongation of the welding wire rod is lengthened, and the elongation of the welding wire rod is longer along with the increase of the number of layers. Therefore, the width, length, area and temperature field of the molten pool at different positions of the same cladding layer are dynamically changed, the molten pools at the same positions of different cladding layers are different in shape and rod elongation of the welding wire due to inclination and the like, and the visual shape of the molten pool is characterized by the change and the difference, so that feasibility is provided for regression of the following network on the cladding layer residual height.

(2) Error analysis of single-layer cladding layer regression results

As shown in fig. 8 to 11, the regression results of the randomly selected residual heights of the four layers of cladding layers are shown, wherein the average error of the residual height of the second layer is 0.0305mm, the average error of the residual height of the fifth layer is 0.0382mm, the average error of the residual height of the eighth layer is 0.0289mm, the average error of the residual height of the tenth layer is 0.0310mm, and the error of the regression results is relatively low. The results show that the regression effect is good. The molten pool in the middle section of the cladding layer changes due to the change of the surrounding heat dissipation conditions and other factors, and the shape of the molten pool and the distribution of the temperature field of the molten pool change accordingly, and the two factors are main factors directly influencing the forming characteristics. For high-temperature objects, the color distribution of the objects in the CCD is directly influenced by the temperature of the objects, so that the visual image gives consideration to the morphological characteristics and the temperature characteristics of the molten pool. The network can regress the height of the single-layer cladding layer according to the visual molten pool shape difference.

(3) Error analysis of different fusion layer residual height regression results

As shown in fig. 14, the regression results of the heights of the layers are obtained when the interlayer cooling time is 3min, the average error of the regression results of the ten layers of heights is 0.0294mm, and the error of the regression results is low. The results show that the regression effect is good. For molten pools of different cladding layers, the inclination angle of each layer is increased along with the increase of the number of layers, so that the positions of the cladding layers at which the inclination starts are different, and the visual morphology of the molten pools is obviously different. And for the same position of different cladding layers, the visual form of a molten pool and the elongation of a wire rod are changed due to factors such as inclination, and the difference enables the network to return to the residual heights of different cladding layers.

(4) Generalization capability of depth residual error network to welding process parameters

The single-channel multilayer same-direction accumulation process is a multivariable strong coupling process, multiple factors such as current, welding speed, interlayer cooling time and the like are coupled together to jointly influence the residual height of the cladding layer, and the interlayer cooling time plays an important role in the residual height of the cladding layer. Therefore, in order to evaluate the generalization ability of the depth residual error network, the interlayer cooling time is changed to 1min, 2min, 3min and 4min under the condition of keeping other welding parameters unchanged, and the molten pool images of the cladding layers are collected. The acquired weld pool images of each layer are then combined into a training set and a test set at a ratio of approximately 4: 1. As shown in fig. 12 to 15, the results of regression of the residual height of the cladding layer at different interlayer cooling times are shown. The model has good performance, the regression average error of the integral height residual is 0.0466mm, wherein the regression error of the height residual of the cladding layer is 0.0358mm when the interlayer cooling time is 1min, the regression error of the height residual of the cladding layer is 0.0816mm when the interlayer cooling time is 2min, the regression error of the height residual of the cladding layer is 0.0294mm when the interlayer cooling time is 3min, and the regression error of the height residual of the cladding layer is 0.0301mm when the interlayer cooling time is 4 min. As can be seen from FIG. 13, the eighth, ninth and tenth cladding layers had poor regressive effects at a cooling time of 2min, with an average error of 0.2405 mm. The reason for the poor regressive effect is that the undercut phenomenon occurs on the eighth cladding layer, and the flow of the molten pool occurs on the left side of the ninth cladding layer and the tenth cladding layer. Therefore, the network regression capability is not influenced by the interlayer cooling time.

In light of the foregoing description of the preferred embodiment of the present invention, many modifications and variations will be apparent to those skilled in the art without departing from the spirit and scope of the invention. The technical scope of the present invention is not limited to the content of the specification, and must be determined according to the scope of the claims.

Claims

1. The additive residual height prediction method based on the molten pool image and the depth residual error network is characterized by comprising the following steps of: the method comprises the following steps:

（1）

in the formula

Is a predicted value of the number of the frames,

is the true value and n is the number of samples.

2. The method of claim 1, wherein the method comprises: the residual error network structure adopts a residual error block as a basic composition unit, and the residual error network structure extracts image characteristics through the basic structure of the residual error block.

3. The method of claim 1, wherein the method comprises: the basic network structure of the characteristic extraction part of the molten pool image is Resnet-34.

4. The method of claim 1, wherein the method comprises: the puddle image size was chosen to be 640 x 350 and scaled to 200 x 100.