CN110321933B

CN110321933B - Fault identification method and device based on deep learning

Info

Publication number: CN110321933B
Application number: CN201910500215.2A
Authority: CN
Inventors: 胡忠民
Original assignee: Wuhan Wendao Renaissance Intelligent Technology Co ltd
Current assignee: Wuhan Wendao Renaissance Intelligent Technology Co ltd
Priority date: 2019-06-11
Filing date: 2019-06-11
Publication date: 2021-09-14
Anticipated expiration: 2039-06-11
Also published as: CN110321933A

Abstract

The invention relates to the field of image recognition, in particular to a fault recognition method and device based on deep learning, which specifically comprises the following steps: acquiring an initial image of a region to be identified, wherein the initial image is specifically a color map, an entropy map, a brightness map and a saturation map; inputting the initial image of the area to be identified into a first neural network to obtain image characteristics; extracting a feature map of an interested area from the image feature input area in a network; inputting the feature map of the data of the region of interest into a second neural network to obtain a fault region mask; and identifying the fault type according to the shape characteristics of the fault area mask. The invention uses a deep learning mode to automatically identify the fault and judge the fault type of the image of the area which is possibly faulted, provides a high-efficiency and high-accuracy fault automatic identification method, provides a basis for fault processing, and prevents equipment damage or danger caused by faults.

Description

Fault identification method and device based on deep learning

[ technical field ] A method for producing a semiconductor device

The invention relates to the field of image recognition, in particular to a fault recognition method and device based on deep learning.

[ background of the invention ]

In order to ensure the operation safety of equipment, the fault detection system is widely applied to various industries. However, the conventional fault detection is generally based on electric signal feedback, and the application range is narrow.

For some faults which can not be detected by using an electric signal, the current common detection method is to firstly use an image acquisition device to obtain an image of a region to be identified and send the image to an image server, and then an inspector manually observes and analyzes the image to find out the region and the type of the fault. However, manual fault identification is time-consuming and prone to missed or false detection. Under the condition, faults can not be timely and accurately found due to the fact that fault identification is not timely in the operation of the equipment, and equipment damage or safety problems occur.

In view of this, how to overcome the defects existing in the prior art, an automated method is used to implement fault detection and identification, and the problem of untimely and inaccurate fault detection caused by manual identification is avoided.

[ summary of the invention ]

In view of the above deficiencies or needs in the art, the present invention addresses the problem of low efficiency and accuracy in manual fault identification.

The embodiment of the invention adopts the following technical scheme:

in a first aspect, the invention provides a fault identification method based on deep learning, which comprises the following steps: acquiring an initial image of a region to be identified, wherein the initial image is specifically a color map, an entropy map, a brightness map and a saturation map; inputting the initial image of the area to be identified into a first neural network to obtain image characteristics; extracting a feature map of an interested area from the image feature input area in a network; inputting the feature map of the data of the region of interest into a second neural network to obtain a fault region mask; and identifying the fault type according to the shape characteristics of the fault area mask.

Preferably, the inputting the initial image of the region to be identified into a first neural network to obtain image features specifically includes: the first neural network is a Resnet-101 model; inputting the color map, the entropy map, the brightness map and the saturation map of the area to be identified into a fourth convolution module of Resnet-101 together for feature extraction; and combining the extracted features in series.

Preferably, the extracting the image feature input area from the network to obtain the feature map of the region of interest specifically includes: the area extraction network is a feature extraction network of a master R-CNN model; inputting the image characteristics into a characteristic extraction network of a master R-CNN model to obtain an interested area; and mapping the acquired region of interest into a feature map.

Preferably, the mapping the acquired region of interest into a feature map further includes: and when the number of height pixels and the number of width pixels of the region of interest are non-integers, mapping the region of interest into the feature map by using the target detection special layer.

Preferably, the method further comprises the following steps: the first neural network and the second neural network need to be trained before use; acquiring a fault image to form a pre-training data set, and pre-training a full convolution neural network by using the data set; the first and second neural networks use the same pre-training parameters as the master R-CNN model.

Preferably, the inputting the feature map of the region of interest data into the second neural network to obtain the fault region mask specifically includes: the second neural network is a tandem model of the fifth convolutional layer of Resnet-101 and at least one fully convolutional network.

Preferably, the inputting the feature map of the region of interest data into the second neural network to obtain the fault region mask specifically includes: carrying out 7X7 convolution operation on the feature map to obtain a second feature map, wherein the channel of the second feature map is 1024 dimensions; performing convolution operation on the fifth convolution layer of Resnet-101 on the second characteristic diagram to obtain a third characteristic diagram; carrying out 7X7 convolution operation on the third feature map after the convolution operation to obtain a fourth feature map, wherein the fourth feature map channel is 2048-dimensional; performing a convolution operation on the fourth feature map using a 4096-dimensional full convolution network; the failure mask is output using a 256-dimensional full convolution network.

Preferably, the identifying the fault type according to the shape feature of the fault area mask specifically includes: calculating the number of fault areas according to the connectivity of the mask images of the fault areas; and judging whether each fault area is a large-area fault or not according to the size of the mask image area of the fault area and a preset fault area threshold value.

In another aspect, the present invention provides a fault identification apparatus based on deep learning: the fault recognition method based on deep learning comprises at least one processor and a memory, wherein the at least one processor and the memory are connected through a data bus, and the memory stores instructions which can be executed by the at least one processor, and the instructions are used for completing the fault recognition method based on deep learning according to any one of claims 1 to 8 after being executed by the processor.

Preferably: the system also comprises an image real-time acquisition device which is used for acquiring an initial image of the area to be identified; and the data output port of the image real-time acquisition device is connected with the processor so that the processor can acquire the initial image to be identified for processing.

Compared with the prior art, the embodiment of the invention has the beneficial effects that: and analyzing and detecting the picture of the fault area to be identified by using a deep learning mode, judging whether a fault exists or not, and judging the type of the fault. By using the deep learning method to automatically identify the fault area, the efficiency and accuracy of fault identification can be improved, and fault missing detection or error identification can be avoided.

The invention provides a fault identification method and device based on deep learning, and aims to replace traditional manual identification with the deep learning method so as to improve the efficiency and accuracy of equipment fault identification.

[ description of the drawings ]

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly described below. It is obvious that the drawings described below are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.

Fig. 1 is a flowchart of a fault identification method based on deep learning according to an embodiment of the present invention;

fig. 2 is a schematic diagram of a neural network structure of a deep learning-based fault identification method according to an embodiment of the present invention;

fig. 3 is a schematic diagram of a neural network structure of a deep learning-based fault identification method according to an embodiment of the present invention;

fig. 4 is a schematic diagram illustrating a principle of extracting a mask of a fault region based on a fault identification method of deep learning according to an embodiment of the present invention;

fig. 5 is a schematic diagram of a neural network structure of a deep learning-based fault identification method according to an embodiment of the present invention;

fig. 6 is a schematic device structure diagram of a fault identification method based on deep learning according to an embodiment of the present invention.

[ detailed description ] embodiments

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The present invention is a system structure of a specific function system, so the functional logic relationship of each structural module is mainly explained in the specific embodiment, and the specific software and hardware implementation is not limited.

In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other. The invention will be described in detail below with reference to the figures and examples.

Example 1：

When the field image is used for identifying and detecting equipment faults, the efficiency is lower by adopting a manual identification detection mode, and false detection and missing detection can be caused by human factors.

With the development of deep learning techniques, techniques for image recognition using deep learning techniques have been provided as a basis for use in more environments. Therefore, the device field image can be automatically identified by using a deep learning technology to replace the existing manual identification mode.

The invention provides a fault identification method based on deep learning, which comprises the following steps as shown in figure 1:

step 101: and acquiring an initial image of the area to be identified, wherein the initial image is specifically a color map, an entropy map, a brightness map and a saturation map.

In manual identification and common image identification software, an image to be identified is only a color image or a gray image, and image information is less. In this case, the recognition is not accurate for the interference conditions such as shadow, occlusion, edge overlap, etc.

In some specific use scenes of the embodiment, through image preprocessing, an entropy diagram, a brightness diagram and a saturation diagram of the same scene are generated from an original color picture of a scene to be identified, which is collected by an image acquisition device, and more image features are extracted, so that the identification accuracy is improved.

In some specific use scenarios of this embodiment, the original color picture is obtained by the real-time image capturing device, and the real-time image capturing device periodically obtains the area photograph or the sensing picture that may have a fault, and analyzes the area photograph or the sensing picture as the color map of the initial image.

Step 201: and inputting the initial image of the area to be identified into a first neural network to obtain image characteristics.

In the embodiment, four image types are used as the initial image for fault identification, each image type strengthens the characteristics of the image in different aspects, and filters partial image information to prevent interference, so that the characteristics of each pixel point in the image in different aspects are analyzed and identified more accurately. On the other hand, different features in the plurality of initial images need to be integrated together for analysis, and the comprehensive feature of each pixel point in the image can be accurately obtained. Therefore, when the initial image is subjected to fault identification, the comprehensive features of all images in the same scene need to be extracted first.

The process of image feature extraction may be implemented using a first neural network. The first neural network may perform space training, extracting different image features from different training sets.

Step 301: and (4) extracting a feature map of the region of interest from the network by inputting the image features into the region.

The range of the region to be identified, i.e. the region of interest, which is generally rectangular, can be roughly obtained from the initial image subjected to feature extraction. The fault position can be preliminarily positioned by acquiring the region of interest, the number of pixels to be processed in further analysis and calculation is reduced, and the calculation efficiency and accuracy are improved.

Step 401: and inputting the feature map of the data of the region of interest into a second neural network to obtain a fault region mask.

After the interesting area is obtained, the label of each pixel point in the interesting area is predicted by using a second neural network, whether the pixel point is a fault area or not is judged, and a pixel point set of the fault area is a fault area mask.

In some practical use scenarios of this embodiment, the fault area mask may be visualized, and the fault area mask is represented by a visual element different from that of the initial picture, for example, a fault area contour line is marked by using different colors or by using different colors, so that relevant personnel can visually obtain fault area information.

Step 501: and identifying the fault type according to the mask shape characteristics of the fault area.

After the fault area mask is obtained, the fault type can be judged and identified according to the geometric shape characteristics of the fault area mask, and fault pictures are classified. Specifically, when the number of pixels in the failure area in one initial picture is 0, it is determined that there is no failure.

The fault identification method provided by the embodiment uses the neural network to perform feature identification and fault type identification on the initial picture of the area which is likely to have the fault, and the process does not need manual intervention. And due to the characteristics of the neural network, the method has higher identification accuracy and identification speed. In some specific use scenarios of the present embodiment, the present embodiment is used for identifying a high-speed rail oil leakage fault, and the processing speed of an initial picture with a size of 3072 × 2048 pixels is less than 1 second, and the identification accuracy is greater than 90%.

Compared with manual detection, the fault identification method provided by the embodiment saves labor cost, and improves working efficiency and detection precision.

Moreover, the automatic fault recognition system can be combined with other automatic control systems to form an automatic fault processing system, and when the fault is recognized, the automatic fault recognition system automatically gives an alarm and processes the fault, so that the fault expansion or potential safety hazard caused by untimely manual recognition and processing is avoided.

Example 2：

At present, a plurality of mature neural network models for image recognition exist, and different neural network models have different characteristics and are suitable for different application fields.

In this embodiment, the first neural network is a Resnet-101 model. Resnet-101 belongs to a residual error network, the residual error network is easier to optimize compared with other types of neural networks, the phenomenon that the accuracy rate is reduced along with depth deepening caused by overfitting can be avoided, and the accuracy rate can be improved by increasing the depth.

The internal structure diagram and parameters of the convolution module of the Resnet-101 model used in this embodiment are as shown in fig. 2, the first layer of 1x1 convolution reduces the number of parameters to 256 dimensions, the second layer of 3x3 convolution ensures the calculation accuracy on the basis of keeping the number of parameters to 256 dimensions, and the last layer of 1x1 convolution restores the parameters to 1024 dimensions to improve the calculation efficiency and avoid the accuracy reduction with the depth increase.

The specific steps of using the Resnet-101 model to extract the features of the image of the area to be recognized are as follows:

step 201-a: and inputting the color map, the entropy map, the brightness map and the saturation map of the area to be identified into a fourth convolution module of Resnet-101 together for feature extraction.

The fault identification method provided by this embodiment needs to extract different features from the four initial images for analysis, and the fourth convolution module of Resnet-101 can extract enough information for judgment, so that the fourth convolution module is at an appropriate depth.

Step 201-b: and combining the extracted features in series.

After feature extraction is performed on the four initial images, the data after feature extraction needs to be combined in series so as to facilitate the next comprehensive analysis and processing.

The fourth convolution module of the Resnet-101 model is selected as a first neural network, namely a feature extraction network, so that feature extraction can be rapidly and accurately carried out on four initial images, and enough information can be obtained for next fault identification, so that the fault identification can be carried out more accurately.

In this embodiment, the feature extraction network of the faster R-CNN model is selected as the regional extraction network. The fast R-CNN model is used as a powerful feature extraction model, can accurately perform target detection, can replace different feature extraction layers, and can extract image features by using VGG or resnet according to needs for subsequent target detection.

The specific steps of extracting the region of interest from the image of the region to be identified after feature extraction by using the feature extraction network of the fast R-CNN model are as follows:

step 301-a: and inputting the image characteristics into a characteristic extraction network of the master R-CNN model to obtain an interested area.

Step 301-b: and mapping the acquired region of interest into the feature map.

As shown in fig. 3, in the specific structure of the neural network, obtaining the region of interest of the feature map from the initial image of the region to be identified requires first using the feature extraction network of the fast R-CNN model to obtain the region of interest of the initial image, and then combining the region of interest with the initial image features obtained in step 201 to obtain the feature map to be identified.

The region of interest and the feature map are obtained, so that possible fault regions can be preliminarily determined, redundant data are reduced for further fault image feature identification, and the identification accuracy and the identification efficiency are improved.

Further, the height w and the width h of the region of interest obtained in step 301-a may be non-integer, and the coordinates of each point in the region of interest may also be non-integer. When the region of interest is mapped to the feature map, block convolution needs to be carried out on the feature map, and when the height w and the width h of the region of interest are non-integers, a decimal part inevitably occurs when the block convolution is carried out. Therefore, it is necessary to use a special layer of object detection, namely ROIAlign, to implement the mapping of the region of interest to the feature map.

As shown in fig. 4, the steps for implementing the feature map when the height w and the width h of the region of interest are floating point numbers by using roiign are as follows:

step 301-b-1: and traversing each candidate area needing block convolution, and keeping the floating point number boundary not to be quantized.

Step 301-b-2: and dividing the candidate area of the block convolution into k x k units, wherein the boundary of each unit is not quantized.

Step 301-b-3: fixed n coordinate positions are calculated in each cell, the values of the n positions are calculated by a bilinear interpolation method, and then the maximum pooling operation is carried out.

The fixed position in step 301-b-3 refers to a position determined in accordance with a fixed rule in each rectangular unit. Specifically, in some specific application scenarios of the present embodiment, the number of sampling points is n-1, and the sampling point is the central point of the cell. In other specific application scenarios of this embodiment, the number of sampling points is n-4, and the sampling point is the central point of each region after the unit is equally divided into four rectangular regions by the transverse and longitudinal center lines. Since the height h and width w of the region are floating point numbers, the coordinates of the sampling point are usually floating point numbers, and therefore, an interpolation method is required to obtain the pixel value thereof. Preferably, the position values of the sampling points may be calculated using bilinear interpolation.

In this embodiment, the first neural network selects a Resnet-101 model, the regional extraction network selects a feature extraction network of the master R-CNN model, and the special condition of the feature map mapping is handled by using roiign. The mature deep learning network model is used, adaptive modification and optimization are carried out on the basis of the model, the calculation efficiency is improved on the basis of ensuring the precision, and fault identification can be carried out more quickly and accurately.

The preliminary calculation result of the fault identification is a fault area mask, namely a set of all pixel points taking the prediction label as a target. The process of obtaining the fault area mask is to predict the label of each pixel point in the area of interest by using a series of convolution operations of the second neural network, and determine whether the label is a target pixel point of the predicted label.

In this embodiment, the second neural network is a tandem model of the fifth convolutional layer of Resnet-101 and at least one fully convolutional network.

Specifically, as shown in fig. 5, the step of calculating the fault region mask using the second neural network is as follows:

step 401-a: and carrying out 7X7 convolution operation on the feature map to obtain a second feature map, wherein the channel of the second feature map is 1024 dimensions.

Step 401-b: and performing convolution operation on the fifth convolution layer of the Resnet-101 on the second characteristic diagram to obtain a third characteristic diagram.

Step 401-c: and performing 7X7 convolution operation on the convolved third feature map to obtain a fourth feature map, wherein the fourth feature map channel is 2048-dimensional.

Step 401-d: the fourth feature map is convolved using a 4096-dimensional full convolution network.

Step 401-e: the failure mask is output using a 256-dimensional full convolution network.

In this embodiment, a fault mask is obtained by using a series model of the fifth convolutional layer of Resnet-101 and 2048-dimensional and 4096-dimensional full convolutional networks, and a fault detection model is finally obtained by using a 256-dimensional full convolutional network for output.

In some specific usage scenarios of this embodiment, the output of the fault mask model is in a picture format, and the fault mask region is identified by a visual symbol different from that of the initial image, for example, a pixel point of the fault mask region is set to a preset warning color, or a polygon frame is used to mark an edge of the fault mask region.

Example 3：

The deep learning neural network needs pre-training and parameter adjustment before use, so that the neural network can identify the characteristics of a specific image.

In some specific implementation scenarios of this embodiment, the fault to be identified is a high-speed rail oil leakage fault. In the scene, the area to be identified is an area where oil leakage is likely to occur, and the initial image of the area to be identified is a real-time photo of the area where oil leakage is likely to occur. Under the scene, the pre-training data set is an oil leakage fault picture, and specifically can be different types of oil leakage fault pictures, such as single-point oil leakage, multi-point oil leakage and the like.

The fault identification method provided by the embodiment mainly extracts the characteristics of the fault and the normal picture through the convolutional neural network so as to judge whether the region shown by the picture has the fault, so that the method can be suitable for different types of fault identification by changing the pre-training data set.

In different use scenarios of the embodiment, the pre-training data set is transformed according to the type of the fault to be identified, such as a true fault of a screw, a moving fault of a part, and the like. Because the fault identification method provided by the embodiment is used for identifying through the picture visual characteristics, the fault identification method is only suitable for fault types which can be distinguished through a visual method, and faults which cannot be distinguished through the visual method, such as electrical faults and the like, cannot be identified.

In the fault identification method provided by this embodiment, the Resnet-101 model neural network used for image feature extraction and fault area mask acquisition uses the same pre-training parameters as the master R-CNN model, and the same pre-training parameters can ensure the consistency of feature identification and ensure the accuracy of identification.

After the initial image of the region to be identified is subjected to deep learning fault identification to obtain a fault region mask, the fault type can be analyzed according to the shape characteristics of the fault region, and the specific fault type is further identified.

In some specific use scenarios of this embodiment, the faults may be classified in different classification modes according to the shape of the fault region, the following is a simple list of common fault classification modes, and the faults may also be classified in different modes according to actual needs:

(1) the method can be divided into the following steps according to the number of fault points: single point failure, multiple point failure. The geometric connectivity of the images in the fault area can be judged, only one connected image in the fault area is a single-point fault, and a plurality of non-connected images in the fault area are multi-point faults. In the high-speed rail oil leakage fault, a single-point fault indicates that only one oil leakage point exists, and a multi-point fault indicates that a plurality of oil leakage points exist.

(2) The method can be divided into the following steps according to the area size of a fault point: large area failures, small area failures. The judgment can be carried out by calculating the area of the image of the fault area, the fault area larger than the preset fault area threshold value is a large-area fault, and the fault area smaller than the preset fault area threshold value is a small-area fault. In the high-speed rail oil leakage fault, a preset fault area threshold value is 2cm x 2cm, a large-area fault indicates that the oil leakage amount is large, and a small-area fault indicates that the oil leakage amount is small.

The faults can be further classified and identified through fault type analysis, and the faults with different severity degrees are processed in different forms, such as alarm with different intensities and the like. Fault classification can better assist manual fault identification, so that major faults are more easily identified, and untimely identification of serious accidents or other adverse consequences caused by low manual fault classification efficiency are avoided.

Furthermore, the fault identification method based on deep learning provided by the embodiment can be integrated into the existing equipment control and fault detection system, and automatically processes the fault of the equipment according to the fault identification result, so that the fault response speed is further improved, and the running safety of the equipment is guaranteed. In some specific use scenarios of this embodiment, the identified fault is a high-speed rail oil leakage fault, and the fault identification method may be integrated into the existing TDFS fault detection system of the high-speed rail fault detection system through a python interface, so that an alarm and fault processing may be performed in time when an oil leakage fault occurs in a high-speed rail, and a high-speed rail operation safety accident caused by the oil leakage fault is prevented.

Example 4：

On the basis of the fault identification method based on deep learning provided in the foregoing embodiments 1 to 3, the present invention further provides a fault identification device based on deep learning, which can be used for implementing the above method, as shown in fig. 6, is a schematic diagram of a device architecture of an embodiment of the present invention. The fault recognition apparatus based on deep learning of the present embodiment includes one or more processors 21 and a memory 22. In fig. 6, one processor 21 is taken as an example.

The processor 21 and the memory 22 may be connected by a bus or other means, and fig. 6 illustrates the connection by a bus as an example.

The memory 22, which is a nonvolatile computer-readable storage medium for a deep learning based fault recognition method, may be used to store nonvolatile software programs, nonvolatile computer-executable programs, and modules, such as the deep learning based fault recognition method in embodiment 1. The processor 21 executes various functional applications and data processing of the deep learning based fault recognition apparatus by running the nonvolatile software programs, instructions, and modules stored in the memory 22, that is, implements the deep learning based fault recognition methods of embodiments 1 and 2.

The memory 22 may include high speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, the memory 22 may optionally include memory located remotely from the processor 21, and these remote memories may be connected to the processor 21 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The program instructions/modules are stored in the memory 22 and, when executed by the one or more processors 21, perform the deep learning based fault identification method of embodiments 1 to 3 described above, for example, perform the respective steps shown in fig. 1 to 5 described above.

Those of ordinary skill in the art will appreciate that all or part of the steps of the various methods of the embodiments may be implemented by associated hardware as instructed by a program, which may be stored on a computer-readable storage medium, which may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like.

Further, the fault recognition device based on deep learning provided by the embodiment further includes an image real-time acquisition device, such as an image sensor. The image real-time acquisition device is used for acquiring a color map of an initial image of an area to be identified, and is source data for fault identification. The data port of the image real-time acquisition device can be directly connected with the processor 21 of the fault identification device, or indirectly connected with the processor 21 through the memory 22, and the collected initial map of the area to be identified is transmitted to the processor 21 through the data port for fault identification.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.

Claims

1. A fault identification method based on deep learning is characterized by comprising the following steps:

acquiring an initial image of a region to be identified, wherein the initial image is specifically a color map, an entropy map, a brightness map and a saturation map;

inputting all the initial images of the area to be identified into a fourth convolution module of a first neural network, respectively extracting features, and performing series combination on the extracted features, wherein the first neural network is a Resnet-101 model;

inputting the image features after series combination into a feature extraction network of a faster R-CNN model to obtain an interested area, and mapping the obtained interested area into a feature map, wherein the region extraction network is the feature extraction network of the faster R-CNN model, and the faster R-CNN model and the Resnet-101 model use the same pre-training parameters;

inputting the feature map of the region of interest into a second neural network to obtain a fault region mask, specifically, performing 7X7 convolution operation on the feature map to obtain a second feature map, wherein a channel of the second feature map is 1024 dimensions; performing convolution operation on the fifth convolution layer of Resnet-101 on the second characteristic diagram to obtain a third characteristic diagram; carrying out 7X7 convolution operation on the third feature map after the convolution operation to obtain a fourth feature map, wherein the fourth feature map channel is 2048-dimensional; performing convolution operation on the fourth feature map by using a 4096-dimensional full convolution network, and outputting a fault area mask by using a 256-dimensional full convolution network;

and identifying the fault type according to the mask shape characteristics of the fault area and/or the number of fault points.

2. The method for fault recognition based on deep learning according to claim 1, wherein the mapping the acquired region of interest into a feature map further comprises:

and when the number of height pixels and the number of width pixels of the region of interest are non-integers, mapping the region of interest into the feature map by using the target detection special layer.

3. The fault identification method based on deep learning according to claim 1 or 2, characterized by further comprising:

the first neural network and the second neural network need to be trained before use;

acquiring a fault image to form a pre-training data set, and pre-training a full convolution neural network by using the data set;

the first and second neural networks use the same pre-training parameters as the master R-CNN model.

4. The method for fault identification based on deep learning of claim 1, wherein the inputting the feature map of the region of interest into a second neural network to obtain a fault region mask specifically comprises:

the second neural network is a tandem model of the fifth convolutional layer of Resnet-101 and at least one fully convolutional network.

5. The method for fault identification based on deep learning of claim 1, wherein the fault type is identified according to the mask shape feature of the fault area, specifically:

calculating the number of fault areas according to the connectivity of the mask images of the fault areas;

and judging whether each fault area is a large-area fault or not according to the size of the mask image area of the fault area and a preset fault area threshold value.

6. The utility model provides a fault identification device based on deep learning which characterized in that:

the fault recognition method based on deep learning comprises at least one processor and a memory, wherein the at least one processor and the memory are connected through a data bus, and the memory stores instructions which can be executed by the at least one processor, and the instructions are used for completing the fault recognition method based on deep learning according to any one of claims 1 to 5 after being executed by the processor.

7. The deep learning based failure recognition device according to claim 6, wherein:

the system also comprises an image real-time acquisition device which is used for acquiring an initial image of the area to be identified;

and the data output port of the image real-time acquisition device is connected with the processor so that the processor can acquire the initial image to be identified for processing.