CN116645325A

CN116645325A - Defect marking method and device for photovoltaic panel, medium and electronic equipment

Info

Publication number: CN116645325A
Application number: CN202310431511.8A
Authority: CN
Inventors: 王广群; 李大钧; 孙金龙; 史明亮; 夏友冬; 刘海龙; 张欣; 张震; 燕艳芬
Original assignee: Guohua Energy Investment Co ltd; Zhongxin Hanchuang Beijing Technology Co Ltd
Current assignee: Guohua Energy Investment Co ltd; Zhongxin Hanchuang Beijing Technology Co Ltd
Priority date: 2023-04-20
Filing date: 2023-04-20
Publication date: 2023-08-25

Abstract

The disclosure relates to a defect labeling method, a device, a medium and electronic equipment for a photovoltaic panel, wherein the defect labeling method for the photovoltaic panel comprises the following steps: acquiring an image to be marked, wherein the image to be marked is an infrared image containing hot spots; inputting an image to be marked into a trained Faster-RCNN model to obtain a target image; the target image is an image to be marked of an area where a hot spot is located, a main network of a fast-RCNN model is a residual network, the residual network comprises a plurality of convolution layers and a full-connection layer which are sequentially connected, the convolution layers are input ends, and the full-connection layer is output end; the convolution manner of the target convolution layer in the multi-layer convolution layers is deformable convolution. According to the method, the convolution mode of the target convolution layer in the multi-layer convolution layer is deformable convolution, and the deformable parameters are introduced, so that the method can be better adapted to deformation of the object, and the accuracy of identification and segmentation is improved.

Description

Defect marking method and device for photovoltaic panel, medium and electronic equipment

Technical Field

The disclosure relates to the technical field of image processing, in particular to a defect labeling method, device, medium and electronic equipment of a photovoltaic panel.

Background

The hot spot effect has a larger influence on the photovoltaic power station, so that the power generation efficiency of the photovoltaic module is influenced, and a fire disaster can be generated, so that the safety of the photovoltaic power station is a larger threat. The generation of the hot spot effect has certain damage to the photovoltaic module, so the photovoltaic module generating the hot spot effect needs to be found out in time through inspection.

In the related art, the unmanned aerial vehicle is utilized to shoot a returned video, and then the hot spot position is manually calibrated, so that the hot spot is formed on an infrared image and has almost no obvious characteristic, the hot spot is influenced by human vision, the manual calibration is easy to miss, and the human resources are wasted.

Disclosure of Invention

In order to solve the problems in the related art, the disclosure provides a defect labeling method, a device, a medium and electronic equipment for a photovoltaic panel.

A first aspect of the present disclosure provides a defect labeling method for a photovoltaic panel, including:

acquiring an image to be marked, wherein the image to be marked is an infrared image containing hot spots;

inputting the image to be marked into a trained Faster-RCNN model to obtain a target image;

the target image is the image to be marked for marking the region where the hot spot is located, the main network of the fast-RCNN model is a residual network, the residual network comprises a plurality of convolution layers and a full-connection layer which are sequentially connected, the convolution layers are input ends, and the full-connection layer is an output end; the convolution mode of the target convolution layer in the multi-layer convolution layers is deformable convolution.

Optionally, the residual network adopts a ResNet18 network, wherein the target convolution layer comprises a plurality of layers, and the plurality of layers of the target convolution layer are connected with the full connection layer after being sequentially connected.

Optionally, the area generating network of the fast-RCNN model adopts an FPN network, where the FPN network is used to extract detailed features of the feature image output by the residual error network, and generate a to-be-detected labeling frame for labeling a target area, and the target area includes an area where the hot spots are located.

Optionally, the FPN network includes a 3×3 convolution layer and two 1×1 branch convolution layers connected to the 3×3 convolution layer, where the 3×3 convolution layer is configured to perform detail feature extraction on the feature image and generate the to-be-detected labeling frame, where one 1×1 branch convolution layer is configured to determine whether the to-be-detected labeling frame includes a hot spot, and the other 1×1 branch convolution layer is configured to position and adjust a position of the to-be-detected labeling frame.

Optionally, the convolution manner of the 3×3 convolution layer is an expanded convolution, and the 3×3 convolution layer includes 3 convolution layers, and expansion rates of the 3 convolution layers are 2, 4, and 6, respectively.

Optionally, the fast-RCNN model includes a coding module, the coding module is connected to an output end of an ROI Pooling layer of the fast-RCNN model, the ROI Pooling layer integrates the feature image and the to-be-detected labeling frame, and outputs a plurality of feature images with the to-be-detected labeling frame having a confidence degree greater than a preset confidence degree, the coding module includes:

the feature conversion module comprises a 1 multiplied by 1 convolution layer and is used for converting the plurality of feature images into the same size and outputting the same size as an image to be coded;

and the plurality of groups of coding classification modules are connected with the characteristic conversion module, and are sequentially connected with each other and used for coding and classifying all pixel positions in the image to be coded, and classification results comprise non-hot spot pixels and hot spot pixels.

Optionally, the fully connected layer of the fast-RCNN model includes:

the information classification module is used for acquiring first position information of an area where the hot spot is located and second position information of a target boundary of the photovoltaic panel in the characteristic image, determining a topological relation between the first position information and the second position information, wherein the topological relation is used for determining a target position of the to-be-marked frame in the to-be-marked image so as to mark the area where the hot spot is located, and obtaining the target image, and the target boundary is any boundary of a photovoltaic assembly, wherein the distance between the photovoltaic panel and the first position information is smaller than a preset distance;

and the classification module is used for classifying the hot spots in the characteristic images.

A second aspect of the present disclosure provides a defect labeling device for a photovoltaic panel, including:

the acquisition module is configured to acquire an image to be marked, wherein the image to be marked is an infrared image containing hot spots;

the obtaining module is configured to input the image to be marked into a trained fast-RCNN model to obtain a target image;

A third aspect of the present disclosure provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method provided by the first aspect of the present disclosure.

A fourth aspect of the present disclosure provides an electronic device, comprising:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement the method provided by the first aspect of the present disclosure.

The method comprises the steps of inputting an image to be marked into a trained Faster-RCNN model to obtain a target image, wherein a main network of the Faster-RCNN model is an improved residual network, and a convolution mode of a target convolution layer in a multi-layer convolution layer of the residual network is deformable convolution. The convolution mode of the target convolution layer in the multi-layer convolution layer is deformable convolution, and deformable parameters are introduced, so that deformation of an object can be well adapted, recognition and segmentation accuracy is improved, offset of pixel points in a feature image input into the deformable convolution layer can be automatically calculated, proper features are extracted from the feature image to carry out convolution, a convolved region is concentrated on a region where hot spots are located as much as possible, and real-time recognition and detection of the hot spot region of the photovoltaic panel are realized. In addition, the deformable convolution mode can also effectively improve the resolution of the image and increase the feature richness of the hot spots in a small range.

Additional features and advantages of the present disclosure will be set forth in the detailed description which follows.

Drawings

The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification, illustrate the disclosure and together with the description serve to explain, but do not limit the disclosure. In the drawings:

fig. 1 is a flowchart illustrating a defect labeling method of a photovoltaic panel according to an exemplary embodiment.

FIG. 2 is a schematic diagram of the architecture of a Faster-RCNN model, in accordance with an illustrative embodiment.

Fig. 3 is a schematic diagram illustrating the connection of a FPN network to a res net18 network, according to an example embodiment.

Fig. 4 is a block diagram illustrating a defect marking apparatus for a photovoltaic panel according to an exemplary embodiment.

Fig. 5 is a block diagram of an electronic device, according to an example embodiment.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.

The solar photovoltaic power generation system is formed by connecting a series of solar component batteries in series. In the operation process of the photovoltaic power station, shielding objects such as bird excrement, floating soil, fallen leaves and the like are inevitably covered on the photovoltaic module, and form partial shadows on the photovoltaic module, so that the current and the voltage of certain battery singlechips in the photovoltaic module are changed, the local temperature of the photovoltaic module is increased, and the phenomenon is called a hot spot effect.

In the related art, the defect detection method based on the conventional image processing method is to extract a defective area and its external rectangle by performing image processing, such as filtering, sharpening, binarization, gray value, morphological processing, etc., on a thermal image, and then perform defect detection by a feature extraction and classifier. This approach requires manual design of features and classifiers, requires redesign for different defect types, and is difficult to handle for complex defects.

Referring to fig. 1, fig. 1 is a flowchart illustrating a method for marking defects of a photovoltaic panel according to an exemplary embodiment, and as shown in fig. 1, the method for marking defects of a photovoltaic panel includes:

s101, acquiring an image to be marked, wherein the image to be marked is an infrared image containing hot spots.

S102, inputting the image to be marked into a trained Faster-RCNN model to obtain a target image.

The target image is an image to be marked of an area where a hot spot is located, a main network of a fast-RCNN model is a residual network, the residual network comprises a plurality of convolution layers and a full-connection layer which are sequentially connected, the convolution layers are input ends, and the full-connection layer is output end; the convolution manner of the target convolution layer in the multi-layer convolution layers is deformable convolution.

Illustratively, the Faster-RCNN model is one feature detection network composed of two neural networks, including a feature extraction network and a region generation network (Region Proposal Network, RPN). The feature extraction network is mainly used for extracting features of an image to be marked so as to provide a feature image of an input area generation network, a classical Convolutional Neural Network (CNN) is generally adopted, a residual error network can be used as the feature extraction network, the ResNet residual error network is used as the feature extraction network, the input is a picture, and the output is a feature map. The RPN network detects detailed features on the feature images according to the feature images obtained by the CNN network, determines the position of the detected object, inputs the detected object as a feature image and an object label, namely the type and the position of the hot spots in the training set; the outputs are Propos al (region box), classification Loss and regression Loss, where Propos al is used as the generated region for subsequent module classification and regression. The fast-RCNN model is formed by adding CNN and RPN networks, inputting the CNN and the RPN networks into pictures, and outputting the pictures into classification and regression, wherein the classification classifies images in the region frame, and the regression is used for predicting the position of the classification frame.

The method comprises the steps of inputting an image to be marked into a trained Faster-RCNN model to obtain a target image, wherein a main network of the Faster-RCNN model is an improved residual network, and a convolution mode of a target convolution layer in a multi-layer convolution layer of the residual network is deformable convolution. The convolution mode of the target convolution layer in the multi-layer convolution layer is deformable convolution, and deformable parameters are introduced, so that the method can be better adapted to deformation of an object, and the accuracy of identification and segmentation is improved. Meanwhile, the offset of the pixel points in the feature map of the input deformable convolution layer can be automatically calculated, proper features are extracted from the feature map to carry out convolution, the convolved region is concentrated on the region where the hot spot is located as much as possible, and the real-time identification and detection of the hot spot region of the photovoltaic panel are realized. In addition, the deformable convolution mode can also effectively improve the resolution of the image and increase the feature richness of the hot spots in a small range.

In some embodiments, the residual network employs a ResNet18 network, wherein the target convolutional layer comprises multiple layers, and the multiple layers of target convolutional layers are connected in sequence and then connected to the full connection layer.

Illustratively, the depth of the ResNet18 is 18, which includes one 7×7 convolution layer, one maximum pooling layer (maxpool), four groups of 3×3 convolution layers, one average pooling layer (avgpool), and one full-connection layer connected in sequence, wherein each group of 3×3 convolution layers includes four layers of 3×3 convolution layers, the target convolution layer may be 3 layers, that is, the convolution manner of the last 3 layers of the last group of 3×3 convolution layers is a deformable convolution.

The residual error network disclosed by the invention adopts the ResNet18 network, the characteristics of the image to be marked are extracted, the network depth is moderate, the network parameters can be reduced, the hot spot detection rate can be improved, and the characteristic loss caused by excessive convolution times is avoided.

In some embodiments, the area generating network of the fast-RCNN model adopts an FPN network, and the FPN network is used for extracting detailed features of the feature image output by the residual network, and generating a to-be-detected labeling frame for labeling a target area, where the target area includes an area where hot spots are located.

For example, when a neural network is used for detecting a hot spot area, the pixel ratio of many small and irregular hot spots on an infrared image is small, and the small target has low resolution and small volume and is difficult to detect. While poor small target detection performance is mainly due to limitations of the network model and unbalance of the training data set. In order to obtain reliable semantic information, many object detection networks attempt to superimpose more and more pooling and downsampling operations so that tiny object features with a smaller number of pixels are gradually lost in forward propagation, thus degrading the detection performance of tiny objects.

The present disclosure proposes a FPN network (feature pyramid network) that combines enhanced context and refined features. And (3) performing dilation convolution on the multi-scale characteristic image output by the residual network to obtain characteristics, and performing fusion injection on the characteristics from top to bottom to supplement context information by the FPN network. The FPN network can alleviate the problem of information diffusion to a certain extent by horizontally fusing the low-resolution feature map and the high-resolution feature map.

In some embodiments, the FPN network includes a 3×3 convolution layer and two 1×1 branch convolution layers connected to the 3×3 convolution layer, where the 3×3 convolution layer is configured to perform detail feature extraction on the feature image and generate a to-be-detected labeling frame, one of the 1×1 branch convolution layers is configured to determine whether the to-be-detected labeling frame includes a hot spot, and the other 1×1 branch convolution layer is configured to position and adjust a size of the to-be-detected labeling frame.

For example, one of the 1×1 branch convolution layers performs a Softmax operation to determine whether a to-be-detected labeling frame contains hot spots, and the other 1×1 branch convolution layer performs a bbox reg operation to position and resize the to-be-detected labeling frame, so as to obtain a plurality of more accurate candidate frames.

In some embodiments, the 3×3 convolution layers are convolved in a spread convolution, and the 3×3 convolution layers comprise 3 convolution layers, and the spread ratios of the 3 convolution layers are 2, 4, and 6, respectively.

Illustratively, the dilation convolution, also known as a hole convolution or dilation convolution, is to inject holes in a standard convolution kernel to increase the receptive field of the model. Compared with the original normal convolution operation, the step size and the filling are increased by one parameter, namely the interval number of points of the convolution kernel. For example, a conventional convolution operation, the parameter adaptation rate is 1. The dilated convolution introduces another parameter for the convolution layer, called the dilation rate, characterizing the spacing between the median values of the convolution kernels. The 3x3 kernel with a dilation rate of 2 has the same field of view as the 5x5 kernel.

In the present disclosure, the 3×3 convolution layer uses 2, 4, 6 dilation convolutions with different dilation rates to obtain context information of different receptive fields, the receptive fields are enlarged without losing the feature resolution, and the 3×3 convolution layer uses different dilation rates to obtain multi-scale information, and the FPN network is injected from top to bottom to enrich the context information.

In some embodiments, the fast-RCNN model includes a coding module, the coding module is connected to an output end of an ROI Pooling layer of the fast-RCNN model, the ROI Pooling layer integrates a feature image and a to-be-detected annotation frame, and outputs a plurality of feature images with to-be-detected annotation frames having a confidence greater than a preset confidence, the coding module includes:

the feature conversion module comprises a 1 multiplied by 1 convolution layer and is used for converting a plurality of feature images into the same size and outputting the same size as an image to be encoded;

the multi-group coding classification module is connected with the feature conversion module, and is used for coding and classifying all pixel positions in the image to be coded, and classification results comprise non-hot spot pixels and hot spot pixels.

For example, after the FPN network detail feature extraction, a branch is added to divide the irregular boundary of the hot spot, and the specific operation is that a plurality of feature images with to-be-detected marking frames, which are output by the ROI Pooling layer and have confidence degrees larger than preset confidence degrees, are amplified to the same size through a 1×1 convolution layer, are output as images to be encoded, all pixel positions in the images to be encoded are encoded and classified by a plurality of groups of encoding classification modules (Transformer encoder layers), the encoding classification modules output matrixes with the same size as the input, and the classification result of the pixel positions comprises two element categories of non-hot spot pixels and hot spot pixels, so that the irregular boundary of the hot spot is divided according to the non-hot spot pixels and the hot spot pixels. The code classification module comprises 5 groups, and the preset confidence coefficient can be 0.8.

In some embodiments, the fully connected layer of the fast-RCNN model includes:

the information classification module is used for acquiring first position information of an area where a hot spot is located and second position information of a target boundary of a photovoltaic panel in a characteristic image, determining a topological relation between the first position information and the second position information, wherein the topological relation is used for determining a target position of the to-be-marked frame in the to-be-marked image so as to mark the area where the hot spot is located, and obtaining the target image, and the target boundary is any boundary of a photovoltaic assembly, wherein the distance between the photovoltaic assembly and the first position information in the photovoltaic panel is smaller than a preset distance;

Illustratively, a full connectivity layer (FC layers) is split into two, outputting two sets of one-dimensional feature information, respectively. Specifically, the information classification module performs a bbox reg operation (a bounding box regression method) for dividing a first group of one-dimensional features into first position information of an area where hot spots are located and second position information of a target boundary of the photovoltaic panel, determining a topological relation between the first position information and the second position information, and performing a Softmax operation on a second group of one-dimensional features by the classification module for classifying all the hot spots in the feature images.

In addition, as the hot spots only appear in the rectangular photovoltaic module in the photovoltaic panel, the topological relation between the hot spots and the boundary of the photovoltaic panel can effectively reduce the gap between the hot spot position predicted by the fast-RCNN model and the actual hot spot position, and the method specifically comprises the steps of removing the pixel positions where the predicted hot spots fall on all the boundary of the photovoltaic module before determining the topological relation between the first position information and the second position information, carrying out k-means clustering on the residual pixel positions where the hot spots are located, and calculating the comprehensive loss by using a Tversky loss function.

In some embodiments, before acquiring the image to be annotated, the method further comprises:

acquiring an image to be processed, wherein the image to be processed is an infrared image containing hot spots;

and carrying out pretreatment operation on the image to be processed to obtain the image to be marked.

By way of example, an infrared image acquired in real time by an infrared camera of an unmanned aerial vehicle is used for preprocessing a row of photovoltaic panels positioned in an image middle vertical bar area in the infrared image by utilizing Gaussian filtering, self-adaptive binarization and morphological filtering image algorithm or image size adjustment, and then the preprocessed infrared image is used for recognizing hot spots by adopting a Faster RCNN model improved by the method.

Referring to fig. 2, fig. 2 is a schematic diagram illustrating the structure of a fast-RCNN model according to an exemplary embodiment. As shown in fig. 2, the Head layer of the fast-RCNN is replaced by a res net18 network, the convolution mode of the last three convolution layers of the res net18 network is replaced by a deformable convolution (Deep Deformable Convolution), and a plurality of FPN networks for extracting detail features are added on the basis of the original Baseline, namely, the part of a dashed line frame in the figure, wherein the FPN networks comprise a 3×3 convolution layer and two 1×1 branch convolution layers connected with the 3×3 convolution layer. And the ROI pulling layer is connected with the output end of the ResNet18 network and the output end of the FPN network, the bbox reg operation is executed through the information classification module of the full connection layer, and the Softmax operation is executed through the classification module of the full connection layer. A branch is also connected to the output of the ROI shaping layer, which includes a 1×1 convolutional layer and a code classification module (Transformer encoder layers) connected to the 1×1 convolutional layer.

The ROI Mapping of the FPN network means that each layer of the FPN network is corresponding to a different scale of the output of the res net18 network, the connection form of the ROI Mapping is shown in fig. 3, for example, the output of the last Block of each group of convolution layers is recorded as C1 (the last Block of the 7×7 convolution layers), C2 (the last Block of the first group of 3×3 convolution layers), C3 (the last Block of the second group of 3×3 convolution layers), C4 (the last Block of the third group of 3×3 convolution layers) and C5 (the last Block of the fourth group of 3×3 convolution layers) in turn, the FPN network comprises P1 to P6 stages, wherein P2 to P5 stages are used for participation in prediction, and the last Block layers C2, C3, C4 and C5 of each group of convolution layers are used as input features of different layers of the FPN network in order, and P6 is obtained by downsampling P6 on the P5 stage for the RPN network. It will be appreciated that the 3×3 convolution layers include 3 convolution layers corresponding to the P2 to P4 stages of the FPN network, respectively, and two 1×1 branch convolution layers corresponding to the P5 stage of the FPN network. And a plurality of paths of feature extraction is added on the basis of the original Baseline, so that feature loss caused by non-optimal Roi Mapping of the FPN of the candidate annotation frame under the defect of the extreme aspect ratio can be compensated.

Specifically, an image to be marked is Input (Input) to a ResNet18 network, the image to be marked is output as a Feature image (Feature map) after Feature extraction, the Feature image is subjected to detail Feature extraction through a 3×3 convolution layer of Roi Mapping of FPN, a mark frame to be marked for a target area is generated, a 1×1 branch convolution layer executes Softmax operation, whether hot spots are contained in the mark frame to be marked is determined, the 1×1 branch convolution layer executes bbox reg operation so as to position and adjust the mark frame to be marked, a plurality of accurate candidate frames are obtained, the Feature image and the mark frame to be marked generated by the FPN are integrated by the ROI Mapping layer, a Feature image containing the mark frame to be marked with high confidence is obtained, the Feature image is further sent to a full connection layer, the hot spot type and the predicted value contained in the area to be marked are calculated, the full connection layer is utilized to execute the Softmax operation, the specific type of the candidate area is determined, and the bbox reg operation is executed again to obtain the final position of the predicted mark frame. The feature images output by the ROI Pooling layer are further converted into images to be encoded by the 1X 1 convolution layer and five groups of encoding classification modules (Transformer encoder layers), the images to be encoded are output into images to be encoded by the 1X 1 convolution layer, all pixel positions in the images to be encoded are encoded and classified by the encoding classification modules, classification results comprise non-hot spot pixels and hot spot pixels, matrixes with the same size as input are output, irregular boundaries of two pixel categories comprising the non-hot spot pixels and the hot spot pixels are output, a Mask image (Mask) is taken as output, a region where the hot spot is located is white, and the rest regions are black. And finally, outputting a target image of the region where the hot spot is marked according to the topological relation (Coordinates) between the first position information of the region where the hot spot is output by the full-connection layer and the second position information of the target boundary of the photovoltaic panel, the Category (Category) of the hot spot and the Mask image (Mask) output by the coding classification module.

After the bbox reg operation is executed by the full connection layer, the two-dimensional distribution information of the actual frame is calculated by the fast-RCNN model to measure the confidence coefficient of the prediction labeling frame, the opposite confidence coefficient threshold reaches 0.6, the distance between the prediction frame and the two-dimensional Gaussian distribution of the actual frame can be calculated, then the label is distributed for the small target hot spot by the confidence coefficient score, and otherwise, the label is predicted again.

Referring to fig. 4, fig. 4 is a flowchart illustrating a method for marking defects of a photovoltaic panel according to an exemplary embodiment, and as shown in fig. 4, a device 300 for marking defects of a photovoltaic panel includes:

an acquisition module 310 configured to acquire an image to be annotated, the image to be annotated being an infrared image containing hot spots;

an obtaining module 320 configured to input an image to be annotated to the trained fast-RCNN model to obtain a target image;

In some embodiments, the fully connected layer of the fast-RCNN model includes:

the information classification module is used for acquiring first position information of an area where the hot spot is located and second position information of a target boundary of the photovoltaic panel in the characteristic image, determining a topological relation between the first position information and the second position information, wherein the topological relation is used for determining a target position of a marking frame to be detected in the image to be marked so as to mark the area where the hot spot is located, and obtaining a target image;

The specific manner in which the respective modules perform the operations of the defect labeling apparatus 300 for a photovoltaic panel in the above-described embodiment has been described in detail in the embodiment regarding the defect labeling method for a photovoltaic panel, and will not be described in detail herein.

Based on the same inventive concept, the present disclosure provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the above-described defect labeling method of a photovoltaic panel.

Based on the same inventive concept, the present disclosure provides an electronic apparatus comprising:

a memory having a computer program stored thereon;

and the processor is used for executing the computer program in the memory to realize the defect marking method of the photovoltaic panel.

Fig. 5 is a block diagram of an electronic device 400, shown in accordance with an exemplary embodiment. As shown in fig. 5, the electronic device 400 may include: a processor 401, a memory 402. The electronic device 400 may also include one or more of a multimedia component 403, an input/output (I/O) interface 404, and a communication component 405.

The processor 401 is configured to control the overall operation of the electronic device 400 to perform all or part of the steps in the method for marking defects of a photovoltaic panel. The memory 402 is used to store various types of data to support operation at the electronic device 400, which may include, for example, instructions for any application or method operating on the electronic device 400, as well as application-related data, such as contact data, transceived messages, pictures, audio, video, and the like. The Memory 402 may be implemented by any type or combination of volatile or non-volatile Memory devices, such as static random access Memory (Static Random Access Memory, SRAM for short), electrically erasable programmable Read-Only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), erasable programmable Read-Only Memory (Erasable Programmable Read-Only Memory, EPROM for short), programmable Read-Only Memory (Programmable Read-Only Memory, PROM for short), read-Only Memory (ROM for short), magnetic Memory, flash Memory, magnetic disk, or optical disk. The multimedia component 403 may include a screen and an audio component. Wherein the screen may be, for example, a touch screen, the audio component being for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 402 or transmitted through the communication component 405. The audio assembly further comprises at least one speaker for outputting audio signals. The I/O interface 404 provides an interface between the processor 401 and other interface modules, which may be a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 405 is used for wired or wireless communication between the electronic device 400 and other devices. Wireless communication, such as Wi-Fi, bluetooth, near field communication (Near Field Communication, NFC for short), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or one or a combination of more of them, is not limited herein. The corresponding communication component 405 may thus comprise: wi-Fi module, bluetooth module, NFC module, etc.

In an exemplary embodiment, the electronic device 400 may be implemented by one or more application specific integrated circuits (Application Specific Integrated Circuit, abbreviated as ASIC), digital signal processor (Digital Signal Processor, abbreviated as DSP), digital signal processing device (Digital Signal Processing Device, abbreviated as DSPD), programmable logic device (Programmable Logic Device, abbreviated as PLD), field programmable gate array (Field Programmable Gate Array, abbreviated as FPGA), controller, microcontroller, microprocessor, or other electronic component for performing the above-described method for defect labeling of photovoltaic panels.

In another exemplary embodiment, a computer readable storage medium is also provided that includes program instructions that, when executed by a processor, implement the above-described method for marking defects of a photovoltaic panel. For example, the computer readable storage medium may be the memory 402 including program instructions described above, which are executable by the processor 401 of the electronic device 400 to perform the method of marking defects of a photovoltaic panel described above.

In another exemplary embodiment, a computer program product is also provided, which comprises a computer program executable by a programmable apparatus, the computer program having code portions for performing the above-described defect marking method of a photovoltaic panel when executed by the programmable apparatus.

The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solutions of the present disclosure within the scope of the technical concept of the present disclosure, and all the simple modifications belong to the protection scope of the present disclosure.

In addition, the specific features described in the above embodiments may be combined in any suitable manner without contradiction. The various possible combinations are not described further in this disclosure in order to avoid unnecessary repetition.

Moreover, any combination between the various embodiments of the present disclosure is possible as long as it does not depart from the spirit of the present disclosure, which should also be construed as the disclosure of the present disclosure.

Claims

1. A method for marking defects of a photovoltaic panel, comprising:

2. The method of claim 1, wherein the residual network employs a res net18 network, wherein the target convolutional layer comprises a plurality of layers, and wherein the plurality of layers of the target convolutional layer are sequentially connected to the full connection layer.

3. The method according to claim 1 or 2, wherein the regional generation network of the fast-RCNN model adopts an FPN network, and the FPN network is used for extracting detailed features of the feature image output by the residual network, and generating a to-be-detected labeling frame for labeling a target region, where the target region includes a region where the hot spot is located.

4. A method according to claim 3, wherein the FPN network comprises a 3x3 convolution layer and two 1 x 1 branch convolution layers connected to the 3x3 convolution layer, the 3x3 convolution layer is used for extracting detailed features of the feature image and generating the to-be-detected labeling frame, one of the 1 x 1 branch convolution layers is used for determining whether the to-be-detected labeling frame contains hot spots, and the other 1 x 1 branch convolution layer is used for positioning the position of the to-be-detected labeling frame and adjusting the size of the to-be-detected labeling frame.

5. The method of claim 4, wherein the 3x3 convolution layers are convolved in a spread convolution, and wherein the 3x3 convolution layers comprise 3 convolution layers, and wherein the spread ratios of the 3 convolution layers are 2, 4, and 6, respectively.

6. The method of claim 3, wherein the fast-RCNN model includes a coding module connected to an output of an ROI Pooling layer of the fast-RCNN model, the ROI Pooling layer integrating the feature image and the annotation frame to be measured, outputting a plurality of feature images with the annotation frame to be measured having a confidence greater than a preset confidence, the coding module comprising:

7. The method of claim 3, wherein the fully connected layer of the fast-RCNN model comprises:

8. A defect marking device for a photovoltaic panel, comprising:

9. A non-transitory computer readable storage medium having stored thereon a computer program, characterized in that the program, when executed by a processor, implements the method of any of claims 1-7.

10. An electronic device, comprising:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement the method of any of claims 1-7.