WO2021082919A1 - Defect detecting method and equipment for screen region of electronic equipment - Google Patents

Defect detecting method and equipment for screen region of electronic equipment Download PDF

Info

Publication number
WO2021082919A1
WO2021082919A1 PCT/CN2020/120875 CN2020120875W WO2021082919A1 WO 2021082919 A1 WO2021082919 A1 WO 2021082919A1 CN 2020120875 W CN2020120875 W CN 2020120875W WO 2021082919 A1 WO2021082919 A1 WO 2021082919A1
Authority
WO
WIPO (PCT)
Prior art keywords
screen area
electronic device
defect
model
backbone network
Prior art date
Application number
PCT/CN2020/120875
Other languages
French (fr)
Chinese (zh)
Inventor
徐鹏
沈圣远
常树林
姚巨虎
Original Assignee
上海悦易网络信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海悦易网络信息技术有限公司 filed Critical 上海悦易网络信息技术有限公司
Publication of WO2021082919A1 publication Critical patent/WO2021082919A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30121—CRT, LCD or plasma display

Definitions

  • This application relates to the field of computer technology, and in particular to a method and equipment for detecting defects in the screen area of an electronic device.
  • the traditional image processing method is based on the selection of the threshold to a large extent, and the screen area of second-hand electronic equipment such as mobile phones has different degrees of difference in various aspects such as color, appearance, aging, etc., it is difficult to give The determined threshold is therefore not applicable to the detection of defects in this screen area based on traditional image processing methods.
  • the purpose of this application is to provide a method and device for detecting defects in the screen area of an electronic device.
  • a method for detecting defects in a screen area of an electronic device including:
  • the defect detection result of the screen area image of the electronic device is received from the FPN network combined with the backbone network model.
  • the defect detection result includes: the defect type of the screen area of the electronic device and the position of the defect in the screen area of the electronic device And the confidence level of the defect detection result.
  • the first two layers of the backbone network adopt a res structure
  • the last two layers of the network adopt an inception structure
  • the method further includes:
  • output result information including the defect type of the screen area of the electronic device and the position of the defect in the screen area of the electronic device.
  • the method further includes:
  • Step one preset the FPN network combined with the backbone network model and its initial model parameters
  • Step 2 Input the screen area image of the sample electronic device into the FPN network with current model parameters combined with the backbone network model to obtain the defect prediction result of the screen area of the sample electronic device.
  • the defect prediction result includes: The type of defects in the screen area, the position of the defects in the screen area of the sample electronic device, and the confidence level of the defect detection results;
  • Step 3 Calculate the difference between the defect prediction result and the actual defect result of the screen area of the sample electronic device based on a preset objective function, and identify whether the difference is greater than a second preset threshold,
  • step 4 after updating the model parameters of the FPN network combined with the backbone network based on the difference, restart execution from step 2;
  • step 5 the FPN network combined with the backbone network model with the current model parameters is used as the model of the FPN network combined with the backbone network after the training.
  • the normalized parameters are updated iteratively after each convolution calculation is completed.
  • an electronic device screen area defect detection device comprising:
  • the first device is used to input the screen area image of the electronic device into the model of the FPN network combined with the backbone network after the training;
  • the second device is used to receive and output the defect detection result of the screen area image of the electronic device from the model of the FPN network combined with the backbone network.
  • the defect detection result includes: the defect type of the screen area of the electronic device and the defect in the electronic device The position in the screen area and the confidence level of the defect detection result.
  • the first two layers of the backbone network adopt a res structure
  • the last two layers of the network adopt an inception structure
  • the second device is also used to identify whether the confidence level of the defect detection result is greater than a first preset threshold, and if it is greater than the first preset threshold, output the defect category including the screen area of the electronic device, Result information on the location of the defect in the screen area of the electronic device.
  • a third device including:
  • the third device is used to preset the FPN network combined with the backbone network model and its initial model parameters
  • the third and second device is used to input the screen area image of the sample electronic device into the FPN network with current model parameters combined with the backbone network model to obtain the defect prediction result of the screen area of the sample electronic device, and the defect prediction result includes: The type of defect in the screen area of the sample electronic device, the position of the defect in the screen area of the sample electronic device, and the confidence level of the defect detection result;
  • the third and third device is used to calculate the difference between the defect prediction result and the real defect result of the sample electronic device based on a preset objective function, and to identify whether the difference is greater than a second preset threshold, and if the difference is If it is greater than the second preset threshold, execute the third and fourth device for updating the model parameters of the FPN network combined with the backbone network based on the difference, and then restart execution from the third and second device;
  • the third device is executed to use the FPN network combined with the backbone network model with the current model parameters as the model of the FPN network combined with the backbone network after the training.
  • the normalized parameters are updated iteratively after each convolution calculation is completed.
  • Computer-readable instructions are stored thereon, and computer-executable instructions are stored thereon.
  • the processor causes the processor:
  • Step S1 input the screen area image of the electronic device into the model of the FPN network combined with the backbone network after the training;
  • Step S2 the defect detection result of the screen area image of the electronic device is received from the model of the FPN network combined with the backbone network.
  • the defect detection result includes: the defect type of the screen area of the electronic device and the defect in the screen area of the electronic device The position and the confidence level of the defect detection result.
  • the present invention also provides a device for detecting defects in the screen area of an electronic device, which includes:
  • a memory arranged to store computer-executable instructions which, when executed, cause the processor to:
  • Step S1 input the screen area image of the electronic device into the model of the FPN network combined with the backbone network after the training;
  • Step S2 the defect detection result of the screen area image of the electronic device is received from the model of the FPN network combined with the backbone network.
  • the defect detection result includes: the defect type of the screen area of the electronic device and the defect in the screen area of the electronic device The position and the confidence level of the defect detection result.
  • the present invention inputs the image of the screen area of the electronic device into the model of the FPN network combined with the backbone network after the training; and receives the output of the screen area image of the electronic device from the model of the FPN network combined with the backbone network.
  • the defect detection result includes: the type of defect in the screen area of the electronic device, the position of the defect in the screen area of the electronic device, and the confidence level of the defect detection result, which can accurately identify the screen area of a second-hand electronic device such as a mobile phone The difference in blemishes.
  • FIG. 1 shows a flowchart of a method for detecting defects in a screen area according to an embodiment of the present invention
  • FIG. 2 shows a schematic diagram of a screen area defect detection result according to an embodiment of the present invention
  • FIG. 3 shows a schematic diagram of a model of an FPN network combined with a backbone network according to an embodiment of the present invention.
  • each module and trusted party of the system includes one or more processors (CPU), input/output interfaces, network interfaces, and memory.
  • processors CPU
  • input/output interfaces network interfaces
  • memory volatile and non-volatile memory
  • Memory may include non-permanent memory in computer-readable media, random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM), programmable read-only memory (PROM), Erase programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory (flash RAM).
  • RAM random access memory
  • ROM read-only memory
  • PROM programmable read-only memory
  • EPROM Erase programmable read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • flash RAM flash RAM
  • Memory is an example of computer readable media.
  • Computer-readable media includes permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology.
  • the information can be computer-readable instructions, data structures, program modules, or other data.
  • Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, Magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices.
  • computer-readable media does not include non-transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.
  • Fig. 1 shows a method for detecting defects in the screen area of an electronic device provided by one aspect of the present application, wherein the method includes:
  • S11 inputs the screen area image of the electronic device into the model of the FPN network after the training and the backbone network;
  • the defect detection result includes: the defect type of the screen area of the electronic device, and the defect in the screen area of the electronic device Confidence of location and defect detection results.
  • the types of defects include, but are not limited to, delamination, penetrating, leaking, broken lines, bright spots (bright spots), and stains (yellow and blue).
  • the model of the FPN network combined with the backbone network can be shown in FIG. 3.
  • the FPN network combined with the backbone network model iteratively updates the normalized parameters after each convolution calculation is completed, where the normalized parameters include the mean and variance, and normalization ensures that each input is reasonable Changes within the range, here, the normalized parameters are continuously updated with the update of the input data.
  • each defect detection result includes cls, x1, y1, x2, y2, score, Among them, cls is the defect type, x1, y1, x2, y2 are the 4 coordinates of the position of the defect in the image of the screen area area, and score is the confidence level of this defect.
  • the present invention mainly utilizes the improved feature pyramid (FPN) network combined with the deep learning model of the backbone network, and can accurately identify the difference in the defects of the screen area of the second-hand electronic equipment such as the mobile phone.
  • FPN improved feature pyramid
  • the first two layers of the backbone network adopt a res structure
  • the last two layers of the network adopt an inception structure
  • step S12 after receiving the output defect detection result of the screen area of the electronic device from the FPN network combined with the backbone network model, further includes:
  • the method before inputting the screen area image into the FPN network combined with the backbone network model, the method further includes:
  • Step one preset the FPN network combined with the backbone network model and its initial model parameters
  • Step 2 Input the screen area image of the sample electronic device into the FPN network with the current model parameters combined with the backbone network model to obtain the defect prediction result of the screen of the sample electronic device, and the defect prediction result includes: the screen of the sample electronic device The type of defect in the area, the position of the defect in the screen area of the sample electronic device, and the confidence level of the defect detection result;
  • Step 3 Calculate the difference between the defect prediction result and the real defect result of the sample electronic device based on a preset objective function, and identify whether the difference is greater than a second preset threshold,
  • step 4 after updating the model parameters of the FPN network combined with the backbone network based on the difference, restart execution from step 2;
  • step 5 the FPN network combined with the backbone network model with the current model parameters is used as the model of the FPN network combined with the backbone network after the training.
  • the model of the FPN network combined with the backbone network is cyclically trained to obtain a reliable model.
  • the present invention also provides a device for detecting defects in the screen area of an electronic device, the device comprising:
  • the first device is used to input the screen area image of the electronic device into the model of the FPN network combined with the backbone network after the training;
  • the second device is used to receive and output the defect detection result of the screen area image of the electronic device from the model of the FPN network combined with the backbone network.
  • the defect detection result includes: the defect type of the screen area of the electronic device and the defect in the electronic device The position in the screen area and the confidence level of the defect detection result.
  • the types of defects include, but are not limited to, delamination, penetrating characters, leakage, broken lines, bright spots (bright spots), and stains (yellow and blue).
  • each defect detection result of the screen area of the electronic device received and output from the FPN network combined with the backbone network model
  • each defect detection result includes cls, x1, y1, x2, y2, score, where cls is the defect type , X1, y1, x2, y2 are the 4 coordinates of the position of the defect in the image of the screen area area, and score is the confidence level of this defect.
  • the FPN network combined with the backbone network model iteratively updates the normalized parameters after each convolution calculation is completed, where the normalized parameters include the mean and variance, and normalization ensures that each input is reasonable Changes within the range, here, the normalized parameters are continuously updated with the update of the input data.
  • the present invention mainly utilizes the improved feature pyramid (FPN) network combined with the deep learning model of the backbone network, which can accurately identify the difference in the screen area of second-hand electronic devices such as mobile phones.
  • FPN improved feature pyramid
  • the first two layers of the backbone network adopt a res structure
  • the last two layers of the network adopt an inception structure
  • the second device is also used to identify whether the confidence level of the defect detection result is greater than a first preset threshold, and if it is greater than the first preset
  • the threshold value outputs result information including the defect type of the screen area of the electronic device and the position of the defect on the screen of the electronic device.
  • the screen area defect detection device of the electronic device of the present invention further includes a third device, including:
  • the third device is used to preset the FPN network combined with the backbone network model and its initial model parameters
  • the third and second device is used to input the screen area image of the sample electronic device into the FPN network combined with the backbone network model with the current model parameters to obtain the defect prediction result of the screen area of the sample electronic device, and the defect prediction result includes: The type of defects in the screen area of the sample electronic device, the position of the defect in the screen area of the sample electronic device, and the confidence level of the defect detection result;
  • the third and third device is used to calculate the difference between the defect prediction result and the real defect result of the sample electronic device based on a preset objective function, and to identify whether the difference is greater than a second preset threshold, and if the difference is If it is greater than the second preset threshold, execute the third and fourth device for updating the model parameters of the FPN network combined with the backbone network based on the difference, and then restart execution from the third and second device;
  • the third device is executed to use the FPN network combined with the backbone network model with the current model parameters as the model of the FPN network combined with the backbone network after the training.
  • the model of the FPN network combined with the backbone network is cyclically trained to obtain a reliable model.
  • Computer-readable instructions are stored thereon, and computer-executable instructions are stored thereon.
  • the processor causes the processor:
  • Step S1 input the screen area image of the electronic device into the model of the FPN network combined with the backbone network after the training;
  • Step S2 the defect detection result of the screen area image of the electronic device is received from the model of the FPN network combined with the backbone network.
  • the defect detection result includes: the defect type of the screen area of the electronic device and the defect in the screen area of the electronic device The position and the confidence level of the defect detection result.
  • the present invention also provides a device for detecting defects in the screen area of an electronic device, which includes:
  • a memory arranged to store computer-executable instructions which, when executed, cause the processor to:
  • Step S1 input the screen area image of the electronic device into the model of the FPN network combined with the backbone network after the training;
  • Step S2 the defect detection result of the screen area image of the electronic device is received from the model of the FPN network combined with the backbone network.
  • the defect detection result includes: the defect type of the screen area of the electronic device and the defect in the screen area of the electronic device The position and the confidence level of the defect detection result.
  • the present invention inputs the image of the screen area of the electronic device into the model of the FPN network combined with the backbone network after the training; and receives the output of the screen area image of the electronic device from the model of the FPN network combined with the backbone network.
  • the defect detection result includes: the type of defect in the screen area of the electronic device, the position of the defect in the screen area of the electronic device, and the confidence level of the defect detection result, which can accurately identify the screen area of a second-hand electronic device such as a mobile phone The difference in blemishes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

A defect detecting method and equipment for the screen region of an electronic equipment. The method comprises: inputting a screen region image of the electronic equipment into a FPN and backbone network combined model on which training is finished (S11); and receiving, from the FPN and backbone network combined model, an output defect detecting result of the screen region image of the electronic equipment, the defect detecting result including: a defect type of the screen region of the electronic equipment, the position of the defect on the screen region of the electronic equipment and the confidence of the defect detecting result (S12). The described method can accurately identify defect differences on the screen region of a second-hand electronic equipment, for example, a mobile phone.

Description

一种电子设备屏幕区域瑕疵检测方法与设备Method and equipment for detecting defects in screen area of electronic equipment 技术领域Technical field
本申请涉及计算机技术领域，尤其涉及一种电子设备屏幕区域瑕疵检测方法与设备。This application relates to the field of computer technology, and in particular to a method and equipment for detecting defects in the screen area of an electronic device.
背景技术Background technique
由于基于传统图像处理方式在很大程度上依赖于阈值的选取，而二手电子设备如手机等的屏幕区域由于在成色、外观、老化程度等各个方面都有不同程度的差异，故很难给出确定的阈值，因此基于传统图像处理方式的在本屏幕区域瑕疵检测中不适用。Because the traditional image processing method is based on the selection of the threshold to a large extent, and the screen area of second-hand electronic equipment such as mobile phones has different degrees of difference in various aspects such as color, appearance, aging, etc., it is difficult to give The determined threshold is therefore not applicable to the detection of defects in this screen area based on traditional image processing methods.
发明内容Summary of the invention
本申请的目的是提供一种电子设备屏幕区域瑕疵检测方法与设备。The purpose of this application is to provide a method and device for detecting defects in the screen area of an electronic device.
根据本申请的一个方面，提供了一种电子设备屏幕区域瑕疵检测方法，所述方法包括：According to an aspect of the present application, there is provided a method for detecting defects in a screen area of an electronic device, the method including:
将电子设备的屏幕区域图像输入训练结束后的FPN网络结合backbone网络的模型；Input the screen area image of the electronic device into the model of the FPN network after the training and the backbone network;
从所述FPN网络结合backbone网络的模型接收输出的电子设备的屏幕区域图像的瑕疵检测结果，所述瑕疵检测结果包括：电子设备的屏幕区域的瑕疵种类、瑕疵在电子设备的屏幕区域中的位置和瑕疵检测结果的置信度。The defect detection result of the screen area image of the electronic device is received from the FPN network combined with the backbone network model. The defect detection result includes: the defect type of the screen area of the electronic device and the position of the defect in the screen area of the electronic device And the confidence level of the defect detection result.
进一步地，其中，所述backbone网络的前2层采用res结构，网络的后2层采用inception结构。Further, wherein the first two layers of the backbone network adopt a res structure, and the last two layers of the network adopt an inception structure.
进一步地，其中，从所述FPN网络结合backbone网络的模型接收输出的电子设备的屏幕区域的瑕疵检测结果之后，还包括：Further, after receiving the output of the defect detection result of the screen area of the electronic device from the model of the FPN network combined with the backbone network, the method further includes:
识别所述瑕疵检测结果的置信度是否大于第一预设阈值，Identifying whether the confidence level of the defect detection result is greater than a first preset threshold,
若大于所述第一预设阈值，则输出包括电子设备的屏幕区域的瑕疵种类、瑕疵在电子设备的屏幕区域中的位置的结果信息。If it is greater than the first preset threshold, output result information including the defect type of the screen area of the electronic device and the position of the defect in the screen area of the electronic device.
进一步地，其中，将所述电子设备的屏幕区域图像输入FPN网络结合backbone网络的模型之前，还包括：Further, before inputting the screen area image of the electronic device into the FPN network combined with the backbone network model, the method further includes:
步骤一，预设FPN网络结合backbone网络的模型及其初始的模型参数；Step one, preset the FPN network combined with the backbone network model and its initial model parameters;
步骤二，将样本电子设备的屏幕区域图像输入带有当前的模型参数的FPN网络结合backbone网络的模型，得到样本电子设备的屏幕区域的瑕疵预测结果，所述瑕疵预测结果包括：样本电子设备的屏幕区域的瑕疵种类、瑕疵在样本电子设备的屏幕区域中的位置和瑕疵检测结果的置信度；Step 2: Input the screen area image of the sample electronic device into the FPN network with current model parameters combined with the backbone network model to obtain the defect prediction result of the screen area of the sample electronic device. The defect prediction result includes: The type of defects in the screen area, the position of the defects in the screen area of the sample electronic device, and the confidence level of the defect detection results;
步骤三，基于预设目标函数计算所述瑕疵预测结果与样本电子设备的屏幕区域的真实瑕疵结果之间的差值，识别所述差值是否大于第二预设阈，Step 3: Calculate the difference between the defect prediction result and the actual defect result of the screen area of the sample electronic device based on a preset objective function, and identify whether the difference is greater than a second preset threshold,
若所述差值大于第二预设阈值，则步骤四，基于所述差值更新所述FPN网络结合backbone网络的模型参数后，重新从步骤二开始执行；If the difference is greater than the second preset threshold, step 4, after updating the model parameters of the FPN network combined with the backbone network based on the difference, restart execution from step 2;
若所述差值小于等于第二预设阈值，则步骤五，将带有当前的模型参数的FPN网络结合backbone网络的模型作为训练结束后的FPN网络结合backbone网络的模型。If the difference is less than or equal to the second preset threshold, step 5, the FPN network combined with the backbone network model with the current model parameters is used as the model of the FPN network combined with the backbone network after the training.
进一步地，其中，所述FPN网络结合backbone网络的模型中每一次卷积计算完成后迭代更新归一化的参数。Further, in the model of the FPN network combined with the backbone network, the normalized parameters are updated iteratively after each convolution calculation is completed.
根据本申请的另一方面，还提供了一种电子设备屏幕区域瑕疵检测设备，所述设备包括：According to another aspect of the present application, there is also provided an electronic device screen area defect detection device, the device comprising:
第一装置，用于将电子设备的屏幕区域图像输入训练结束后的FPN网络结合backbone网络的模型；The first device is used to input the screen area image of the electronic device into the model of the FPN network combined with the backbone network after the training;
第二装置，用于从所述FPN网络结合backbone网络的模型接收输出的电子设备的屏幕区域图像的瑕疵检测结果，所述瑕疵检测结果包括：电子设备的屏幕区域的瑕疵种类、瑕疵在电子设备的屏幕区域中的位置和瑕疵检测结果的置信度。The second device is used to receive and output the defect detection result of the screen area image of the electronic device from the model of the FPN network combined with the backbone network. The defect detection result includes: the defect type of the screen area of the electronic device and the defect in the electronic device The position in the screen area and the confidence level of the defect detection result.
进一步地，其中，所述backbone网络的前2层采用res结构，网络的后2层采用inception结构。Further, wherein the first two layers of the backbone network adopt a res structure, and the last two layers of the network adopt an inception structure.
进一步地，其中，第二装置还用于识别所述瑕疵检测结果的置信度是否大于第一预设阈值，若大于所述第一预设阈值，则输出包括电子设备的屏幕区域的瑕疵种类、瑕疵在电子设备的屏幕区域中的位置的结果信息。Further, wherein the second device is also used to identify whether the confidence level of the defect detection result is greater than a first preset threshold, and if it is greater than the first preset threshold, output the defect category including the screen area of the electronic device, Result information on the location of the defect in the screen area of the electronic device.
进一步地，其中，还包括第三装置，包括：Further, it also includes a third device, including:
第三一装置，用于预设FPN网络结合backbone网络的模型及其初始的模 型参数；The third device is used to preset the FPN network combined with the backbone network model and its initial model parameters;
第三二装置，用于将样本电子设备的屏幕区域图像输入带有当前的模型参数的FPN网络结合backbone网络的模型，得到样本电子设备的屏幕区域的瑕疵预测结果，所述瑕疵预测结果包括：样本电子设备的屏幕区域的瑕疵种类、瑕疵在样本电子设备的屏幕区域中的位置和瑕疵检测结果的置信度；The third and second device is used to input the screen area image of the sample electronic device into the FPN network with current model parameters combined with the backbone network model to obtain the defect prediction result of the screen area of the sample electronic device, and the defect prediction result includes: The type of defect in the screen area of the sample electronic device, the position of the defect in the screen area of the sample electronic device, and the confidence level of the defect detection result;
第三三装置，用于基于预设目标函数计算所述瑕疵预测结果与样本电子设备的真实瑕疵结果之间的差值，识别所述差值是否大于第二预设阈，若所述差值大于第二预设阈值，则执行第三四装置，用于基于所述差值更新所述FPN网络结合backbone网络的模型参数后，重新从第三二装置开始执行；The third and third device is used to calculate the difference between the defect prediction result and the real defect result of the sample electronic device based on a preset objective function, and to identify whether the difference is greater than a second preset threshold, and if the difference is If it is greater than the second preset threshold, execute the third and fourth device for updating the model parameters of the FPN network combined with the backbone network based on the difference, and then restart execution from the third and second device;
若所述差值小于等于第二预设阈值，则执行第三三装置，将带有当前的模型参数的FPN网络结合backbone网络的模型作为训练结束后的FPN网络结合backbone网络的模型。If the difference is less than or equal to the second preset threshold, the third device is executed to use the FPN network combined with the backbone network model with the current model parameters as the model of the FPN network combined with the backbone network after the training.
进一步地，其中，所述FPN网络结合backbone网络的模型中每一次卷积计算完成后迭代更新归一化的参数。Further, in the model of the FPN network combined with the backbone network, the normalized parameters are updated iteratively after each convolution calculation is completed.
根据本申请的再一方面，还提供了一种计算机可读介质，其中，According to another aspect of the present application, there is also provided a computer-readable medium, wherein:
其上存储有计算机可读指令，其上存储有计算机可执行指令，其中，该计算机可执行指令被处理器执行时使得该处理器：Computer-readable instructions are stored thereon, and computer-executable instructions are stored thereon. When the computer-executable instructions are executed by a processor, the processor causes the processor:
步骤S1，将电子设备的屏幕区域图像输入训练结束后的FPN网络结合backbone网络的模型；Step S1, input the screen area image of the electronic device into the model of the FPN network combined with the backbone network after the training;
步骤S2，从所述FPN网络结合backbone网络的模型接收输出的电子设备的屏幕区域图像的瑕疵检测结果，所述瑕疵检测结果包括：电子设备的屏幕区域的瑕疵种类、瑕疵在电子设备的屏幕区域中的位置和瑕疵检测结果的置信度。Step S2, the defect detection result of the screen area image of the electronic device is received from the model of the FPN network combined with the backbone network. The defect detection result includes: the defect type of the screen area of the electronic device and the defect in the screen area of the electronic device The position and the confidence level of the defect detection result.
本发明还提供一种电子设备屏幕区域瑕疵检测设备，其中，包括：The present invention also provides a device for detecting defects in the screen area of an electronic device, which includes:
处理器；以及Processor; and
被安排成存储计算机可执行指令的存储器，所述可执行指令在被执行时使所述处理器：A memory arranged to store computer-executable instructions which, when executed, cause the processor to:
步骤S1，将电子设备的屏幕区域图像输入训练结束后的FPN网络结合backbone网络的模型；Step S1, input the screen area image of the electronic device into the model of the FPN network combined with the backbone network after the training;
步骤S2，从所述FPN网络结合backbone网络的模型接收输出的电子设备的屏幕区域图像的瑕疵检测结果，所述瑕疵检测结果包括：电子设备的屏幕区域的瑕疵种类、瑕疵在电子设备的屏幕区域中的位置和瑕疵检测结果的置信度。Step S2, the defect detection result of the screen area image of the electronic device is received from the model of the FPN network combined with the backbone network. The defect detection result includes: the defect type of the screen area of the electronic device and the defect in the screen area of the electronic device The position and the confidence level of the defect detection result.
与现有技术相比，本发明通过将电子设备的屏幕区域图像输入训练结束后的FPN网络结合backbone网络的模型；从所述FPN网络结合backbone网络的模型接收输出的电子设备的屏幕区域图像的瑕疵检测结果，所述瑕疵检测结果包括：电子设备的屏幕区域的瑕疵种类、瑕疵在电子设备的屏幕区域中的位置和瑕疵检测结果的置信度，能够准确地识别二手电子设备如手机的屏幕区域的瑕疵差异。Compared with the prior art, the present invention inputs the image of the screen area of the electronic device into the model of the FPN network combined with the backbone network after the training; and receives the output of the screen area image of the electronic device from the model of the FPN network combined with the backbone network. Defect detection result, the defect detection result includes: the type of defect in the screen area of the electronic device, the position of the defect in the screen area of the electronic device, and the confidence level of the defect detection result, which can accurately identify the screen area of a second-hand electronic device such as a mobile phone The difference in blemishes.
附图说明Description of the drawings
通过阅读参照以下附图所作的对非限制性实施例所作的详细描述，本发明的其它特征、目的和优点将会变得更明显：By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, purposes and advantages of the present invention will become more apparent:
图1示出本发明一实施例的屏幕区域瑕疵检测方法的流程图；FIG. 1 shows a flowchart of a method for detecting defects in a screen area according to an embodiment of the present invention;
图2示出本发明一实施例的屏幕区域瑕疵检测结果的示意图；FIG. 2 shows a schematic diagram of a screen area defect detection result according to an embodiment of the present invention;
图3示出本发明一实施例的FPN网络结合backbone网络的模型的示意图。FIG. 3 shows a schematic diagram of a model of an FPN network combined with a backbone network according to an embodiment of the present invention.
附图中相同或相似的附图标记代表相同或相似的部件。The same or similar reference signs in the drawings represent the same or similar components.
具体实施方式Detailed ways
下面结合附图对本发明作进一步详细描述。The present invention will be described in further detail below in conjunction with the accompanying drawings.
在本申请一个典型的配置中，系统各模块和可信方均包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。In a typical configuration of this application, each module and trusted party of the system includes one or more processors (CPU), input/output interfaces, network interfaces, and memory.
内存可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RAM)和/或非易失性内存等形式，如只读存储器(ROM)、可编程只读存储器(PROM)、可擦除可编程只读存储器(EPROM)、电可擦除可编程只读存储器(EEPROM)或闪存(flash RAM)。内存是计算机可读介质的示例。Memory may include non-permanent memory in computer-readable media, random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM), programmable read-only memory (PROM), Erase programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory (flash RAM). Memory is an example of computer readable media.
计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以 由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。按照本文中的界定，计算机可读介质不包括非暂存电脑可读媒体(transitory media)，如调制的数据信号和载波。Computer-readable media includes permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, Magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices. According to the definition in this article, computer-readable media does not include non-transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.
为更进一步阐述本申请所采取的技术手段及取得的效果，下面结合附图及优选实施例，对本申请的技术方案，进行清楚和完整的描述。In order to further illustrate the technical means adopted by this application and the effects achieved, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings and preferred embodiments.
图1示出本申请一个方面提供的一种用于电子设备屏幕区域瑕疵检测方法，其中，该方法包括：Fig. 1 shows a method for detecting defects in the screen area of an electronic device provided by one aspect of the present application, wherein the method includes:
S11将电子设备的屏幕区域图像输入训练结束后的FPN网络结合backbone网络的模型；S11 inputs the screen area image of the electronic device into the model of the FPN network after the training and the backbone network;
S12从所述FPN网络结合backbone网络的模型接收输出的电子设备的屏幕区域图像的瑕疵检测结果，所述瑕疵检测结果包括：电子设备的屏幕区域的瑕疵种类、瑕疵在电子设备的屏幕区域中的位置和瑕疵检测结果的置信度。其中，瑕疵种类包括但不限于分层、透字、漏液、断线、亮点(亮斑)、色斑(发黄发青)。S12 receives the defect detection result of the screen area image of the electronic device from the model of the FPN network combined with the backbone network. The defect detection result includes: the defect type of the screen area of the electronic device, and the defect in the screen area of the electronic device Confidence of location and defect detection results. Among them, the types of defects include, but are not limited to, delamination, penetrating, leaking, broken lines, bright spots (bright spots), and stains (yellow and blue).
所述FPN网络结合backbone网络的模型可如图3所示。其中，所述FPN网络结合backbone网络的模型中每一次卷积计算完成后迭代更新归一化的参数，其中，归一化参数包括均值和方差，通过归一化确保每次输入都在合理的范围内变化，在此，所述归一化的参数随着输入数据的更新不断更新。The model of the FPN network combined with the backbone network can be shown in FIG. 3. Wherein, the FPN network combined with the backbone network model iteratively updates the normalized parameters after each convolution calculation is completed, where the normalized parameters include the mean and variance, and normalization ensures that each input is reasonable Changes within the range, here, the normalized parameters are continuously updated with the update of the input data.
在此，从所述FPN网络结合backbone网络的模型接收输出的电子设备的屏幕区域的瑕疵检测结果，如图2所示，每个瑕疵检测结果包含cls，x1，y1，x2，y2，score，其中，cls是缺陷类型，x1，y1，x2，y2是屏幕 区域区域图像中瑕疵所在位置的4个坐标，score为这个瑕疵的置信度。Here, the defect detection results of the screen area of the electronic device received and output from the FPN network combined with the backbone network model, as shown in Figure 2, each defect detection result includes cls, x1, y1, x2, y2, score, Among them, cls is the defect type, x1, y1, x2, y2 are the 4 coordinates of the position of the defect in the image of the screen area area, and score is the confidence level of this defect.
本发明主要利用改进的特征金字塔(FPN)网络结合backbone网络的深度学习模型，能够准确地识别二手电子设备如手机的屏幕区域的瑕疵差异。The present invention mainly utilizes the improved feature pyramid (FPN) network combined with the deep learning model of the backbone network, and can accurately identify the difference in the defects of the screen area of the second-hand electronic equipment such as the mobile phone.
本发明的屏幕区域瑕疵检测方法一实施例中，所述backbone网络的前2层采用res结构，网络的后2层采用inception结构。In an embodiment of the screen area defect detection method of the present invention, the first two layers of the backbone network adopt a res structure, and the last two layers of the network adopt an inception structure.
本发明的屏幕区域瑕疵检测方法一实施例中，步骤S12，从所述FPN网络结合backbone网络的模型接收输出的电子设备的屏幕区域的瑕疵检测结果之后，还包括：In an embodiment of the screen area defect detection method of the present invention, step S12, after receiving the output defect detection result of the screen area of the electronic device from the FPN network combined with the backbone network model, further includes:
识别所述瑕疵检测结果的置信度是否大于第一预设阈值，Identifying whether the confidence level of the defect detection result is greater than a first preset threshold,
若大于所述第一预设阈值，则输出包括电子设备的屏幕的瑕疵种类、瑕疵在电子设备的屏幕中的位置的结果信息。If it is greater than the first preset threshold, output the result information including the defect type of the screen of the electronic device and the position of the defect on the screen of the electronic device.
本实施例通过识别所述瑕疵检测结果的置信度，可以从瑕疵检测结果中筛选出可靠的结果进行输出。In this embodiment, by identifying the confidence level of the defect detection result, reliable results can be screened out from the defect detection results for output.
本发明的屏幕区域瑕疵检测方法一实施例中，将所述屏幕区域图像输入FPN网络结合backbone网络的模型之前，还包括：In an embodiment of the screen area defect detection method of the present invention, before inputting the screen area image into the FPN network combined with the backbone network model, the method further includes:
步骤一，预设FPN网络结合backbone网络的模型及其初始的模型参数；Step one, preset the FPN network combined with the backbone network model and its initial model parameters;
步骤二，将样本电子设备的屏幕区域图像输入带有当前的模型参数的FPN网络结合backbone网络的模型，得到样本电子设备的屏幕的瑕疵预测结果，所述瑕疵预测结果包括：样本电子设备的屏幕区域的瑕疵种类、瑕疵在样本电子设备的屏幕区域中的位置和瑕疵检测结果的置信度；Step 2: Input the screen area image of the sample electronic device into the FPN network with the current model parameters combined with the backbone network model to obtain the defect prediction result of the screen of the sample electronic device, and the defect prediction result includes: the screen of the sample electronic device The type of defect in the area, the position of the defect in the screen area of the sample electronic device, and the confidence level of the defect detection result;
步骤三，基于预设目标函数计算所述瑕疵预测结果与样本电子设备的真实瑕疵结果之间的差值，识别所述差值是否大于第二预设阈，Step 3: Calculate the difference between the defect prediction result and the real defect result of the sample electronic device based on a preset objective function, and identify whether the difference is greater than a second preset threshold,
若所述差值大于第二预设阈值，则步骤四，基于所述差值更新所述FPN网络结合backbone网络的模型参数后，重新从步骤二开始执行；If the difference is greater than the second preset threshold, step 4, after updating the model parameters of the FPN network combined with the backbone network based on the difference, restart execution from step 2;
若所述差值小于等于第二预设阈值，则步骤五，将带有当前的模型参数的FPN网络结合backbone网络的模型作为训练结束后的FPN网络结合backbone网络的模型。If the difference is less than or equal to the second preset threshold, step 5, the FPN network combined with the backbone network model with the current model parameters is used as the model of the FPN network combined with the backbone network after the training.
在此，通过识别所述差值是否大于第二预设阈，来循环训练FPN网络结合backbone网络的模型，能够得到可靠的模型。Here, by identifying whether the difference is greater than the second preset threshold, the model of the FPN network combined with the backbone network is cyclically trained to obtain a reliable model.
本发明还提供一种电子设备屏幕区域瑕疵检测设备，所述设备包括：The present invention also provides a device for detecting defects in the screen area of an electronic device, the device comprising:
第一装置，用于将电子设备的屏幕区域图像输入训练结束后的FPN网络结合backbone网络的模型；The first device is used to input the screen area image of the electronic device into the model of the FPN network combined with the backbone network after the training;
第二装置，用于从所述FPN网络结合backbone网络的模型接收输出的电子设备的屏幕区域图像的瑕疵检测结果，所述瑕疵检测结果包括：电子设备的屏幕区域的瑕疵种类、瑕疵在电子设备的屏幕区域中的位置和瑕疵检测结果的置信度。其中，瑕疵种类包括但不限于分层、透字、漏液、断线、亮点(亮斑)、色斑(发黄发青)。The second device is used to receive and output the defect detection result of the screen area image of the electronic device from the model of the FPN network combined with the backbone network. The defect detection result includes: the defect type of the screen area of the electronic device and the defect in the electronic device The position in the screen area and the confidence level of the defect detection result. Among them, the types of defects include, but are not limited to, delamination, penetrating characters, leakage, broken lines, bright spots (bright spots), and stains (yellow and blue).
在此，从所述FPN网络结合backbone网络的模型接收输出的电子设备的屏幕区域的瑕疵检测结果，每个瑕疵检测结果包含cls，x1，y1，x2，y2，score，其中，cls是缺陷类型，x1，y1，x2，y2是屏幕区域区域图像中瑕疵所在位置的4个坐标，score为这个瑕疵的置信度。Here, the defect detection result of the screen area of the electronic device received and output from the FPN network combined with the backbone network model, each defect detection result includes cls, x1, y1, x2, y2, score, where cls is the defect type , X1, y1, x2, y2 are the 4 coordinates of the position of the defect in the image of the screen area area, and score is the confidence level of this defect.
其中，所述FPN网络结合backbone网络的模型中每一次卷积计算完成后迭代更新归一化的参数，其中，归一化参数包括均值和方差，通过归一化确保每次输入都在合理的范围内变化，在此，所述归一化的参数随着输入数据的更新不断更新。Wherein, the FPN network combined with the backbone network model iteratively updates the normalized parameters after each convolution calculation is completed, where the normalized parameters include the mean and variance, and normalization ensures that each input is reasonable Changes within the range, here, the normalized parameters are continuously updated with the update of the input data.
本发明主要利用改进的特征金字塔(FPN)网络结合backbone网络的深度学习模型，能够准确地识别二手电子设备如手机的屏幕区域的差异。The present invention mainly utilizes the improved feature pyramid (FPN) network combined with the deep learning model of the backbone network, which can accurately identify the difference in the screen area of second-hand electronic devices such as mobile phones.
本发明的电子设备的屏幕区域瑕疵检测设备一实施例中，所述backbone网络的前2层采用res结构，网络的后2层采用inception结构。In an embodiment of the screen area defect detection device of the electronic device of the present invention, the first two layers of the backbone network adopt a res structure, and the last two layers of the network adopt an inception structure.
本发明的电子设备的屏幕区域瑕疵检测设备一实施例中，所述第二装置，还用于识别所述瑕疵检测结果的置信度是否大于第一预设阈值，若大于所述第一预设阈值，则输出包括电子设备的屏幕区域的瑕疵种类、瑕疵在电子设备的屏幕中的位置的结果信息。In an embodiment of the screen area defect detection device of an electronic device of the present invention, the second device is also used to identify whether the confidence level of the defect detection result is greater than a first preset threshold, and if it is greater than the first preset The threshold value outputs result information including the defect type of the screen area of the electronic device and the position of the defect on the screen of the electronic device.
本实施例通过识别所述瑕疵检测结果的置信度，可以从瑕疵检测结果中筛选出可靠的结果进行输出。In this embodiment, by identifying the confidence level of the defect detection result, reliable results can be screened out from the defect detection results for output.
本发明的电子设备的屏幕区域瑕疵检测设备一实施例中，还包括第三 装置，包括：In an embodiment of the screen area defect detection device of the electronic device of the present invention, it further includes a third device, including:
第三一装置，用于预设FPN网络结合backbone网络的模型及其初始的模型参数；The third device is used to preset the FPN network combined with the backbone network model and its initial model parameters;
第三二装置，用于将样本电子设备的屏幕区域图像输入带有当前的模型参数的FPN网络结合backbone网络的模型，得到样本电子设备的屏幕区域的瑕疵预测结果，所述瑕疵预测结果包括：样本电子设备的屏幕区域的瑕疵种类、瑕疵在样本电子设备的屏幕区域中的位置和瑕疵检测结果的置信度；The third and second device is used to input the screen area image of the sample electronic device into the FPN network combined with the backbone network model with the current model parameters to obtain the defect prediction result of the screen area of the sample electronic device, and the defect prediction result includes: The type of defects in the screen area of the sample electronic device, the position of the defect in the screen area of the sample electronic device, and the confidence level of the defect detection result;
第三三装置，用于基于预设目标函数计算所述瑕疵预测结果与样本电子设备的真实瑕疵结果之间的差值，识别所述差值是否大于第二预设阈，若所述差值大于第二预设阈值，则执行第三四装置，用于基于所述差值更新所述FPN网络结合backbone网络的模型参数后，重新从第三二装置开始执行；The third and third device is used to calculate the difference between the defect prediction result and the real defect result of the sample electronic device based on a preset objective function, and to identify whether the difference is greater than a second preset threshold, and if the difference is If it is greater than the second preset threshold, execute the third and fourth device for updating the model parameters of the FPN network combined with the backbone network based on the difference, and then restart execution from the third and second device;
若所述差值小于等于第二预设阈值，则执行第三三装置，将带有当前的模型参数的FPN网络结合backbone网络的模型作为训练结束后的FPN网络结合backbone网络的模型。If the difference is less than or equal to the second preset threshold, the third device is executed to use the FPN network combined with the backbone network model with the current model parameters as the model of the FPN network combined with the backbone network after the training.
在此，通过识别所述差值是否大于第二预设阈，来循环训练FPN网络结合backbone网络的模型，能够得到可靠的模型。Here, by identifying whether the difference is greater than the second preset threshold, the model of the FPN network combined with the backbone network is cyclically trained to obtain a reliable model.
根据本申请的再一方面，还提供了一种计算机可读介质，其中，According to another aspect of the present application, there is also provided a computer-readable medium, wherein:
其上存储有计算机可读指令，其上存储有计算机可执行指令，其中，该计算机可执行指令被处理器执行时使得该处理器：Computer-readable instructions are stored thereon, and computer-executable instructions are stored thereon. When the computer-executable instructions are executed by a processor, the processor causes the processor:
步骤S1，将电子设备的屏幕区域图像输入训练结束后的FPN网络结合backbone网络的模型；Step S1, input the screen area image of the electronic device into the model of the FPN network combined with the backbone network after the training;
步骤S2，从所述FPN网络结合backbone网络的模型接收输出的电子设备的屏幕区域图像的瑕疵检测结果，所述瑕疵检测结果包括：电子设备的屏幕区域的瑕疵种类、瑕疵在电子设备的屏幕区域中的位置和瑕疵检测结果的置信度。Step S2, the defect detection result of the screen area image of the electronic device is received from the model of the FPN network combined with the backbone network. The defect detection result includes: the defect type of the screen area of the electronic device and the defect in the screen area of the electronic device The position and the confidence level of the defect detection result.
本发明还提供一种电子设备屏幕区域瑕疵检测设备，其中，包括：The present invention also provides a device for detecting defects in the screen area of an electronic device, which includes:
处理器；以及Processor; and
被安排成存储计算机可执行指令的存储器，所述可执行指令在被执行时使所述处理器：A memory arranged to store computer-executable instructions which, when executed, cause the processor to:
步骤S1，将电子设备的屏幕区域图像输入训练结束后的FPN网络结合backbone网络的模型；Step S1, input the screen area image of the electronic device into the model of the FPN network combined with the backbone network after the training;
步骤S2，从所述FPN网络结合backbone网络的模型接收输出的电子设备的屏幕区域图像的瑕疵检测结果，所述瑕疵检测结果包括：电子设备的屏幕区域的瑕疵种类、瑕疵在电子设备的屏幕区域中的位置和瑕疵检测结果的置信度。Step S2, the defect detection result of the screen area image of the electronic device is received from the model of the FPN network combined with the backbone network. The defect detection result includes: the defect type of the screen area of the electronic device and the defect in the screen area of the electronic device The position and the confidence level of the defect detection result.
与现有技术相比，本发明通过将电子设备的屏幕区域图像输入训练结束后的FPN网络结合backbone网络的模型；从所述FPN网络结合backbone网络的模型接收输出的电子设备的屏幕区域图像的瑕疵检测结果，所述瑕疵检测结果包括：电子设备的屏幕区域的瑕疵种类、瑕疵在电子设备的屏幕区域中的位置和瑕疵检测结果的置信度，能够准确地识别二手电子设备如手机的屏幕区域的瑕疵差异。Compared with the prior art, the present invention inputs the image of the screen area of the electronic device into the model of the FPN network combined with the backbone network after the training; and receives the output of the screen area image of the electronic device from the model of the FPN network combined with the backbone network. Defect detection result, the defect detection result includes: the type of defect in the screen area of the electronic device, the position of the defect in the screen area of the electronic device, and the confidence level of the defect detection result, which can accurately identify the screen area of a second-hand electronic device such as a mobile phone The difference in blemishes.
对于本领域技术人员而言，显然本发明不限于上述示范性实施例的细节，而且在不背离本发明的精神或基本特征的情况下，能够以其他的具体形式实现本发明。因此，无论从哪一点来看，均应将实施例看作是示范性的，而且是非限制性的，本发明的范围由所附权利要求而不是上述说明限定，因此旨在将落在权利要求的等同要件的含义和范围内的所有变化涵括在本发明内。不应将权利要求中的任何附图标记视为限制所涉及的权利要求。此外，显然“包括”一词不排除其他单元或步骤，单数不排除复数。装置权利要求中陈述的多个单元或装置也可以由一个单元或装置通过软件或者硬件来实现。第一，第二等词语用来表示名称，而并不表示任何特定的顺序。For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description, and therefore it is intended to fall into the claims. All changes within the meaning and scope of the equivalent elements of are included in the present invention. Any reference signs in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "including" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claims can also be implemented by one unit or device through software or hardware. Words such as first and second are used to denote names, but do not denote any specific order.

Claims (12)

  1. 一种电子设备屏幕区域瑕疵检测方法，所述方法包括：A method for detecting defects in the screen area of an electronic device, the method comprising:
    将电子设备的屏幕区域图像输入训练结束后的FPN网络结合backbone网络的模型；Input the screen area image of the electronic device into the model of the FPN network after the training and the backbone network;
    从所述FPN网络结合backbone网络的模型接收输出的电子设备的屏幕区域图像的瑕疵检测结果，所述瑕疵检测结果包括：电子设备的屏幕区域的瑕疵种类、瑕疵在电子设备的屏幕区域中的位置和瑕疵检测结果的置信度。The defect detection result of the screen area image of the electronic device is received from the FPN network combined with the backbone network model. The defect detection result includes: the defect type of the screen area of the electronic device and the position of the defect in the screen area of the electronic device And the confidence level of the defect detection result.
  2. 根据权利要求1所述的方法，其中，所述backbone网络的前2层采用res结构，网络的后2层采用inception结构。The method according to claim 1, wherein the first two layers of the backbone network adopt a res structure, and the last two layers of the network adopt an inception structure.
  3. 根据权利要求1所述的方法，其中，从所述FPN网络结合backbone网络的模型接收输出的电子设备的屏幕区域的瑕疵检测结果之后，还包括：The method according to claim 1, wherein after receiving the output defect detection result of the screen area of the electronic device from the model of the FPN network combined with the backbone network, the method further comprises:
    识别所述瑕疵检测结果的置信度是否大于第一预设阈值，Identifying whether the confidence level of the defect detection result is greater than a first preset threshold,
    若大于所述第一预设阈值，则输出包括电子设备的屏幕区域的瑕疵种类、瑕疵在电子设备的屏幕区域中的位置的结果信息。If it is greater than the first preset threshold, output result information including the defect type of the screen area of the electronic device and the position of the defect in the screen area of the electronic device.
  4. 根据权利要求1所述的方法，其中，将所述电子设备的屏幕区域图像输入FPN网络结合backbone网络的模型之前，还包括：The method according to claim 1, wherein before inputting the screen area image of the electronic device into the FPN network combined with the backbone network model, the method further comprises:
    步骤一，预设FPN网络结合backbone网络的模型及其初始的模型参数；Step one, preset the FPN network combined with the backbone network model and its initial model parameters;
    步骤二，将样本电子设备的屏幕区域图像输入带有当前的模型参数的FPN网络结合backbone网络的模型，得到样本电子设备的屏幕区域的瑕疵预测结果，所述瑕疵预测结果包括：样本电子设备的屏幕区域的瑕疵种类、瑕疵在样本电子设备的屏幕区域中的位置和瑕疵检测结果的置信度；Step 2: Input the screen area image of the sample electronic device into the FPN network with current model parameters combined with the backbone network model to obtain the defect prediction result of the screen area of the sample electronic device. The defect prediction result includes: The type of flaws in the screen area, the position of the flaws in the screen area of the sample electronic device, and the confidence level of the flaw detection results;
    步骤三，基于预设目标函数计算所述瑕疵预测结果与样本电子设备的屏幕区域的真实瑕疵结果之间的差值，识别所述差值是否大于第二预设阈，Step 3: Calculate the difference between the defect prediction result and the actual defect result of the screen area of the sample electronic device based on a preset objective function, and identify whether the difference is greater than a second preset threshold,
    若所述差值大于第二预设阈值，则步骤四，基于所述差值更新所述FPN网络结合backbone网络的模型参数后，重新从步骤二开始执行；If the difference is greater than the second preset threshold, step 4, after updating the model parameters of the FPN network combined with the backbone network based on the difference, restart execution from step 2;
    若所述差值小于等于第二预设阈值，则步骤五，将带有当前的模型参数的FPN网络结合backbone网络的模型作为训练结束后的FPN网络结合backbone网络的模型。If the difference is less than or equal to the second preset threshold, step 5, the FPN network combined with the backbone network model with the current model parameters is used as the model of the FPN network combined with the backbone network after the training.
  5. 根据权利要求1所述的方法，其中，所述FPN网络结合backbone网络的模型中每一次卷积计算完成后迭代更新归一化的参数。The method according to claim 1, wherein the FPN network combined with the backbone network model iteratively updates the normalized parameters after each convolution calculation is completed.
  6. 一种电子设备屏幕区域瑕疵检测设备，所述设备包括：A defect detection device for a screen area of an electronic device, the device comprising:
    第一装置，用于将电子设备的屏幕区域图像输入训练结束后的FPN网络结合backbone网络的模型；The first device is used to input the screen area image of the electronic device into the model of the FPN network combined with the backbone network after the training;
    第二装置，用于从所述FPN网络结合backbone网络的模型接收输出的电子设备的屏幕区域图像的瑕疵检测结果，所述瑕疵检测结果包括：电子设备的屏幕区域的瑕疵种类、瑕疵在电子设备的屏幕区域中的位置和瑕疵检测结果的置信度。The second device is used to receive and output the defect detection result of the screen area image of the electronic device from the model of the FPN network combined with the backbone network. The defect detection result includes: the defect type of the screen area of the electronic device and the defect in the electronic device The position in the screen area and the confidence level of the defect detection result.
  7. 根据权利要求6所述的设备，其中，所述backbone网络的前2层采用res结构，网络的后2层采用inception结构。The device according to claim 6, wherein the first two layers of the backbone network adopt a res structure, and the last two layers of the network adopt an inception structure.
  8. 根据权利要求6所述的设备，其中，第二装置还用于识别所述瑕疵检测结果的置信度是否大于第一预设阈值，若大于所述第一预设阈值，则输出包括电子设备的屏幕区域的瑕疵种类、瑕疵在电子设备的屏幕区域中的位置的结果信息。7. The device according to claim 6, wherein the second device is further configured to identify whether the confidence level of the defect detection result is greater than a first preset threshold, and if it is greater than the first preset threshold, outputting information including electronic equipment The result information of the defect type of the screen area and the position of the defect in the screen area of the electronic device.
  9. 根据权利要求6所述的设备，其中，还包括第三装置，包括：The device according to claim 6, further comprising a third device, comprising:
    第三一装置，用于预设FPN网络结合backbone网络的模型及其初始的模型参数；The third device is used to preset the FPN network combined with the backbone network model and its initial model parameters;
    第三二装置，用于将样本电子设备的屏幕区域图像输入带有当前的模型参数的FPN网络结合backbone网络的模型，得到样本电子设备的屏幕区域的瑕疵预测结果，所述瑕疵预测结果包括：样本电子设备的屏幕区域的瑕疵种类、瑕疵在样本电子设备的屏幕区域中的位置和瑕疵检测结果的置信度；The third and second device is used to input the screen area image of the sample electronic device into the FPN network combined with the backbone network model with the current model parameters to obtain the defect prediction result of the screen area of the sample electronic device, and the defect prediction result includes: The type of defects in the screen area of the sample electronic device, the position of the defect in the screen area of the sample electronic device, and the confidence level of the defect detection result;
    第三三装置，用于基于预设目标函数计算所述瑕疵预测结果与样本电子设备的真实瑕疵结果之间的差值，识别所述差值是否大于第二预设阈，若所述差值大于第二预设阈值，则执行第三四装置，用于基于所述差值更新所述FPN网络结合backbone网络的模型参数后，重新从第三二装置开始执行；The third and third device is used to calculate the difference between the defect prediction result and the real defect result of the sample electronic device based on a preset objective function, and to identify whether the difference is greater than a second preset threshold, if the difference is If it is greater than the second preset threshold, execute the third and fourth device for updating the model parameters of the FPN network combined with the backbone network based on the difference, and then restart execution from the third and second device;
    若所述差值小于等于第二预设阈值，则执行第三五装置，将带有当前的模型参数的FPN网络结合backbone网络的模型作为训练结束后的FPN网络结合backbone网络的模型。If the difference is less than or equal to the second preset threshold, the third and fifth device is executed to use the FPN network combined with the backbone network model with the current model parameters as the model of the FPN network combined with the backbone network after the training.
  10. 根据权利要求6所述的设备，其中，所述FPN网络结合backbone网络的模型中每一次卷积计算完成后迭代更新归一化的参数。7. The device according to claim 6, wherein the FPN network combined with the backbone network model iteratively updates the normalized parameters after each convolution calculation is completed.
  11. 一种计算机可读介质，其中，A computer readable medium in which,
    其上存储有计算机可读指令，所述计算机可读指令可被处理器执行以实现如权利要求1至5任一项所述的方法。There are computer-readable instructions stored thereon, and the computer-readable instructions can be executed by a processor to implement the method according to any one of claims 1 to 5.
  12. 一种电子设备屏幕区域瑕疵检测设备，其中，所述设备包括：An electronic device screen area defect detection device, wherein the device includes:
    一个或多个处理器；以及One or more processors; and
    存储有计算机可读指令的存储器，所述计算机可读指令在被执行时使所述处理器执行如权利要求1至5中任一项所述方法的操作。A memory storing computer-readable instructions, which when executed, cause the processor to perform the operation of the method according to any one of claims 1 to 5.
PCT/CN2020/120875 2019-10-28 2020-10-14 Defect detecting method and equipment for screen region of electronic equipment WO2021082919A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911032900.3 2019-10-28
CN201911032900.3A CN110796647A (en) 2019-10-28 2019-10-28 Method and device for detecting defects of screen area of electronic device

Publications (1)

Publication Number Publication Date
WO2021082919A1 true WO2021082919A1 (en) 2021-05-06

Family

ID=69441592

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/120875 WO2021082919A1 (en) 2019-10-28 2020-10-14 Defect detecting method and equipment for screen region of electronic equipment

Country Status (2)

Country Link
CN (1) CN110796647A (en)
WO (1) WO2021082919A1 (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114037700A (en) * 2021-12-31 2022-02-11 成都数联云算科技有限公司 Panel defect detection method and device, electronic equipment and storage medium
US11798250B2 (en) 2019-02-18 2023-10-24 Ecoatm, Llc Neural network based physical condition evaluation of electronic devices, and associated systems and methods
CN117132584A (en) * 2023-09-22 2023-11-28 山东省计算中心（国家超级计算济南中心） Liquid crystal display screen flaw detection method and device based on deep learning
US11843206B2 (en) 2019-02-12 2023-12-12 Ecoatm, Llc Connector carrier for electronic device kiosk
US11922467B2 (en) 2020-08-17 2024-03-05 ecoATM, Inc. Evaluating an electronic device using optical character recognition
US11989710B2 (en) 2018-12-19 2024-05-21 Ecoatm, Llc Systems and methods for vending and/or purchasing mobile phones and other electronic devices
US12033454B2 (en) 2020-08-17 2024-07-09 Ecoatm, Llc Kiosk for evaluating and purchasing used electronic devices

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110675399A (en) * 2019-10-28 2020-01-10 上海悦易网络信息技术有限公司 Screen appearance flaw detection method and equipment
CN110796646A (en) * 2019-10-28 2020-02-14 上海悦易网络信息技术有限公司 Method and device for detecting defects of screen area of electronic device
CN110796647A (en) * 2019-10-28 2020-02-14 上海悦易网络信息技术有限公司 Method and device for detecting defects of screen area of electronic device

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875381A (en) * 2017-01-17 2017-06-20 同济大学 A kind of phone housing defect inspection method based on deep learning
US20180300880A1 (en) * 2017-04-12 2018-10-18 Here Global B.V. Small object detection from a large image
CN109859190A (en) * 2019-01-31 2019-06-07 北京工业大学 A kind of target area detection method based on deep learning
CN109859163A (en) * 2018-12-19 2019-06-07 重庆邮电大学 A kind of LCD defect inspection method based on feature pyramid convolutional neural networks
CN110097544A (en) * 2019-04-25 2019-08-06 武汉精立电子技术有限公司 A kind of display panel open defect detection method
CN110796647A (en) * 2019-10-28 2020-02-14 上海悦易网络信息技术有限公司 Method and device for detecting defects of screen area of electronic device

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109711474B (en) * 2018-12-24 2023-01-17 中山大学 Aluminum product surface defect detection algorithm based on deep learning
CN109886077B (en) * 2018-12-28 2021-07-09 北京旷视科技有限公司 Image recognition method and device, computer equipment and storage medium
CN110378420A (en) * 2019-07-19 2019-10-25 Oppo广东移动通信有限公司 A kind of image detecting method, device and computer readable storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875381A (en) * 2017-01-17 2017-06-20 同济大学 A kind of phone housing defect inspection method based on deep learning
US20180300880A1 (en) * 2017-04-12 2018-10-18 Here Global B.V. Small object detection from a large image
CN109859163A (en) * 2018-12-19 2019-06-07 重庆邮电大学 A kind of LCD defect inspection method based on feature pyramid convolutional neural networks
CN109859190A (en) * 2019-01-31 2019-06-07 北京工业大学 A kind of target area detection method based on deep learning
CN110097544A (en) * 2019-04-25 2019-08-06 武汉精立电子技术有限公司 A kind of display panel open defect detection method
CN110796647A (en) * 2019-10-28 2020-02-14 上海悦易网络信息技术有限公司 Method and device for detecting defects of screen area of electronic device

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11989710B2 (en) 2018-12-19 2024-05-21 Ecoatm, Llc Systems and methods for vending and/or purchasing mobile phones and other electronic devices
US11843206B2 (en) 2019-02-12 2023-12-12 Ecoatm, Llc Connector carrier for electronic device kiosk
US11798250B2 (en) 2019-02-18 2023-10-24 Ecoatm, Llc Neural network based physical condition evaluation of electronic devices, and associated systems and methods
US11922467B2 (en) 2020-08-17 2024-03-05 ecoATM, Inc. Evaluating an electronic device using optical character recognition
US12033454B2 (en) 2020-08-17 2024-07-09 Ecoatm, Llc Kiosk for evaluating and purchasing used electronic devices
CN114037700A (en) * 2021-12-31 2022-02-11 成都数联云算科技有限公司 Panel defect detection method and device, electronic equipment and storage medium
CN117132584A (en) * 2023-09-22 2023-11-28 山东省计算中心（国家超级计算济南中心） Liquid crystal display screen flaw detection method and device based on deep learning
CN117132584B (en) * 2023-09-22 2024-02-13 山东省计算中心（国家超级计算济南中心） Liquid crystal display screen flaw detection method and device based on deep learning

Also Published As

Publication number Publication date
CN110796647A (en) 2020-02-14

Similar Documents

Publication Publication Date Title
WO2021082919A1 (en) Defect detecting method and equipment for screen region of electronic equipment
WO2021082918A1 (en) Screen appearance defect detection method and device
WO2021082923A1 (en) Electronic device screen area defect detection method and device
WO2021082921A1 (en) Back cover appearance defect detection method for electronic apparatus, and apparatus
WO2021082920A1 (en) Method and device for detecting border appearance defects of electronic device
US11010285B2 (en) Fault detection and localization to generate failing test cases using combinatorial test design techniques
US10997965B2 (en) Automated voice processing testing system and method
US11941721B2 (en) Using watermark information and weight information to train an embedded neural network model
US11010282B2 (en) Fault detection and localization using combinatorial test design techniques while adhering to architectural restrictions
WO2020029608A1 (en) Method and apparatus for detecting burr of electrode sheet
CN110827244A (en) Method and equipment for detecting appearance flaws of electronic equipment
US11080406B2 (en) Detection of vulnerable code
WO2021056914A1 (en) Automatic modeling method and apparatus for object detection model
CN111080633A (en) Screen defect detection method and device, terminal equipment and storage medium
WO2022116720A1 (en) Target detection method and apparatus, and electronic device
CN110796669A (en) Vertical frame positioning method and equipment
CN111539456B (en) Target identification method and device
US20210263837A1 (en) Real time fault localization using combinatorial test design techniques and test case priority selection
CN111382223B (en) Electronic map display method, terminal and electronic equipment
WO2023000444A1 (en) Method and apparatus for detecting noise of loudspeaker, and electronic device and storage medium
CN111429431A (en) Element positioning and identifying method based on convolutional neural network
US11423654B2 (en) Identification of continuity errors in video by automatically detecting visual inconsistencies in video frames
CN113469696A (en) User abnormality degree evaluation method and device and computer readable storage medium
WO2022242630A1 (en) Method and device for determining textual picture
US11508143B2 (en) Automated salience assessment of pixel anomalies

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20883481

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20883481

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20883481

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 26.01.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20883481

Country of ref document: EP

Kind code of ref document: A1