CN117876381B

CN117876381B - AI visual detection method and system for identifying and analyzing concrete structure cracks

Info

Publication number: CN117876381B
Application number: CN202410283163.9A
Authority: CN
Inventors: 邱学刚; 魏晓斌
Original assignee: Jiangsu Testing Center For Quality Of Construction Engineering Co ltd
Current assignee: Jiangsu Testing Center For Quality Of Construction Engineering Co ltd
Priority date: 2024-03-13
Filing date: 2024-03-13
Publication date: 2024-05-10
Anticipated expiration: 2044-03-13
Also published as: CN117876381A

Abstract

The invention discloses an AI visual detection method and system for identifying and analyzing cracks of a concrete structure, comprising the steps of collecting basic data according to a preset period, including reading image data of a multispectral camera, and obtaining monitoring data of a stress-strain sensor; initially segmenting a crack region image from the extracted image data; extracting a crack displacement field based on the crack region image, and screening the crack region image through the crack displacement field to obtain a key crack region image; invoking a constructed random forest classification optimization module to classify the key crack region images to obtain an image dataset composed of at least two types of crack region images; and extracting a crack skeleton according to each type of crack region image and the monitoring data of the corresponding space time of the crack region image, quantitatively analyzing a crack displacement field, and giving out a crack development trend analysis result and outputting the analysis result. The invention solves the problems of environmental adaptation, process reconstruction and three-dimensional reconstruction in the crack identification process.

Description

AI visual detection method and system for identifying and analyzing concrete structure cracks

Technical Field

The invention relates to a building quality monitoring and analyzing technology, in particular to an AI visual detection method and system for identifying and analyzing cracks of a concrete structure.

Background

The concrete structure cracks are important indexes for reflecting the health condition of the building, and timely and accurately identifying and analyzing the cracks is important for guaranteeing the safety of the building and prolonging the service life. Traditional crack detection mainly relies on manual inspection, and has the problems of low efficiency, strong subjectivity and the like. In recent years, artificial Intelligence (AI) technology represented by computer vision has been rapidly developed, providing a new idea for solving the above-mentioned problems. The AI visual detection is introduced into the concrete crack recognition, so that automation, intellectualization and refinement of crack detection are hopefully realized, and the speed, accuracy and reliability of crack analysis are improved, so that maintenance and repair of an engineering structure are better guided.

Currently, scholars at home and abroad develop a series of researches around the application of AI visual detection in concrete crack identification. Some students use traditional image processing methods such as edge detection, threshold segmentation, etc. for crack extraction, but such methods have difficulty in coping with crack recognition in complex backgrounds. Still other scholars explore new methods of crack detection based on texture features, such as Gabor filtering, wavelet transformation, etc., however, manually designed features are sensitive to noise and have limited generalization ability. In recent years, deep learning, particularly Convolutional Neural Networks (CNNs), has made breakthrough progress in the field of computer vision. Some researchers began to apply deep learning models such as CNN to concrete crack detection and achieved good results. The methods can automatically learn multi-level Gao Panbie force characteristic representation of the crack, and overcome the defects of the traditional methods.

Although the existing research lays a foundation for AI visual detection of concrete cracks, a plurality of targeted technical problems and individual demands are still faced in engineering application, and the method mainly comprises the steps that ① in special structures such as tunnels, high-speed rails, bridges and complex buildings, the characteristics of the concrete surface are rich, various pseudo defects such as skinning, cracks and scratches exist, the imaging environment is uneven in illumination and changeable in visual angle, and great challenges are brought to the resolution of cracks and backgrounds. How to accurately divide cracks in a noisy background is a difficult problem to overcome. ② . In the early aging or special stress state of concrete, some microscopic cracks which are difficult to be perceived by naked eyes can appear. These cracks, although small in size, reflect early health problems of the structure. The detection rate of the existing method for the fine defects is still to be improved. ③ . The engineering field not only focuses on the existence of cracks, but also needs to describe the width, length, shape, trend and the like of the cracks in a fine quantitative manner so as to judge the types of the cracks and analyze the causative mechanism. However, most visual detection methods focus only on pixel-level segmentation of the crack region, and it is difficult to accurately acquire such semantic-level information. ④ . The final purpose of crack detection is to judge the structural health state and predict the disease development trend. How to construct a growth model facing the crack propagation process according to continuously observed crack images is another problem to be studied deeply. ⑤ . And (5) carrying out three-dimensional fine reconstruction on the crack. Most of the current visual detection methods only can obtain two-dimensional plane distribution of a crack region, cannot accurately represent three-dimensional characteristic parameters such as longitudinal depth, volume and the like of the crack, and have limitation on judgment of the severity of the crack.

As can be seen, intelligent visual detection of concrete structure cracks still has many technical challenges such as environmental adaptation, defect sensitivity, semantic analysis, process modeling, three-dimensional reconstruction, and the like.

Disclosure of Invention

The invention aims to provide an AI visual detection method and system for identifying and analyzing cracks of a concrete structure, so as to solve the problems in the prior art.

According to an aspect of the present application, there is provided an AI visual inspection method for identifying and analyzing cracks of a concrete structure, including the steps of:

Step S1, basic data are collected according to a preset period, wherein the basic data comprise image data of a multispectral camera are read, and monitoring data of a stress-strain sensor are obtained;

S2, calling a pre-trained rough segmentation module based on transfer learning or a pyramid rough segmentation module of a multi-scale fusion attention mechanism, and primarily segmenting a crack region image from the extracted image data; extracting a crack displacement field based on the crack region image, and screening the crack region image through the crack displacement field to obtain a key crack region image;

S3, invoking a constructed random forest classification optimization module to classify the key crack region images to obtain an image dataset composed of at least two types of crack region images;

And S4, extracting a crack skeleton according to each type of crack region image and the monitoring data of the corresponding space time of the crack region image, quantitatively analyzing a crack displacement field, and giving out a crack development trend analysis result and outputting the analysis result.

According to one aspect of the present application, the step S1 is further:

Step S11, based on structural analysis of constructional engineering, arranging stress strain sensors including strain gauges, vibration sensors, displacement meters and grating optical fibers;

step S12, designing a camera shooting site, and periodically collecting image data of a preset area through a multispectral camera;

and S13, mapping the monitoring data of different modes to a unified space-time coordinate system based on the positions of the stress strain sensor and the multispectral camera and the sampling time stamp to form a standard monitoring data set.

According to one aspect of the present application, the process of constructing the pre-trained coarse segmentation module based on the migration learning in the step S2 further includes:

s21, constructing or calling a surface fracture image, and forming a surface fracture image training set by the fracture image synthesized based on the GAN generator;

S22, constructing an end-to-end crack segmentation depth neural network by adopting U-Net, and aligning source domain data distribution with target domain data distribution through countermeasure learning;

S23, optimizing model parameters by adopting a back propagation algorithm; and verifying the effect of the rough segmentation module through the verification set, and outputting the rough segmentation module after meeting the preset standard.

According to one aspect of the present application, in the step S2, the process of extracting the crack displacement field based on the crack region image, screening the crack region image by the crack displacement field, and obtaining the key crack region image further includes:

step S2a, based on texture similarity of a crack region in a sequence image between a front frame and a rear frame, determining homonymous pixels through a normalized cross-correlation criterion, searching a target subregion which is most similar to a reference subregion in the crack image, and calculating a displacement vector of each pixel through coordinate mapping between the reference subregion and the target subregion to obtain a dense displacement field;

S2b, performing smooth filtering and threshold segmentation on the dense displacement field, eliminating random noise, calculating a displacement gradient field, extracting a displacement mutation region, and taking the mutation region as a key crack candidate region;

And S2c, performing morphological filtering on the displacement mutation region by using priori knowledge of the crack region, removing non-crack regions with regular shapes and dimensions smaller than a threshold value, obtaining candidate regions of suspected key cracks, and forming a key crack region image set.

According to one aspect of the present application, the step S3 is further:

s31, extracting geometric features, texture features and displacement field features of key crack areas, and constructing area feature vectors; geometric features including area, perimeter, aspect ratio and rectangularity, and texture features including gray level co-occurrence matrix and local binary pattern; the displacement field features comprise a mean value and a variance of displacement amplitude values in the region;

s32, selecting a typical candidate area, and marking the typical candidate area as a positive sample and a negative sample according to whether the typical candidate area is a real crack; randomly sampling a sample by adopting a Bootstrap method, constructing a plurality of decision trees, generating a random forest model by taking Gini coefficients as splitting criteria by tree nodes, and training;

S33, classifying and scoring confidence degrees of all the candidate areas by using a trained random forest model;

Converting the decision tree output into posterior probability based on the Softmax principle, wherein the multi-tree synthesis can obtain the confidence that the region is a crack; and setting a threshold value to filter the candidate region, removing the misclassified region, obtaining an optimized crack region with high confidence, and forming an image data set consisting of at least two types of crack region images.

According to one aspect of the present application, the step S4 is further:

Step S41, performing morphological refinement on the segmented crack region aiming at the crack region image, extracting a crack skeleton, and decomposing the crack skeleton into single crack segments by utilizing a skeleton endpoint and intersection detection algorithm;

S42, taking a single section of a framework as a central line, sampling the crack boundary in the vertical direction, measuring the width of the crack, and calculating the length of the crack by multiplying the number of pixels of the framework by the spatial resolution; generating a length and width sequence of each section of crack, and calculating a maximum value, a minimum value and an average value;

Step S43, fitting a minimum circumscribed rectangle of a single section of the framework, counting the number and total length of cracks with different directions along the long axis direction of the rectangle, analyzing the direction distribution rule of the cracks, constructing a topological structure of the framework network of the cracks, analyzing the connectivity of the cracks through a graph theory algorithm, and excavating penetrating cracks;

step S44, associating the dense displacement field with a crack segmentation result, extracting displacement subfields of each section of crack, calculating statistical characteristics of the crack displacement field, calculating displacement differences of areas on two sides of the crack, and evaluating crack fracture degree, wherein the statistical characteristics comprise maximum displacement value, average displacement and displacement direction distribution;

S45, decomposing a displacement field into a normal displacement component vertical to the trend of the crack and a tangential displacement component parallel to the trend of the crack, respectively analyzing the normal displacement and the tangential displacement, evaluating the opening degree and the shearing slip degree of the crack, and combining time sequence analysis of the normal displacement and the tangential displacement to judge the dynamic evolution rule of the crack opening and closing;

Step S46, estimating the expansion rate of a single crack by combining the crack width and the displacement analysis result, clustering and analyzing the growth rates of cracks in different positions and different directions, analyzing the spatial difference of crack development, establishing a crack growth curve based on a time sequence, and predicting the future crack development trend.

According to an aspect of the present application, the step S46 further includes:

step S46a, adding a time dimension on a crack region identified by an image to form a space-time three-dimensional cube for crack development, fusing synchronously acquired strain and temperature, and forming a plurality of physical fields aligned with the crack region in the space-time cube;

step S46b, the time stamp of the image sequence is a time axis, the image pixel grid is a space coordinate base, and the crack region label, the displacement vector and the temperature and humidity value at each pixel are mapped to a hexahedral unit;

Step S46c, introducing a correlation metric between adjacent units on the basis of the time-space unit attribute, and constructing an incidence matrix for reflecting time-space dependence;

and step S46d, dividing the crack development time course by using a wavelet transformation method.

According to one aspect of the application, the method further comprises the step S5 of reconstructing the crack area in three dimensions

Step S51, multi-view multispectral image data are acquired, back projection is carried out through parameters in a camera and parameters outside the view angles, pixel points under an image coordinate system are transformed into a three-dimensional space coordinate system, and local point clouds under each view angle are obtained; carrying out space transformation and fusion on local point clouds of different visual angles by adopting a registration algorithm to obtain a surface dense point cloud under a unified coordinate system;

Step S52, extracting a complete crack region point cloud segment from the surface dense point cloud by using the marked crack points as seeds through a three-dimensional point cloud segmentation algorithm; iteratively expanding in a proper spatial neighborhood until all connected crack points are gathered in the same point set;

Step S53, obtaining a central curve skeleton of the crack point cloud through a Chen shrinkage algorithm; the skeleton reflects the topological structure of the crack; and (3) taking the central curve skeleton as a reference, locally establishing a tangential plane in the crack, and extracting a point set in the plane as a cross section of the crack. Obtaining a cross-sectional shape through curve fitting, and continuously reconstructing along a skeleton line to obtain a series of cross-sectional curves;

S54, using a framework as a longitudinal reference and a cross section as contour constraint, and generating a smooth continuous crack three-dimensional curved surface model by a parameterized curved surface fitting method;

Step S55, measuring the length, the height, the width and the volume of the crack on the reconstructed three-dimensional curved surface of the crack; based on geometric parameters including length, width, and number of branches, automatic evaluation of fracture severity is performed against fracture grading criteria.

According to one aspect of the present application, in step S2, the process of constructing the pyramid coarse segmentation module of the multi-scale fusion attention mechanism is further as follows:

s2i, constructing a multi-scale image pyramid, setting scale parameters, generating a series of downsampled images, and continuously using Gaussian smoothing and downsampling to obtain an image sequence with gradually reduced scale;

step S2ii, calculating the significance weight of each pixel on the original image, and highlighting the area similar to the appearance of the crack; extracting a saliency characteristic map, and multiplying the characteristic vector of each pixel in the multi-scale image pyramid by a corresponding saliency weight to obtain a self-adaptive enhanced characteristic map;

Step S2iii, sending the segmentation probability graphs with multiple scales into a decision fusion module, adopting a weighted average or voting mechanism, adaptively integrating the multi-scale decision result, outputting a final rough segmentation graph, and primarily segmenting out a crack region image.

According to an aspect of the present application, the step S44 further includes:

Step S44i, constructing a twin network, and calculating a feature vector f (t) and a feature vector f (t+1) of an image pair to be matched;

step S44ii, calculating cosine similarity of the subvectors of different crack areas of the feature vector f (t) and the feature vector f (t+1), and generating a similarity graph;

in step S44iii, a region with the highest similarity is found in the feature vector f (t+1) with the feature vector f (t) as a reference, and the region corresponds to the matched fracture region.

According to one aspect of the present application, there is also provided an AI visual inspection system for crack identification and analysis of a concrete structure, comprising:

at least one processor; and

A memory communicatively coupled to at least one of the processors; wherein,

The memory stores instructions executable by the processor for execution by the processor to implement the AI visual detection method for concrete structure crack identification and analysis of any one of the above-described aspects.

The method has the beneficial effects that the comprehensive, multi-angle and refined characterization of cracks from static to dynamic, from surface to inside and from single to whole is carried out on the cracks from static to dynamic, from surface to inside and from single to whole by combining with advanced methods such as machine learning, image analysis and motion measurement. The method can greatly improve the efficiency and accuracy of crack detection, save manpower and material resources, and more importantly, the technology can help engineers to evaluate the structural performance and make management and maintenance decisions by upgrading the crack detection from 'visible' to 'understandable' through intelligent research and judgment on crack type, cause, severity and trend prediction, and has great significance in improving the safety and durability of a concrete structure.

Drawings

Fig. 1 is a flow chart of the present invention.

Fig. 2 is a flowchart of step S1 of the present invention.

Fig. 3 is a flow chart of step S2 of the present invention.

Fig. 4 is a flowchart of step S3 of the present invention.

Fig. 5 is a flowchart of step S4 of the present invention.

Detailed Description

As shown in fig. 1, according to an aspect of the present application, there is provided an AI visual inspection method for recognition and analysis of cracks of a concrete structure, comprising the steps of:

By synchronously collecting high-definition images of the concrete surface and internal physical response data, a comprehensive crack characteristic description system is established. The multispectral imaging can capture textures and spectral features of cracks in different wave bands, provides abundant visual priori for semantic segmentation, and the monitoring quantity of stress strain and the like reflects the stress state around the cracks and reveals the internal mechanism of generation and expansion of the cracks. The complementary fusion of the multi-mode data can construct a crack image with 'both exterior and interior repair', and can more describe static-dynamic, local-integral properties of the crack than a single data source. In addition, accurate registration and synchronous acquisition of space-time data also lay a foundation for multi-dimensional correlation analysis of subsequent cracks.

by adopting a migration learning strategy and utilizing a segmentation model pre-trained on a large-scale fracture data set, the training cost of the model under a new working condition can be remarkably reduced, and the rapid adaptation under the conditions of cross-scene and small sample is realized. The introduction of the multi-scale pyramid structure and the attention mechanism further enhances the extraction capability of the model on multi-scale and detail characteristics of the crack. On the basis, the displacement field information of the crack region is skillfully utilized to carry out self-adaptive optimization, and the active cracks with larger influence on structural safety can be effectively identified through the abnormal detection of the local motion mode, so that the automatic screening of the key monitoring region is realized. This strategy avoids the limitation of "one view of the same kernel" for all cracks, contributing to focus major contradictions.

The key crack areas of the preliminary screening are finely classified by adopting a random forest algorithm, and the method can be regarded as crack attribution based on area characteristics. The random forest integrates the judging results of a plurality of decision trees, so that the importance of each feature can be evaluated while the classification generalization is improved, and key factors influencing the crack attribute are mined. The scheme not only considers the geometric and texture characteristics of the crack region, but also integrates the implicit semantics of displacement field statistics and the like, and forms a high-dimensional feature space of 'form+physical'. In addition, the robustness and the adaptability of the classifier are further enhanced by optimization measures such as positive and negative sample balance, self-help sampling and the like. By accurately distinguishing the type, cause and risk of the crack, the step realizes the span from qualitative description to quantitative analysis.

Through fracture skeleton extraction and morphological parameter measurement, the scheme realizes parameterization and vectorization expression of the fracture, and converts morphological characteristics such as length, width, trend and the like into quantifiable indexes, so that engineering management is facilitated. And the extraction of the skeleton topological structure reveals the organization degree of the crack in space from higher layers such as connectivity, direction distribution and the like, reflects the change of a load transmission path and a stress flow field, and has important significance for judging the structural stability. Further, by tracking and analyzing the displacement field evolution of the crack unit, the scheme innovatively introduces a kinematic view angle, and breaks through the traditional 'static-like' analysis paradigm. The space-time difference of the statistical characteristics of the displacement field can identify the expansion characteristics of the crack in different directions and parts, and quantitative indexes such as opening degree, dislocation and the like provide basis for evaluating the influence range of the crack and the instability risk. Based on the growth curve and community characteristics of the crack, the scheme can also perform extrapolation prediction and multi-scale correlation analysis of the development trend of the crack, and has important value for the prediction of the service period, reinforcement and maintenance decision of the concrete structure.

As shown in fig. 2, according to an aspect of the present application, the step S1 is further:

The method is mainly used for solving the data acquisition problem in concrete structure crack detection. By arranging various sensors, such as strain gauges, vibration sensors and the like, the stress-strain state of the concrete structure can be comprehensively obtained, and early weak signals generated by cracks can be captured. Meanwhile, the multispectral camera is utilized to periodically acquire the concrete surface image, so that the morphological characteristics of the crack can be recorded in a visual way. Step S13, unifying monitoring data of different sources and different modes under the same reference system through space-time data registration to form a standard data set with consistent space-time and semantic matching. The method provides a data basis for subsequent multi-source data fusion and multi-angle crack characterization. The physical perception and the visual perception are comprehensively utilized, and the acquisition efficiency, the precision and the coverage range of the crack data can be obviously improved.

In some embodiments, three-dimensional space coordinates (x, y, z) of each sensor such as a strain gauge, a displacement meter and the like are obtained through measuring equipment such as a total station or a laser tracker and the like, and a corresponding relation between the space coordinates of the sensor and the coordinates of a structural analysis model is established to form a sensor coordinate mapping matrix M_sensor.

And registering the camera external parameters (R, T) with the structure analysis model coordinates to establish a mapping relation M_camera of the camera pixel coordinates and the structure actual position.

And aligning the sampling time of the sensor with the image time stamp by linear interpolation and other methods to form a uniform time coordinate system.

According to the sensor coordinate mapping matrix M_sensor, sensor data such as strain, displacement and the like are mapped to three-dimensional space coordinates (X, Y, Z) of a structural model, image pixel coordinates (u, v) are associated with corresponding structural space coordinates (X, Y, Z) by utilizing camera external parameter mapping M_camera, and space-time information of different mode data is matched under a unified time coordinate system to form a heterogeneous monitoring data set with consistent space-time. Through the steps, the unification and registration of the monitoring data of different modes in space-time dimension are realized, and a foundation is laid for the subsequent multi-mode fusion analysis.

In some embodiments, a multispectral camera array is added in the step S11 to acquire images of different wave bands, in the step S13, multispectral data are added into feature fusion, and the spectrum-stress-strain multimode fingerprints of cracks are extracted to construct a high-dimensional space-time data cube. And optimizing a method for registering and synchronizing the time stamps of the monitoring data to realize the precise registration of the multi-source data. And acquiring spectral response data of the concrete structure by utilizing spectrum cameras of a plurality of wave bands such as short wave infrared and medium wave infrared, and revealing spectrum fingerprint differences caused by internal defects and material degradation. Spectral resonance imaging can be used for nondestructively detecting internal cracks, enhancing surface texture and being a beneficial supplement to visible light imaging. The fusion of physical quantities such as multispectral data and stress strain is expected to realize accurate characterization of crack causes and severity. The specific process is as follows:

and arranging a visible light camera and Short Wave Infrared (SWIR) and Medium Wave Infrared (MWIR) spectrum cameras to form a multispectral camera array.

And controlling the multispectral camera array to synchronously acquire the concrete surface image, and obtaining visible light, SWIR and MWIR spectrum data.

And carrying out radiation correction and spectrum calibration on the original spectrum data according to the imaging parameters to obtain a radiation brightness image with definite physical meaning.

Preprocessing the corrected multispectral image, including noise suppression, illumination normalization and the like.

The spectral curve of each pixel is extracted, and spectral characteristics such as spectral attenuation coefficient, reflectivity, absorption peak position and the like are calculated.

Spectral clustering is carried out through spectral similarity measurement (such as a spectral angle and a correlation coefficient), material properties are distinguished, and suspected crack areas are marked.

And correlating the physical quantity data acquired by the stress strain sensor with the space coordinates thereof, and interpolating to generate a continuous physical quantity distribution map.

And carrying out scale normalization and coordinate transformation on the physical quantity distribution map, and carrying out geometric registration with the multispectral image.

And extracting pixel values at corresponding positions in the spectrum image and the physical quantity diagram, and constructing a multi-mode attribute vector.

And carrying out block division on the registered multi-mode data, wherein each block corresponds to a local area of the concrete.

And at the block level, the fusion of the spectrum characteristic and the physical quantity characteristic is realized by means of characteristic series connection, data alignment and the like.

High-level semantic representations of multi-modal features, such as Convolutional Neural Networks (CNNs), long-term memory networks (LSTMs), and the like, are extracted using deep neural networks.

And according to the time stamps of the multispectral camera and the physical quantity sensor, aligning the data of different modes according to time to form a time sequence.

A space-time consistency loss function is introduced to restrict the continuity of the multi-modal features of adjacent time steps in the time dimension.

Meanwhile, the consistency of the space neighborhood is considered, and the correlation between the local areas is modeled by using methods such as graph convolution and the like, so that the smoothing in the space dimension is realized.

Based on the fused multi-modal characteristics, training a crack detection model through a deep learning method to realize the end-to-end mapping of spectrum-stress-strain fingerprints.

And (3) reasoning the newly acquired multispectral data by using a trained model, and predicting the internal crack distribution according to the spectral characteristics and the physical quantity change.

And fusing the predicted crack region with the visible light image to generate a spectrum resonance image with internal defect information.

As shown in fig. 3, according to an aspect of the present application, the process of constructing the pre-trained coarse segmentation module based on the migration learning in the step S2 is further:

In the embodiment, suspicious crack areas are rapidly and accurately segmented from the acquired concrete surface images, and candidate targets are provided for further fine analysis. The method comprises the steps of firstly adopting a migration learning method, pre-training a rough segmentation model on a large-scale fracture image data set, and obtaining general visual characteristics of the fracture. Then, a small amount of crack samples under the target working condition are utilized, and the model is finely adjusted through means such as countermeasure learning, so that the model is quickly adapted to new imaging conditions. Meanwhile, the step also introduces an automatic screening mechanism of the key area based on displacement field analysis. The activity degree of the crack can be judged by estimating the displacement change of the crack region between the continuous image frames, and an auxiliary decision is provided for consultation of experts. By combining migration learning and displacement field analysis, the method can greatly reduce model training cost and improve the automation degree of crack screening while guaranteeing segmentation accuracy.

In some embodiments, the decision condition and threshold selection process for extracting key cracks based on the displacement abrupt regions are as follows:

displacement gradient threshold adaptive determination

Calculating a gradient magnitude plot G (x, y) =sqrt ((dD/dx)/(2+ (dD/dy)/(2)) for the dense displacement field D;

Normalizing the gradient amplitude graph G to obtain a normalized gradient amplitude graph G_norm (x, y);

And (3) adaptively determining a gradient amplitude threshold value T_grad by using an Otsu threshold segmentation algorithm, and extracting a high gradient region as a displacement mutation candidate region.

Crack region morphological filtering

Calculating geometric characteristics of the displacement mutation region R_i which is initially extracted, wherein the geometric characteristics comprise an area A_i, a perimeter P_i, an aspect ratio R_wh, a rectangle degree R_rect and the like;

Setting an area threshold T_A, a perimeter threshold T_P, an aspect ratio threshold T_R and a rectangle threshold T_rect;

The non-cracked regions of area A < T_A, or perimeter P < T_P, or aspect ratio R_wh > T_R, or rectangle R_rect > T_rect are culled.

Fracture zone consolidation and optimization

Filling voids in the crack through morphological closing operation (expansion and corrosion) in the selected suspected crack region, and smoothing the crack edge;

Combining the intermittent crack fragments by using connected domain analysis to obtain a complete key crack region;

Of the suspected fracture regions, the region with the area ratio of the first k% (e.g., 10%) is reserved as the final key fracture candidate region.

In some embodiments, in step S22, the self-calibration layer is designed to dynamically adjust the weight distribution of the source domain and the target domain features, and the model is guided to adapt to different working conditions, in step S23, a small number of crack regions with the most representation and discrimination are selected in an active learning mode based on the image entropy and the region confidence score, a small sample fine adjustment set is constructed, and the continuous self-optimization of the model is realized by repeatedly iterating the process. Aiming at the problems of model self-adaption and generalization faced by small sample segmentation of cracks, a new thought of combining self-calibration transfer learning with active learning is provided. Self-calibration transfer learning improves generalization and ensures precision by adaptively adjusting weight distribution between a source domain and a target domain, and active learning performs model fine tuning by inquiring sample selection and using a small amount of representative crack samples, so that marking cost can be remarkably saved. The combination of the two can realize efficient self-adaption and incremental learning of the model.

The segmentation model is pre-trained on a large-scale fracture image dataset (source domain) to extract a generic fracture feature representation.

And fine-tuning the pre-training model by using a small amount of crack samples (target domains) under the target working condition, and introducing a self-calibration layer to adjust the weights of the source domain and the target domain characteristics.

The distribution difference of the source domain and the target domain features is evaluated by a domain discriminator and used as a part of a loss function, and the optimization of the self-calibration layer is guided.

And predicting the target domain data by using the trimmed model, and calculating the entropy value or confidence score of each sample.

And selecting a sample with the largest prediction uncertainty (such as the highest entropy value and the lowest confidence coefficient) as an inquiry sample of the next training.

And presenting the inquiry sample to a manual annotator for annotating, adding the inquiry sample into a training set, and updating the model.

Repeating the steps, and updating the parameters of the self-calibration layer and the training sample set in each iteration.

And evaluating the performance of the model on the target domain verification set, and stopping training when a preset convergence condition or iteration number is reached.

Outputting the optimized crack segmentation model for crack detection and segmentation tasks under the target working condition.

As shown in fig. 4, according to an aspect of the present application, the step S3 is further:

In the embodiment, the authenticity and the type of the crack are automatically judged by extracting the multidimensional geometric and physical characteristics of the crack area and constructing a random forest classifier. In the aspect of feature engineering, the step is used for characterizing a crack region from three view angles, wherein geometric features (such as area and length-width ratio) are used for describing the shape of the crack, texture features (such as gray level co-occurrence matrix) are used for measuring material changes inside the crack, and displacement field features (such as displacement average) are used for reflecting the kinematic characteristics of the crack. The rich characteristic system lays a foundation for intelligent classification. In addition, the method skillfully utilizes the integrated learning advantage of the random forest, and enhances the robustness and generalization capability of the classifier through methods such as self-help sampling. Confidence assessment based on the softmax principle can give a quantified confidence level to each crack region, and provide auxiliary diagnosis comments for engineers. The step enables crack screening to go from qualitative to quantitative, and automatic judgment of crack type and severity degree is preliminarily achieved.

In some embodiments of the present invention, in some embodiments,

Random forest hyper-parameter selection: optimizing key super parameters of a random forest by using a grid search method and the like, wherein the key super parameters comprise the number n_ estimators of decision trees, the maximum tree depth max_depth, the minimum sample size min_samples_split required by node splitting and the like, evaluating the performance of super parameter combinations by adopting cross verification, such as 5-fold cross verification, and selecting the parameter combination with the optimal F1 value on a verification set.

Positive and negative sample equalization strategy: counting the proportion r_pn of positive and negative samples (real cracks/non-cracks), such as 1:10, balancing the number of the positive and negative samples by adopting an up-sampling (copying few types of samples) or down-sampling (removing most types of samples) method to make the proportion approximate to 1:1, performing random forest training on the balanced data set, and improving the recognition capability of the model on the cracks.

An ensemble learning strategy: self-service sampling (Bootstrap) is carried out on an original data set by adopting a Bagging concept, so that a plurality of training subsets are generated;

And integrating the prediction results of the plurality of subtrees, and making a final classification decision by a voting mechanism.

As shown in fig. 5, according to an aspect of the present application, the step S4 is further:

The present example performs fine analysis and trend prediction of crack regions from a morphology and dynamics perspective. By skeletonizing the crack region, the step extracts the high-order shape characteristics of the crack such as topological structure, direction distribution and the like. And the connectivity and penetrability analysis of the fracture skeleton network can reveal the change of the load transmission path and judge the weak part of the structure. In addition, continuous measurement of the width and length of the crack on the framework provides morphological basis for quantitative characterization of the crack. In the aspect of dynamics analysis, the step is to correlate a crack displacement field with a framework and evaluate pathological features such as local opening, dislocation and the like. And the independent analysis of normal and tangential displacement components of the crack can identify different damage modes such as tension, shearing and the like. In the time dimension, wavelet time-frequency analysis is introduced in the step, and the evolution process of the crack is described in a multi-scale manner by constructing a crack growth curve. By combining the morphological characteristics of the crack and the multi-physical field coupling information, the step can establish a crack development model facing the space-time process, quantitatively pre-judge the health trend of the structure and evaluate the residual service life of the structure. In summary, step S4 deeply analyzes the evolution mechanism of the crack from space to time and from static to dynamic, so that the detection analysis goes from 'visible' to 'understandable', which is a key for realizing intelligent diagnosis and prediction of the concrete structure.

In some embodiments, existing methods are primarily limited to extracting morphological features of individual fracture skeletons, lacking characterization of topological associations between multiple fractures. The graph convolution neural network is adopted to model the crack skeleton into graph structure data, semantic embedded representation of the crack skeleton nodes is learned, and high-order topological relations in the crack skeleton network, such as parallel connection, serial connection, intersection and the like, are excavated, so that the cause, development stage and trend of the crack can be researched and judged. Graph reasoning of fracture skeleton topology is sublimation of traditional geometric analysis. And constructing a graph model of the fracture skeleton network by taking the line segments of the fracture skeleton as nodes and the adjacent, crossed and other relations among the skeletons as edges. The geometric and semantic characteristics of the nodes are input, hidden layer characteristics of the nodes are updated through graph rolling operation, and topology dependence of the crack skeleton elements is learned. And establishing hierarchical graph representation through mechanisms such as graph pooling, graph attention and the like, and finally learning the topological structure of the crack skeleton network through tasks such as graph classification, graph matching and the like.

And carrying out morphological refinement on the separated crack region, and extracting a crack skeleton line.

And performing topological coding on the skeleton line by using a skeleton tracking algorithm (such as Zhang-Suen refinement), and extracting topological features such as endpoints, intersection points and the like.

And decomposing the skeleton line into line segments according to topological characteristics, wherein each line segment represents one crack branch.

Each crack branch is regarded as a node of the graph, and the geometric attribute (such as length and direction) and the semantic attribute (such as width and depth) of the node are extracted.

And analyzing the spatial adjacent relation between the crack branches, and if the distance between the two branch end points is smaller than a threshold value, establishing an undirected edge between the corresponding nodes.

And organizing a fracture skeleton network by a graph structure, and initializing node characteristics and an adjacent matrix.

A design drawing convolutional neural network (GCN) model is input with node characteristics and an adjacency matrix of a fracture skeleton drawing.

Multiple layers of graph convolution layers are stacked, and each layer updates node characteristics by aggregating characteristic information of node neighbors.

In the process of graph convolution, weights are adaptively distributed to different neighborhoods through an attention mechanism, and key topological dependencies among crack branches are extracted.

And introducing a graph pooling operation at the top layer of the GCN model, and aggregating node characteristics into characteristics at the level of subgraph.

And converting the fracture skeleton diagram into a layered diagram representation by recursively executing diagram pooling, and describing topological structures under different scales.

After each pooling layer a graph attention module is inserted, which adaptively selects important sub-graph regions by attention weights.

And designing corresponding graph task layers, such as graph classification, graph regression, graph matching and the like, aiming at different reasoning targets.

And inputting the hierarchical graph features extracted by the GCN into a graph task layer, and learning a high-level semantic representation of the crack skeleton topology.

And (3) through end-to-end training, optimizing the loss function of the graph task layer, and realizing reasoning of the topological relation of the crack skeleton.

And classifying or regressing the topological structure of the crack framework by utilizing the output of the graph task layer, and predicting the cause, development stage and the like of the crack.

Through visualization of the attention weights of the graph, interactions and importance levels between different fracture branches are analyzed.

And (3) correlating the topological relation with the crack attribute to form comprehensive judgment on the crack development trend.

In some embodiments, the wavelet basis function is selected: according to the time-frequency characteristics of the crack development process, proper wavelet basis functions such as Daubechies wavelet (db), symlets wavelet (sym) and the like are selected, and the order of the wavelet basis is selected to be suitable for the scale and smoothness of the crack development, such as db4, sym6 and the like.

Multiscale wavelet decomposition: discrete wavelet transformation is carried out on the crack development time sequence x (t) to obtain wavelet coefficients w (a, b), wherein a is a scale parameter, b is a time shift parameter, a proper decomposition layer number n (such as 3-5 layers) is selected, and multi-scale decomposition is carried out on the wavelet coefficients to obtain low-frequency coefficients and high-frequency coefficients of each layer.

Wavelet energy spectrum analysis: the method comprises the steps of calculating the energy E_i of wavelet coefficients under each scale, reflecting the intensity distribution of crack development under different scales, drawing a wavelet energy spectrogram, analyzing the multi-scale characteristics of crack development, and identifying key time nodes.

Wavelet coefficient thresholding: and the threshold value can be adaptively determined according to the statistical characteristics of the wavelet coefficients, such as a 3-time standard deviation criterion and the like.

Wavelet reconstruction and inverse transformation: and (3) carrying out inverse transformation on the processed wavelet coefficient, reconstructing a crack development time sequence x' (t), and segmenting the reconstructed time sequence to divide different development stages of the crack.

In some embodiments, the phase space reconstruction is performed on the crack development time sequence, and chaotic dynamics parameters such as time delay, embedding dimension and the like are extracted. And carrying out quantitative analysis of Lyapunov indexes, associated dimensions, entropy and the like on the basis to characterize the nonlinear and non-stationary characteristics of crack development. Further considering the dynamics mechanism of the crack under the coupling of multiple physical fields such as stress field, temperature field and the like, establishing a chaotic dynamics model under the constraint of a partial differential equation, and predicting bifurcation and mutation behaviors of crack development.

And carrying out phase space reconstruction on a one-dimensional time sequence of crack development (such as a change curve of the length and the width of the crack along with time). And adopting a delay coordinate embedding method, selecting proper time delay tau and embedding dimension m, and mapping the time sequence to a phase space. In the m-dimensional phase space, the crack development track is embodied as a curve, and the dynamic characteristics of system evolution are reflected.

And calculating a large Lyapunov index (MLE) of the reconstructed phase space track, and quantifying the chaos degree of crack development. MLE >0, which indicates that crack development is sensitive to initial conditions and has chaotic characteristics.

And calculating the correlation dimension and describing the fractal structure complexity of the attractor. The correlation dimension reflects self-organization and emerging behavior during crack development. And calculating sample entropy, displacement entropy and the like, and measuring uncertainty and randomness of crack development. The larger the entropy value, the more difficult it is to predict the dynamic behavior of crack development.

And (3) mechanism constraint modeling, namely introducing concrete fracture mechanics and damage evolution equations, such as Paris fracture expansion law, and describing the relation between a fracture tip stress field and expansion rate. And (3) establishing a multi-scale fracture model by considering the mesoscopic structural characteristics (such as aggregate distribution, porosity and the like) of the concrete. And fusing the influence of environmental factors (temperature, humidity and the like) on crack propagation to construct a multi-field coupling fracture dynamics equation.

And (3) data-mechanism fusion, namely assimilating the characteristic quantity obtained by chaotic time sequence analysis with a fracture mechanics model by adopting methods such as ensemble Kalman filtering, particle filtering and the like. The chaos characteristic is used as a state variable and is introduced into a fracture dynamics equation to correct and update model parameters. Through collaborative learning, mutual promotion and constraint between the chaotic characteristics based on data and the fracture behaviors based on mechanisms are realized.

And (3) dynamic prediction, namely carrying out multi-step forward prediction of crack development on the basis of a fusion model. And extracting an initial state from the historical data of the chaotic time series as a starting condition of a fracture dynamics equation. And (5) iteratively solving a fracture equation to obtain a fracture state variable at a future moment. Mutation behavior of crack development was predicted by bifurcation analysis. And (3) solving bifurcation points of the fracture equation, and judging critical conditions of the crack from stable expansion to instability. And performing Monte Carlo simulation, and generating a probability prediction interval of crack development by considering random uncertainty of material parameters and environmental factors.

In some embodiments, multi-view image acquisition: the method comprises the steps of selecting proper camera parameters such as focal length, aperture, depth of field and the like, ensuring clear imaging of a crack region, planning shooting positions and angles of a camera according to field environment conditions, preferably selecting a plurality of view angles such as 45 degrees and 90 degrees, and controlling the overlapping degree between adjacent view angles to be more than 60 percent to obtain sufficient matching characteristic points.

Feature point extraction and matching: and performing cross-view angle characteristic point matching by using metrics such as Euclidean distance, hamming distance and the like to obtain a homonymous point pair.

Calibrating camera parameters: according to the imaging principle, the Zhang's calibration method, the beam adjustment method and the like are adopted to calibrate the parameters (focal length, principal point and distortion coefficient) in the cameras of each view angle, and the external parameters (rotation matrix R and translation vector T) between the cameras are estimated through the epipolar geometric constraint or PnP algorithm by combining the homonymy point pairs.

Sparse point cloud generation: and refining the coordinates of the three-dimensional points by using the pixel coordinates corresponding to the homonymous points through least square optimization and the like.

Point cloud registration and optimization: and (3) based on the sparse point cloud, registering among the dense point clouds obtained by scanning at different view angles through registration algorithms such as ICP, point-to-surface ICP and the like, evaluating the point cloud superposition accuracy, eliminating outliers and ensuring the integrity and the authenticity of the three-dimensional model.

In some embodiments, conventional methods reconstruct the three-dimensional morphology of the fracture using simplified skeleton lines and cross-sections, with insufficient texture detail of the fracture surface. The method comprises the steps of firstly introducing deep poisson curved surface reconstruction into crack three-dimensional reconstruction, extracting implicit expression of point cloud through deep learning, and learning detailed characteristics of a crack surface. And combining the fairing constraint of the poisson reconstruction, a crack three-dimensional model with precise geometry and vivid texture can be obtained. In step S54, the skeleton line and the cross section sketch are used as a priori, the point cloud of the crack region is input, and the hidden function is solved through poisson reconstruction, so as to obtain an initial crack curved surface. On the basis, a deep learning module is introduced to extract local features of the point cloud, a complex hidden function is solved in an end-to-end mode through a higher-order term in a neural network regression Poisson equation, and a fine crack three-dimensional model is generated. The loss function fuses the data item and the fairing item, balancing fidelity and continuity.

The three-dimensional fine characterization process of the crack based on the reconstruction of the depth poisson curved surface is as follows:

In step S51, local point clouds obtained by scanning at different view angles are registered under a unified coordinate system through ICP, point-plane registration and other algorithms. And screening the feature points with high confidence, optimizing the registration matrix, and realizing the accurate splicing of the point cloud.

And (3) guiding the skeleton priori, namely projecting the crack skeleton line extracted in the step S53 into a three-dimensional point cloud to serve as the shape priori of a crack region. And taking the skeleton line as a center, and extracting a point cloud segment of the crack cross section in a local adjacent area.

Poisson reconstruction, namely uniformly sampling seed points on a crack skeleton line, and gradually growing and reconstructing a curved surface from a point cloud by taking a normal vector as a guide. And (3) constructing a poisson equation, solving a hidden function, and fitting a gradient field of the poisson equation with a point cloud normal vector field to obtain a smooth and watertight initial curved surface. And (3) applying soft constraint of the skeleton shape at the seed points to enable the reconstructed curved surface to be attached to the skeleton line.

And (3) optimizing local detail, namely subdividing an initial curved surface obtained by poisson reconstruction into local fragments, and dividing by taking a skeleton line as a reference. On each local segment, fine surface detail is captured by moving a least squares fit local hidden function. And interpolating the local hidden function obtained by fitting to a global grid, and refining the initial curved surface.

Deep poisson reconstruction, namely initializing the geometric representation of the deep neural network by using the fairing curved surface M obtained by poisson reconstruction. The network input is a local point cloud mass and the output is an implicit representation of the corresponding surface block (e.g., SDF symbol distance field). And extracting the local characteristics of the point cloud through the full convolution network. Features are input into the fully connected layer, predicting the symbol distance at each point.

The loss function includes a data error term that minimizes the L1 loss between the predicted symbol distance and the approximation of M. And (3) smoothing regularization term, namely minimizing the difference of the normal vector between adjacent surface blocks. The full convolution feature extraction and full connection prediction are trained alternately, and the implicit curved surface is optimized end to end.

Texture mapping, namely mapping color image information onto a reconstructed crack curved surface, and giving a sense of realism. And projecting the image pixels to a three-dimensional curved surface through the camera pose, and establishing UV mapping. And optimizing texture coordinates.

at least one processor; and

A memory communicatively coupled to at least one of the processors; wherein,

The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to the specific details of the above embodiments, and various equivalent changes can be made to the technical solution of the present invention within the scope of the technical concept of the present invention, and all the equivalent changes belong to the protection scope of the present invention.

Claims

1. The AI visual detection method for identifying and analyzing the cracks of the concrete structure is characterized by comprising the following steps:

S4, extracting a crack skeleton according to each type of crack region image and the monitoring data of the corresponding space time of the crack region image, quantitatively analyzing a crack displacement field, and giving out a crack development trend analysis result and outputting the analysis result;

The step S4 is further:

2. The AI visual inspection method for concrete structure crack identification and analysis according to claim 1, wherein the step S1 is further:

3. The AI visual inspection method for concrete structure crack identification and analysis according to claim 1, wherein the process of constructing the pre-trained coarse segmentation module based on transfer learning in step S2 is further:

4. The AI visual inspection method for identifying and analyzing a crack of a concrete structure according to claim 3, wherein in the step S2, a crack displacement field is extracted based on the crack region image, the crack region image is screened by the crack displacement field, and the process of obtaining the key crack region image is further as follows:

5. The AI visual inspection method for concrete structure crack identification and analysis as set forth in claim 4, wherein said step S3 is further:

6. The AI visual inspection method for concrete structure crack identification and analysis as set forth in claim 1, wherein said step S46 further includes:

7. The AI visual inspection method for concrete structure crack identification and analysis as set forth in claim 6, further comprising the step of S5, three-dimensionally reconstructing a crack region,

step S53, obtaining a central curve skeleton of the crack point cloud through a Chen shrinkage algorithm; the skeleton reflects the topological structure of the crack; a central curve skeleton is taken as a reference, a tangent plane is locally established in a crack, a point set in the plane is extracted as a crack cross section, the cross section shape is obtained through curve fitting, and a series of cross section curves are obtained through continuous reconstruction along a skeleton line;

8. The AI visual inspection method for concrete structure crack identification and analysis according to claim 7, wherein in step S2, the process of constructing a pyramid coarse segmentation module of a multiscale fused attention mechanism is further as follows:

9. The AI visual inspection method for concrete structure crack identification and analysis as set forth in claim 7, wherein said step S44 further includes:

10. An AI visual inspection system for crack identification and analysis of a concrete structure, comprising:

at least one processor; and

A memory communicatively coupled to at least one of the processors; wherein,

The memory stores instructions executable by the processor for execution by the processor to implement the AI visual detection method for concrete structure crack identification and analysis of any one of claims 1 to 9.