CN108257135A

CN108257135A - The assistant diagnosis system of medical image features is understood based on deep learning method

Info

Publication number: CN108257135A
Application number: CN201810103893.0A
Authority: CN
Inventors: 胡海蓉
Original assignee: Zhejiang De Image Solutions Co ltd
Current assignee: Zhejiang De Image Solutions Co ltd
Priority date: 2018-02-01
Filing date: 2018-02-01
Publication date: 2018-07-06

Abstract

The present invention relates to complementary medicine diagnostic fields, it is desirable to provide the assistant diagnosis system of medical image features is understood based on deep learning method.The assistant diagnosis system that medical image features are understood based on deep learning method includes process：It reads the medical image data of lesion and is pre-processed；Image is chosen, establishes convolutional neural networks framework, automatic study is partitioned into focal area, and lesion shape is refined；The CNN models for building a convolutional neural networks framework again understand good pernicious focus characteristic automatically, and the assistant diagnosis system that medical image feature is understood based on deep learning method is obtained after training.The present invention can not be partitioned into focal area automatically only by depth convolutional neural networks, the deficiency of weak boundary cannot be solved the problems, such as based on active contour etc. by compensating for, and can learn to extract valuable feature combination automatically, avoid the complexity of artificial selected characteristic.

Description

The assistant diagnosis system of medical image features is understood based on deep learning method

Technical field

The present invention relates to complementary medicine diagnostic fields, more particularly to understand medical image features based on deep learning method Assistant diagnosis system.

Background technology

In recent years, with the rapid development of computer technology and digital image processing techniques, digital image processing techniques are got over Come it is more be applied to complementary medicine diagnostic field, principle is exactly to being divided by the medical image that different modes obtain It cuts, reconstructs, be registrated, the image processing techniques such as identification, so as to obtain valuable medical diagnostic information, main purpose is to make doctor Raw observation diseased region is more directly and clear, and auxiliary reference is provided for doctor's clinical definite, and there is very important reality to anticipate Justice.

Based on medical image, find lesion to differentiating that its good pernicious, clinical treatment and surgical selection are significant early. And based on the ultrasonic examination of ultrasonic imaging technique because can real time imagery, inspection fee it is relatively low, to sufferer hurtless measure etc.. It is widely used in clinical diagnosis.And the good pernicious master of diagnosis lesion (such as thyroid nodule, Breast Nodules, lymph node etc.) By puncturing living tissue cells inspection, such workload can be very big, is additionally present of situation about excessively detecting, and doctor understands Features of ultrasound pattern subjectivity is stronger, and mainly by experience, result suffers from the imaging mechanism of medical imaging devices, obtains Condition shows the influences of factors such as equipment and easily causes mistaken diagnosis or fail to pinpoint a disease in diagnosis.Therefore, using computer technology, at digital picture Reason technology, statistical method etc. realize that ultrasonoscopy auxiliary diagnosis is very necessary.But intrinsic image-forming mechanism causes clinical acquisitions The ultrasonograph quality arrived is poor, and the accuracy and automation for leading to auxiliary diagnosis are affected, so current segmentation Lesion in ultrasonoscopy it is most be the semi-automatic segmentation based on active contour, classification mainly manually selects feature, so The Classification and Identifications such as traditional machine learning method support vector machines (SVM), K- neighbours (KNN), decision tree, these classification are utilized afterwards Device can only can have preferable effect to Small Sample Database.But almost without real understanding medical image, such auxiliary system System is undoubtedly a flight data recorder for final user.And medical data is magnanimity, the Classification and Identification of large sample, Especially the deciphering of characteristics of image can just have better booster action to medical diagnosis.

Invention content

It is a primary object of the present invention to overcome deficiency of the prior art, provide a kind of based on the deciphering of deep learning method The assistant diagnosis system of medical image features.In order to solve the above technical problems, the solution of the present invention is：

The assistant diagnosis system that medical image features are understood based on deep learning method is provided, including following processes：

First, the medical image data of lesion is read：

The medical image (can be picture format or the dicom pictures of standard) of lesion is read, including at least The image of 10000 benign lesions and the image of at least 10000 pernicious lesions；

2nd, medical image is pre-processed：

The lesion image that process one is read first is carried out image gray processing, and is removed using the gray value of surrounding pixel point Doctor is to measure the label that tubercle correlative is done in ultrasonoscopy, recycles gaussian filtering denoising, finally utilizes grey level histogram Equalization enhancing contrast, obtains pretreated enhancing image；

3rd, image is chosen, establishes first convolutional neural networks framework, i.e. CNN (convolutional neural Network), automatic study is partitioned into focal area, referred to as area-of-interest, i.e. ROI (region of interest), and right Lesion shape is refined；Specifically include following step：

1st step：The pretreated enhancing image 20000 of selection process two is opened, the image each 10000 including good pernicious lesion ；

2nd step：To each pictures, area-of-interest, i.e. focal area are sketched out manually (by expert) first；Then lead to It crosses first CNN framework and trains automatic parted pattern, it is SegCNN models to remember this automatic parted pattern；

The network structure that the SegCNN models are made of 15 layers of convolutional layer, 4 layers of down-sampling layer；The convolution of each convolutional layer The size of core is respectively：First layer is 13 × 13, and the second layer is 11 × 11 with third layer, and the 4th layer is 5 × 5, remaining each layer is 3 ×3；The step-length of convolutional layer is respectively：First convolutional layer is 2, remaining is all 1；The size of down-sampling layer is all 3 × 3, step Length is all 2；

3rd step：It is applied to all lesion images using the SegCNN models that the 2nd step obtains, i.e., the 1st step is chosen 20000 images are divided automatically, then establish a figure and cut model, and the focal area that SegCNN models obtain is carried out certainly Dynamic refinement segmentation, finally obtains ROI, i.e., all good pernicious lesions；

4th, the CNN models for establishing second convolutional neural networks framework understand good pernicious focus characteristic automatically, remember this CNN models are RecCNN models；

The network structure that the RecCNN models are made of 6 layers of convolutional layer, 4 layers of down-sampling layer, 3 layers of full articulamentum；3 The neuron node number of full articulamentum is respectively 4096,4096,1；The size of the convolution kernel of each convolutional layer is respectively：First layer is 13 × 13, the second layer is 11 × 11 with third layer, and the 4th layer is 5 × 5, remaining each layer is 3 × 3；The step-length of convolutional layer is respectively： First convolutional layer is 2, remaining is all 1；The size of down-sampling layer is all 3 × 3, and step-length is all 2；

The ROI that three SegCNN models of process are partitioned into automatically is divided into p groups, for training RecCNN models (i.e. The training RecCNN models of journey five：Feature is extracted, and normalizing is carried out to characteristic using ROI of the RecCNN models to each group Change, i.e., using the feature of each group of ROI of RecCNN model extractions, then these features to extracting carry out linear transformation, make it End value is mapped to [0,1])；The p is the positive integer not less than 2；

5th, p-1 groups data in process four are selected and make training set, training set is used to train RecCNN models, one group of residue Data make test set, and test set is used to test trained RecCNN models；

RecCNN models are trained using training set, for understanding medical image features, can automatically be split to all Focal area extraction feature (specific training process is：It extracts the method for feature divides automatically with three SegCNN models of process The method of middle extraction characteristic procedure is the same, i.e., is all to extract feature by respective each convolutional layer and pond layer, this two The effect of class functional layer is the same, and calculation formula and update method are the same, but the object of RecCNN models is For focal area, and automatic partitioning portion is to be carried out at the same time extraction feature with focal area for non-focal area, and RecCNN models and the convolutional layer of SegCNN models and pond layer window size, step sizes, filling size setting is different, so Mutual convolutional layer is different from pond layer sphere of action)；

Then polytypic grader can be carried out by constructing one using Softmax, and the feature extracted is analyzed, This process is to solve the optimal value of a loss function, that is, optimizes loss function

Wherein, the i refers to i-th of sample；The j refers to jth class；The l refers to l classes；The m represents shared m A sample, m value ranges are arbitrary positive integer；The c represents that these samples can be divided into c classes in total, and c value ranges is arbitrarily just Integer；It is describedIt is a matrix, is the parameter corresponding to a classification, i.e. weight and biasing per a line；The θ^T _jIt is Refer to the transposition of the parameter vector of jth class, the θ^T _lRefer to the transposition of the parameter vector of l classes, the θ_ijRefer to parameter matrix The element of i-th row jth；1 { } is an indicative function, i.e., when the value in braces is true, the result of the function is 1, otherwise as a result 0；The λ is the parameter for balancing fidelity item (first item) and regular terms (Section 2), and λ takes positive number here (its size is adjusted according to experimental result)；The e represents Euler's numbers 2.718281828, e^xIt is exactly exponential function；The T is table Show the transposition operator in matrix calculating；Log represents natural logrithm, i.e., using Euler's numbers as the logarithm at bottom；x⁽ⁱ⁾It is input vector I-th dimension；y⁽ⁱ⁾It is the i-th dimension of each sample label；

The classification number c of Softmax graders is equal to 5, i.e., the echo characteristics, edge feature, structure for representing lesion respectively are special Sign, calcification feature, five category feature of aspect ratio features；Each class has different subclasses, and echo characteristics has four subclasses：High echo, Equal echo, low echo or extremely low echo, echoless；Edge feature has two subclasses：Finishing, not finishing；Structure feature has four sons Class：Based on reality, reality, based on capsule, capsule；Calcification feature has two subclasses：Microcalcification, without Microcalcification；Aspect ratio features have Two subclasses：More than 1, less than or equal to 1；Then, which the feature vector exported by stochastic gradient descent method is belonging respectively to (detailed process is similar with Forecasting Methodology in cutting procedure automatic in process three to the probability of the subclass of category feature, is all optimization One loss function, only here be a polytypic Softmax function, which is subordinate to according to output feature vector As soon as the probabilistic forecasting of category feature goes out a tag along sort, also the feature of a lesion is classified, can further be obtained The type of the corresponding each feature of good pernicious lesion)；

6th, repetitive process five, do p crosscheck, i.e., the p group data divided for process four select one group not every time Same data make test set, and remaining p-1 groups data make training set, until each group of data all made test set；

By p crosscheck, the weight and offset parameter of convolutional neural networks model RecCNN can be preserved every time, and According to the accuracy rate assessment result on test set；The calculation formula of accuracy rate isWherein AC represents accuracy rate, The correct sample number of TN presentation classes, the sample number of FN presentation class mistakes；Finally take the highest primary crosscheck of accuracy rate In weight and offset parameter, as the optimal parameter of RecCNN models, obtained trained RecCNN models, i.e., it is final really The assistant diagnosis system based on deep learning method deciphering medical image feature is determined；

The lesion image for needing to understand is input to this auxiliary based on deep learning method deciphering medical image feature Diagnostic system, you can obtain the feature of the lesion, and analyzed, and then can be diagnosed according to these features per category feature it Good pernicious lesion.

In the present invention, the 2nd step and the 3rd step in the process three, specially：

(1) by the convolutional layer of CNN and the automatic learning characteristic of down-sampling layer, and feature is extracted, the specific steps are：

Step A：In a convolutional layer, the feature maps of last layer carries out convolution by a convolution kernel that can learn, so As soon as output feature map can be obtained by an activation primitive afterwards；Each output is one input of convolution nuclear convolution or combination The value of multiple convolution inputs (what we selected here is the value for combining the multiple maps that come in and go out of convolution)：

Wherein, symbol * represents convolution operator；The l represents the number of plies；The i represents l-1 layers of i-th of neuron section Point；The j represents l layers of j-th of neuron node；The M_jRepresent the set of the input maps of selection；It is describedRefer to L-1 layers of output, it is described as l layers of inputRefer to j-th of component of l layers of output；The f is activation primitive, this In take sigmoid functionsAs activation primitive, e represents Euler's numbers 2.718281828, e^xIt is exactly exponential function； The k is convolution operator, the k^l _ijRefer to the element of (i, j) position of l layers of convolution kernel；The b is biasing, describedIt is Refer to j-th of component of l layers of biasing；Each output map can give an additional biasing b, but specific defeated for one Go out map, the convolution kernel that convolution each inputs maps is different；

Calculated by gradient, to update sensitivity, sensitivity for how much representing b variations, error can change how much：

Wherein, the l represents the number of plies；The j represents l layers of j-th of neuron node；It is describedRepresent each element phase Multiply；The δ represents the sensitivity of output neuron, that is, biases the change rate of b, describedRefer to j-th point of l layers of sensitivity Amount, the δ^l+1 _jRefer to j-th of component of l+1 layers of sensitivity；The s^l=W^lx^l-1+b^l, x^l-1Refer to l-1 layers of output, W For weight, b is biasing, describedRefer to l layers of s^l=W^lx^l-1+b^lJ-th of component, the W^lRefer to l layers of weight ginseng Number, the b^lRefer to l layers of biasing；The f is activation primitive, takes sigmoid functions hereAs activation Function, e represent Euler's numbers 2.718281828, e^xIt is exactly exponential function；F " (x) is the derived function of f (x) (i.e. if f takes Sigmoid functionsThen f'(x)=(1-f (x)) f (x))；It is describedRepresent the weights that each layer is shared；It is described Up () represents that (if the decimation factor of down-sampling is n, up-sampling operation is exactly by each pixel for a up-sampling operation Both horizontally and vertically upper copy n times, can thus restore original size)；

Then it sums to all nodes in the sensitivity map in l layers, the quick gradient for calculating biasing b：

Wherein, the l represents the number of plies；The j represents l layers of j-th of neuron node；The b represents biasing, described Refer to j-th of component of l layers of biasing；The δ represents the sensitivity of output neuron, that is, biases the change rate of b；The u, V represents (u, v) position of output maps；It is describedU, v refer to the element of l layers of sensitivity (u, v) position；The E is to miss Difference function, here(if the problem of two classification, then label can for the dimension of the C expressions label To be denoted as y_h∈ { 0,1 }, C=1, can also be denoted as y at this time_h∈ { (0,1), (1,0) }, at this time C=2)；It is describedIt represents n-th The h dimensions of sample corresponding label；It is describedRepresent h-th of output of the corresponding network output of n-th of sample；

Finally using Back Propagation Algorithm, stochastic gradient descent is carried out to loss function, calculates the weights of convolution kernel：

Wherein, the W is weight parameter, and the △ W refer to the knots modification of weight parameter；The W^lRefer to l layers of power Weight parameter；The E is error function, andThe C represents the dimension of label (if two classification are asked Topic, then label can be denoted as y_h∈ { 0,1 }, C=1, can also be denoted as y at this time_h∈ { (0,1), (1,0) }, at this time C=2)；Institute It statesRepresent the h dimensions of n-th of sample corresponding label；It is describedRepresent h-th of the output of n-th of sample corresponding network it is defeated Go out；The η is learning rate, i.e. step-length；Since the weights much connected are shared, for a given weights, need Gradient is asked to the point with the weights associated connection to all, then sum to these gradients：

Wherein, the l represents the number of plies；The i represents l layers of i-th of neuron node；The j represents j-th of l layers Neuron node；B represents biasing, and the δ represents the sensitivity of output neuron, that is, biases the change rate of b；The u, v are represented Export (u, v) position of maps；It is describedU, v refer to the element of (u, v) position of l layers of sensitivity；The E is error letter Number, here(if the problem of two classification, then label can be remembered the dimension of the C expressions label For y_h∈ { 0,1 }, C=1, can also be denoted as y at this time_h∈ { (0,1), (1,0) }, at this time C=2)；It is describedRepresent n-th of sample The h dimensions of corresponding label；It is describedRepresent h-th of output of the corresponding network output of n-th of sample；It is describedIt is convolution kernel； It is describedIt isIn element when convolution withBy the patch of element multiplication, i.e., all and convolution kernel size All region units in identical picture, the value of (u, v) position of output convolution map is by last layer (u, v) position Patch and convolution kernelBy the result of element multiplication；

Step B：Down-sampling layer has N number of input maps, just there is N number of output maps, and only each output map becomes smaller, Then have：

Wherein, it is describedRefer to j-th of component of l layers of output, the X^l-1 _jRefer to the jth of l-1 layers of output A component；The f is activation primitive, takes sigmoid functions hereAs activation primitive, e represents Euler's numbers 2.718281828 e^xIt is exactly exponential function；It is describedRepresent the weights that each layer is shared；The down () represents a down-sampling Function；It is describedRefer to j-th of component of l layers of biasing；The all pixels of the block of different n × n of input picture are asked With export image in this way and all reduce n times on two dimensions, n value ranges are positive integer (here exactly by input picture Each element takes the block of fixed 3 × 3 sizes, and then wherein all elements are summed as the element in the output image Value, so that output image all reduces 3 times on two dimensions)；Each output map corresponds to an one's own power Weight parameter beta (biasing of multiplying property) and an additivity biasing b；

By gradient descent method come undated parameter β and b：

Wherein, the f " (x) refers to the derivative of activation primitive f (x)；It is describedRepresent each element multiplication；The conv2 It is two-dimensional convolution operator；The rot180 is rotation 180 degree；It is described ' full ' refer to carry out complete convolution；The l expression layers Number；The i represents l layers of i-th of neuron node；The j represents l layers of j-th of neuron node；The b represents biasing, The b_jRefer to the jth component of offset parameter；The δ represents the sensitivity of output neuron, that is, biases the change rate of b, describedRefer to j-th of component of l layers of sensitivity, the δ^l+1 _jRefer to j-th of component of l+1 layers of sensitivity；The u, v are represented Export (u, v) position of maps；It is describedU, v refer to the element of (u, v) position of l layers of sensitivity；The E is error letter Number, expression formula are same as above, i.e.,The C represent label dimension (if the problem of two classification, then label Y can be denoted as_h∈ { 0,1 }, C=1, can also be denoted as y at this time_h∈ { (0,1), (1,0) }, at this time C=2)；It is describedIt represents The h dimensions of n-th of sample corresponding label；It is describedRepresent h-th of output of the corresponding network output of n-th of sample；The β is Weight parameter (general value is in [0,1]), the β_jRefer to j-th of component of weight parameter；The down () is represented under one Sampling function；It is describedIt is l+1 layers of convolution kernel；It is describedJ-th of the neuron node of the output of l-1 layers for being；Institute State s^l=W^lx^l-1+b^l, wherein W is weight parameter, and b is biasing,It is s^lJ-th of component；

Step C：The combination of the automatic learning characteristic map of CNN, then j-th of feature map be combined as：

s.t.∑_iα_ij0≤α of=1, and_ij≤1.

Wherein, symbol * represents convolution operator；The l represents the number of plies；The i represents l layers of i-th of neuron node； The j represents l layers of j-th of neuron node；The f is activation primitive, takes sigmoid functions hereAs Activation primitive, e represent Euler's numbers 2.718281828, e^xIt is exactly exponential function；It is describedIt is i-th point of l-1 layers of output Amount, it is describedRefer to j-th of component of l layers of output；The N_inRepresent the map numbers of input；It is describedIt is convolution kernel；Institute It statesIt is biasing；The α_ijRepresent l-1 layer when exporting map as l layers of input, l-1 layers obtain j-th output map's Wherein i-th weights for inputting map or contribution；

(2) focal area is automatically identified using the feature combination Softmax extracted in step (1), exports the general of segmentation Rate figure determines the model divided automatically；As soon as specific Softmax identification process is exactly given sample, a probability is exported Value, what which represented is that this sample belongs to several probability of classification, and loss function is：

Wherein, the i refers to i-th of sample；The j refers to jth class；The l refers to l classes；The m represents shared m A sample, m value ranges are arbitrary positive integer；The c represents that these samples can be divided into c classes in total, and c value ranges is arbitrarily just Integer；It is describedIt is a matrix, is the parameter corresponding to a classification, i.e. weight and biasing per a line；The θ^T _jIt is Refer to the transposition of the parameter vector of jth class, the θ^T _lRefer to the transposition of the parameter vector of l classes, the θ_ijRefer to parameter matrix The element of i-th row jth；1 { } is an indicative function, i.e., when the value in braces is true, the result of the function is 1, otherwise as a result 0；The λ is the parameter for balancing fidelity item (first item) and regular terms (Section 2), and λ takes positive number here (its size is adjusted according to experimental result)；The J (θ) refers to the loss function of system；The e represents Euler's numbers 2.718281828 e^xIt is exactly exponential function；The T is the transposition operator during representing matrix calculates；Log represents natural logrithm, I.e. using Euler's numbers as the logarithm at bottom；x⁽ⁱ⁾It is the i-th dimension of input vector；y⁽ⁱ⁾It is the i-th dimension of each sample label；Then ladder is utilized Degree solves：

Wherein, the θ^T _j、i、j、c、l、θ^T _lIt is respectively identical meaning with what is represented in above-mentioned loss function J (θ)；The m represents to share m sample；It is describedIt is a square Battle array is the parameter corresponding to a classification, i.e. weight and biasing per a line；The θ_jRefer to the parameter corresponding to jth class；It is described 1 { } is an indicative function, i.e., when the value in braces is true, the result of the function is 1, otherwise as a result 0；Institute It is the parameter for balancing fidelity item (first item) and regular terms (Section 2) to state λ, and λ takes positive number (to adjust it according to experimental result here Size)；The J (θ) refers to the loss function of system；It is J (θ) derived function；The e represents Euler's numbers 2.718281828 e^xIt is exactly exponential function；The T is the transposition operator during representing matrix calculates；Log represents natural logrithm, I.e. using Euler's numbers as the logarithm at bottom；x⁽ⁱ⁾It is the i-th dimension of input vector；y⁽ⁱ⁾It is the i-th dimension of each sample label；(used here as Be a kind of new Softmax graders, i.e., the Softmax graders of only two classification, for a medical image, root The probability provided according to softmax can obtain a probability graph for distinguishing all focal areas and non-focal area, root Figure can obtain the coarse segmentation to focal area accordingly)

(3) using the medical image of the automatic divided ownerships of SegCNN, that is, focal area and non-focal area is distinguished, is found The boundary of focal area, and the lesion shape being partitioned into is refined, it is point refined using the method that figure is cut here It cuts, is exactly specifically：Remember I:X ∈ V → R is are defined on regionOn 2D ultrasound image datas, S is all pixels in V The set of point, Nx is the 6- neighborhood point sets of pixel x；Assuming that l_x∈ { 0,1 } is the label of pixel x, wherein 0 and 1 difference It represents the pixel and belongs to background (non-focal area) and prospect (focal area)；Then need the energy found below minimization general Tally set l={ the l of letter_x, x ∈ S },

WhereinParameter lambda is used for adjusting data penalty term E_D(l) and boundary penalty term E_B(l) balance between, λ value ranges are arbitrary real number；The V refers to the regional extent of image；Area item D_x(l_x) for retouching State the similarity of pixel x and prospect or background；Edge detection function B_xy(x, y) features not connecting between pixel x and y Continuous property, andβ is constant term, and the I (x) refers to the gray value at pixel x on image, institute It states I (y) and refers to gray value on image at pixel y；Next, define a gray threshold function：

Wherein, the ζ refers to pixel minimum gradation value in focal area, and the η refers to the maximum of pixel in focal area Gray value；The gray value interval of lesion thus can be roughly estimated from initial focal areaDefinition is by one group of spy The part characterization item that sign distribution is formed, the feature of selection have the gray value I (x) of image, improved local binary patternsWith Local gray level variance VAR_P,r；These features are combined into a union featureτ, P, r are Normal number；Here

Wherein I_p(p=0,1 ..., P-1) is corresponding to be generally evenly distributed in using c ∈ Ω as the center of circle, and r is on the circle of radius The gray value of P point, I_cIt is the gray value of circle centre position；The I_mRefer to using c ∈ Ω as the center of circle, r is P point on the circle of radius The mean value of gray value, the sign refer to sign function, and when x is more than 0, sign (x) is more than 0, and otherwise sign (x) is less than 0；H(x) It is Heaviside functions, i.e.,

NoteAccumulation histogram for ith features of the pixel x in local neighborhood O (x)；It is that ith feature exists Average accumulated histogram in initialization area, variance are denoted asThen it locally characterizes item and can be defined asW¹() is one-dimensional L1Wasserstein distances；The focal zone that last combination S egCNN is obtained The segmentation probability graph L (x) in domain, gray threshold function F (x) and part characterization P (x), obtain data item expression formula D_x(l_x) be：

D_x(l_x)=max (- R (x), 0) l_x+max(R(x),0)(1-l_x)

Hereγ is normal number；The max refers to take maximum Value；So as to which the figure for having obtained that focal area can be carried out refinement segmentation cuts model, using this figure cut model can to by The focal area that SegCNN models obtain carries out refinement segmentation.

Compared with prior art, the beneficial effects of the invention are as follows：

The present invention can not be partitioned into focal area automatically only by depth convolutional neural networks, compensate for based on activity Profile etc. cannot solve the problems, such as the deficiency of weak boundary, and can learn to extract valuable feature combination automatically, avoid The complexity of artificial selected characteristic, the feature extracted in this way are more advantageous to finding the main rule information of lesion, and right Ultrasonic image feature is classified, and can objectively quantify main clinical medicine index, and it is good pernicious to improve diagnosis lesion Accuracy rate, and obtain the adaptability of height.

Description of the drawings

Fig. 1 is the flow chart that medical image features are understood based on depth convolutional neural networks method.

Fig. 2 is the raw ultrasound image of lesion used in embodiment.

Fig. 3 is the mask pictures of focal area in Fig. 2 that expert draws.

Fig. 4 is the raw ultrasound image of lesion in embodiment.

Fig. 5 is the effect picture for being partitioned into Fig. 4 focal areas automatically using SegCNN.

Specific embodiment

The present invention is described in further detail with specific embodiment below in conjunction with the accompanying drawings：

The following examples can make the professional technician of this profession that the present invention be more fully understood, but not with any side The formula limitation present invention.

As shown in Figure 1, a kind of assistant diagnosis system that medical image features are understood based on deep learning method, including following Step：

First, the medical image data of lesion is read：

The medical image of lesion is read, includes the image and at least 10000 pernicious lesions of at least 10000 benign lesions Image；Image can be picture format or the dicom pictures of standard.

2nd, medical image is pre-processed：

The lesion image that process one is read first is carried out image gray processing, and is removed using the gray value of surrounding pixel point Doctor is to measure the label that tubercle correlative is done in ultrasonoscopy, recycles gaussian filtering denoising, finally utilizes grey level histogram Equalization enhancing contrast, obtains pretreated enhancing image.

3rd, image is chosen, establishes first convolutional neural networks framework, i.e. CNN (convolutional neural Network), automatic study is partitioned into focal area, referred to as area-of-interest, i.e. ROI (region of interest), and right Lesion shape is refined.Specifically include following step：

2nd step：To each pictures, area-of-interest, i.e. focal area, Ran Houtong are sketched out manually (by expert) first It crosses first CNN framework and trains automatic parted pattern, this CNN model is denoted as SegCNN；

The network structure that the SegCNN is made of 15 layers of convolutional layer, 4 layers of down-sampling layer；The convolution kernel of each convolutional layer Size is respectively：First layer is 13 × 13, and the second layer is 11 × 11 with third layer, and the 4th layer is 5 × 5, remaining each layer is 3 × 3； The step-length of convolutional layer is respectively：First convolutional layer is 2, remaining is all 1；The size of down-sampling layer is all 3 × 3, and step-length is all It is 2.

The specific method that automatic parted pattern SegCNN is trained by first CNN framework is：

Wherein, symbol * represents convolution operator；The l represents the number of plies；The i represents l-1 layers of i-th of neuron section Point；The j represents l layers of j-th of neuron node；The M_jRepresent the set of the input maps of selection；It is describedRefer to L-1 layers of output, as l layers of input；The f is activation primitive, takes sigmoid functions hereAs sharp Function living, e represent Euler's numbers 2.718281828, e^xIt is exactly exponential function；The k is convolution operator；The b is biasing；It is each A output map can give an additional biasing b, but for a specific output map, convolution each inputs the convolution of maps Core is all different；

This step also needs to carry out gradient calculating, and to update sensitivity, how much sensitivity is for representing b variations, error meeting How much is variation：

Wherein, the l represents the number of plies；The j represents l layers of j-th of neuron node；It is describedRepresent each element phase Multiply；The δ represents the sensitivity of output neuron, that is, biases the change rate of b；The s^l=W^lx^l-1+b^l, x^l-1Refer to l-1 layers Output, W is weight, and b is biasing；The f is activation primitive, takes sigmoid functions hereAs activation letter Number, e represent Euler's numbers 2.718281828, e^xIt is exactly exponential function；F " (x) is the derived function of f (x) (i.e. if f takes sigmoid FunctionThen f'(x)=(1-f (x)) f (x))；It is describedRepresent the weights that each layer is shared；Up () table Show that (if the decimation factor of down-sampling is n, up-sampling operation is exactly by each pixel level and hangs down for a up-sampling operation Nogata copies n times upwards, can thus restore original size)；

Wherein, the l represents the number of plies；The j represents l layers of j-th of neuron node；The b represents biasing；The δ It represents the sensitivity of output neuron, that is, biases the change rate of b；The u, v represent (u, v) position of output maps；The E is Error function, hereThe C represents the dimension of label, if the problem of two classification, then label is just Y can be denoted as_h∈ { 0,1 }, C=1, can also be denoted as y at this time_h∈ { (0,1), (1,0) }, at this time C=2；It is describedRepresent n-th The h dimensions of a sample corresponding label；It is describedRepresent h-th of output of the corresponding network output of n-th of sample；

Wherein, the W is weight parameter；The E is error function, andThe C represents label Dimension, if two classification the problem of, then label can be denoted as y_h∈ { 0,1 }, C=1, can also be denoted as y at this time_h∈ { (0,1), (1,0) }, at this time C=2；It is describedRepresent the h dimensions of n-th of sample corresponding label；It is describedRepresent n-th of sample H-th of output of corresponding network output；The η is learning rate, i.e. step-length；Due to the weights much connected be it is shared, because This needs to seek gradient to the point with the associated connection of the weights to all, then to these ladders for a given weights Degree is summed：

Wherein, the l represents the number of plies；The i represents l layers of i-th of neuron node；The j represents j-th of l layers Neuron node；B represents biasing, and the δ represents the sensitivity of output neuron, that is, biases the change rate of b；The u, v are represented Export (u, v) position of maps；The E is error function, hereThe C represents the dimension of label, If the problem of two classification, then label can be denoted as y_h∈ { 0,1 }, C=1, can also be denoted as y at this time_h∈{(0,1),(1, 0) }, C=2 at this time；It is describedRepresent the h dimensions of n-th of sample corresponding label；It is describedRepresent the corresponding network of n-th of sample H-th of output of output；It is describedIt is convolution kernel；It is describedIt isIn element when convolution withBy element The patch of multiplication, i.e., all region units in all pictures identical with convolution kernel size, (u, v) position of output convolution map Value be by the patch and convolution kernel of last layer (u, v) positionBy the result of element multiplication；

Wherein, the f is activation primitive, takes sigmoid functions hereAs activation primitive, e represents Europe Draw number 2.718281828, e^xIt is exactly exponential function；It is describedRepresent the weights that each layer is sharedRepresent the weights that each layer is shared； The down () represents a down-sampling function；It sums to all pixels of the block of the different nxn of input picture, in this way It (is exactly that each element of input picture is taken a fixed 3x3 size here that output image all reduces n times on two dimensions Block, the value that then wherein all elements are summed as the element in the output image, so that output image is in two dimensions 3 times are all reduced on degree)；Each output map corresponds to an one's own weight parameter β (biasing of multiplying property) and an additivity Bias b；

By gradient descent method come undated parameter β and b：

Wherein, the f " (x) refers to the derivative of activation primitive f (x)；The conv2 is two-dimensional convolution operator；It is described Rot180 is rotation 180 degree；It is described ' full ' refer to carry out complete convolution；The l represents the number of plies；The i represents the i-th of l layers A neuron node；The j represents l layers of j-th of neuron node；The b represents biasing；The δ represents output neuron Sensitivity, that is, bias b change rate；The u, v represent (u, v) position of output maps；The E is error function, expression Formula is same as above, i.e.,The C represents the dimension of label, if the problem of two classification, then label It is denoted as y_h∈ { 0,1 }, C=1, can also be denoted as y at this time_h∈ { (0,1), (1,0) }, at this time C=2；It is describedRepresent n-th of sample The h dimensions of this corresponding label；It is describedRepresent h-th of output of the corresponding network output of n-th of sample；The β is weight ginseng Number (general value is in [0,1])；The down () represents a down-sampling function；It is describedIt is l+1 layers of convolution kernel；Institute It statesJ-th of the neuron node of the output of l-1 layers for being；The s^l=W^lx^l-1+b^l, wherein being weight parameter, b is biasing,It is s^lJ-th of component.

s.t.∑_iα_ij0≤α of=1, and_ij≤1.

Wherein, symbol * represents convolution operator；The l represents the number of plies；The i represents l layers of i-th of neuron node； The j represents l layers of j-th of neuron node；The f is activation primitive, takes sigmoid functions hereAs Activation primitive, e represent Euler's numbers 2.718281828, e^xIt is exactly exponential function；It is describedIt is i-th point of l-1 layers of output Amount；The N_inRepresent the map numbers of input；It is describedIt is convolution kernel；It is describedIt is biasing；The α_ijRepresent l-1 layers of output When map is as l layers of input, the weights of l-1 layers of wherein i-th input map for obtaining j-th of output map or contribution；

(2) the feature combination Softmax extracted in (1) is utilized to automatically identify focal area, exports the probability graph of segmentation, Determine the model divided automatically；As soon as specific Softmax identification process is exactly given sample, a probability value is exported, it should What probability value represented is that this sample belongs to several probability of classification, and loss function is：

Wherein, the m represents to share m sample；The c represents that these samples can be divided into c classes in total；It is described It is a matrix, is the parameter corresponding to a classification, i.e. weight and biasing per a line；1 { } is an indicative letter Number, i.e., when the value in braces is true, the result of the function is 1, otherwise as a result 0；The λ is balance fidelity item (the One) with the parameter of regular terms (Section 2), here λ take positive number (its size is adjusted according to experimental result)；The J (θ) refers to The loss function of system；The e represents Euler's numbers 2.718281828, e^xIt is exactly exponential function；The T is that representing matrix calculates In transposition operator；Log represents natural logrithm, i.e., using Euler's numbers as the logarithm at bottom；N represents weight and the dimension of offset parameter Degree；x⁽ⁱ⁾It is the i-th dimension of input vector；y⁽ⁱ⁾It is the i-th dimension of each sample label；Then it is solved using gradient：

Wherein,The m represents to share m sample；It is describedIt is one A matrix is the parameter corresponding to a classification, i.e. weight and biasing per a line；1 { } is an indicative function, i.e., When the value in braces is true, the result of the function is 1, otherwise as a result 0；The λ is balance fidelity item (first item) With the parameter of regular terms (Section 2), here λ take positive number (its size is adjusted according to experimental result)；The J (θ) refers to system Loss function；It is J (θ) derived function；The e represents Euler's numbers 2.718281828, e^xIt is exactly exponential function；The T It is the transposition operator during representing matrix calculates；Log represents natural logrithm, i.e., using Euler's numbers as the logarithm at bottom；x⁽ⁱ⁾Be input to The i-th dimension of amount；y⁽ⁱ⁾It is the i-th dimension of each sample label；(used herein is a kind of new Softmax graders, that is, is only had The Softmax graders of two classification, for a medical image, the probability provided according to softmax can be obtained institute The probability graph that some focal areas are distinguished with non-focal area, the rough segmentation to focal area can have been obtained according to this figure It cuts；)

(3) using the medical image of the automatic divided ownerships of SegCNN, that is, focal area and non-focal area is distinguished, is found The boundary of focal area, and the lesion shape being partitioned into is refined, we are refined using the method that figure is cut here Divide, be exactly specifically：Remember I:X ∈ Ω → R is are defined on regionOn 2D ultrasound image datas, S is owns in Ω The set of pixel, Nx are the 6- neighborhood point sets of pixel x.Assuming that l_x∈ { 0,1 } is the label of pixel x, wherein 0 and 1 The pixel is represented respectively belongs to background (non-focal area) and prospect (focal area).Then we need to find below minimization Energy functional tally set l={ l_x, x ∈ S },

WhereinParameter lambda is used for adjusting data penalty term E_D(l) and boundary penalty term E_B (l) balance between.Area item D_x(l_x) for describing the similarity of pixel x and prospect or background.Edge detection function B_xy(x, y) features the discontinuity between pixel x and y, andβ is constant term.It connects Get off, we also need to define a gray threshold function：

The gray value interval of lesion thus can be roughly estimated from initial focal areaDefinition is by one group The part characterization item that feature distribution is formed, the feature of selection have the gray value I (x) of image, improved local binary patterns With local gray variance VAR_P,r.These features are combined into a union feature

τ, P, r are normal numbers, here

Wherein I_p(p=0,1 ..., P-1) is corresponding to be generally evenly distributed in using c ∈ Ω as the center of circle, and r is on the circle of radius The gray value of P point, I_cIt is the gray value of circle centre position.H (x) is Heaviside functions, i.e.,：

NoteAccumulation histogram for ith features of the pixel x in local neighborhood O (x).It is that ith feature exists Average accumulated histogram in initialization area, variance are denoted asThen it locally characterizes item and can be defined asW¹() is one-dimensional L1Wasserstein distances.The focal zone that last combination S egCNN is obtained The segmentation probability graph L (x) in domain, gray threshold function F (x) and part characterization P (x) obtain data item expression formula D_x(l_x) be,

D_x(l_x)=max (- R (x), 0) l_x+max(R(x),0)(1-l_x)

Hereγ is normal number.So as to which we can be obtained by Figure cuts model, and refinement segmentation is carried out to focal area.

3rd step：It is applied to all lesion images using the SegCNN models that the 2nd step obtains, i.e., the 1st step is chosen 20000 images are divided automatically, then establish a figure and cut model, the focal area that SegCNN is obtained are carried out automatic Refinement segmentation.Finally obtain ROI, i.e., all good pernicious lesions.

4th, it establishes second convolutional neural networks framework and understands good pernicious focus characteristic automatically, by three SegCNN moulds of process The ROI (i.e. all good pernicious lesions) that type is partitioned into automatically is divided into p groups, this CNN model is denoted as the RecCNN established, Characteristic is normalized again.It is partitioned into lesion automatically using SegCNN models and then is carried using RecCNN models The feature of these lesions is taken, linear transformation is carried out to these features extracted, end value is made to be mapped to [0,1].

Wherein, the p value ranges are the positive integer more than or equal to 2；

The network structure that the RecCNN is made of 6 layers of convolutional layer, 4 layers of down-sampling layer, 3 layers of full articulamentum, three complete The neuron node number of linking layer is respectively 4096,4096,1；The size of the convolution kernel of each convolutional layer is respectively：First layer is 13 × 13, the second layer is 11 × 11 with third layer, and the 4th layer is 5 × 5, remaining each layer is 3 × 3；Step-length is respectively：First convolution Layer is 2, remaining is all 1；The size of down-sampling layer is all 3 × 3, and step-length is all 2.

5th, p-1 groups data in step 4 are selected and make training set, training set is used to train RecCNN models, remaining one group of work Test set, test set are used to test trained RecCNN models.

RecCNN models are trained using training set, for understanding medical image features, can automatically be split to all Focal area extraction feature, then analyzed.Specifically training process is：It extracts the method for feature and process three The method of extraction characteristic procedure is the same during SegCNN models are divided automatically, i.e., is all by respective each convolutional layer and pond Change layer extraction feature, the effect of this two classes functional layer be it is the same, calculation formula with update method be it is the same, still The automatic partitioning portion of SegCNN models is to be carried out at the same time extraction feature for non-focal area and focal area, in this process five The object of RecCNN models is just for focal area, and automatic partitioning portion is to be directed to non-focal area and focal area simultaneously Extract feature, and the convolution kernel size of RecCNN models and SegCNN models and pond window size and convolutional layer and The step sizes of pond layer are different with filling size setting, so mutual convolutional layer is different from pond layer sphere of action.

Then polytypic grader can be carried out by constructing one using Softmax, and the feature extracted is analyzed, This process is actually to solve the optimal value of a loss function, is usually optimization loss functionWherein, the i refers to i-th of sample；

The j refers to jth class；The l refers to l classes；The m represents shared m sample, and m value ranges is arbitrarily just Integer；The c represents that these samples can be divided into c classes in total, and c value ranges are arbitrary positive integer；It is describedIt is a square Battle array is the parameter corresponding to a classification, i.e. weight and biasing per a line；The θ^T _jRefer to the transposition of the parameter vector of jth class, The θ^T _lRefer to the transposition of the parameter vector of l classes, the θ_ijRefer to the element of the i-th row jth of parameter matrix；1 { } It is an indicative function, i.e., when the value in braces is true, the result of the function is 1, otherwise as a result 0；The λ is The parameter of fidelity item (first item) and regular terms (Section 2) is balanced, λ takes positive number (adjusting its size according to experimental result) here； The e represents Euler's numbers 2.718281828, e^xIt is exactly exponential function；The T is the transposition operator during representing matrix calculates； Log represents natural logrithm, i.e., using Euler's numbers as the logarithm at bottom；x⁽ⁱ⁾It is the i-th dimension of input vector；y⁽ⁱ⁾It is each sample label I-th dimension；The classification number c of Softmax graders be equal to 5 (represent respectively the echo characteristics of lesion, edge feature, structure feature, Calcification feature, five category feature of aspect ratio features), each class has different subclasses, and wherein echo characteristics has high echo, waits back Sound, low echo or extremely low echo, four class of echoless, edge feature have finishing and two class of not finishing, and structure feature has real property, real Based on property, based on capsule, four class of capsule, calcification feature has Microcalcification and no two class of Microcalcification, aspect ratio features have more than 1 with it is small In equal to 1 liang class；The feature vector that can be obtained by output by stochastic gradient descent method is belonging respectively to the son of which category feature The probability of class；Detailed process is：It is similar with Forecasting Methodology in the automatic cutting procedure of SegCNN models in process three, is all excellent Change a loss function, only here be a polytypic Softmax function, be subordinate to according to output feature vector As soon as the probabilistic forecasting of which category feature goes out the probability which category feature a tag along sort belongs to get the feature to the lesion, also right One focus characteristic is classified, and can further obtain the corresponding feature type of good pernicious lesion.

6th, step 5 is repeated, does p crosscheck, i.e., the p group data divided for process four select one group not every time Same data make test set, and remaining p-1 groups data make training set, until each group of data all made test set.

By p crosscheck of process five and process six, the power of convolutional neural networks model RecCNN can be preserved every time Weight and offset parameter, according to the accuracy rate assessment result on test set, the calculation formula of accuracy rate is here Wherein AC represents accuracy rate；The correct sample number of TN presentation classes；The sample number of FN presentation class mistakes.Each accuracy rate with The average value of p times is little, just takes one group of weight and parameter of the offset parameter as RecCNN that wherein accuracy rate is slightly higher, i.e., The optimal parameter of RecCNN is finally obtained, that is, has trained RecCNN models, so as to finally determine based on deep learning Method understands the assistant diagnosis system of medical image feature.

The lesion image for needing to understand is input to this assistant diagnosis system, you can five category features of the lesion are obtained, And it is analyzed per category feature, and then good pernicious lesion can be diagnosed according to these features.

Fig. 2, Fig. 3 are to illustrate to be used for training the raw ultrasound image of the lesion of SegCNN models and corresponding disease in experiment The mask pictures in stove region；Fig. 4, Fig. 5 illustrate the raw ultrasound image of a lesion with being partitioned into disease automatically using SegCNN The effect picture of stove region mask.

Finally it should be noted that listed above is only specific embodiments of the present invention.It is clear that the invention is not restricted to Above example can also have many variations.Those of ordinary skill in the art can directly lead from present disclosure All deformations for going out or associating, are considered as protection scope of the present invention.

Claims

1. the assistant diagnosis system of medical image features is understood based on deep learning method, which is characterized in that including following processes：

First, the medical image data of lesion is read：

The medical image of lesion is read, the figure of image and at least 10000 pernicious lesions including at least 10000 benign lesions Picture；

2nd, medical image is pre-processed：

The lesion image that process one is read first carries out image gray processing, and removes ultrasound using the gray value of surrounding pixel point Doctor is to measure the label that tubercle correlative is done in image, recycles gaussian filtering denoising, finally utilizes gray-level histogram equalization Change enhancing contrast, obtain pretreated enhancing image；

3rd, image is chosen, establishes first convolutional neural networks framework, i.e. CNN, automatic study is partitioned into focal area, referred to as Area-of-interest, i.e. ROI, and lesion shape is refined；Specifically include following step：

1st step：The pretreated enhancing image 20000 of selection process two is opened, each 10000 of the image including good pernicious lesion；

2nd step：To each pictures, area-of-interest, i.e. focal area are sketched out manually first；Then pass through first CNN Framework trains automatic parted pattern, and it is SegCNN models to remember this automatic parted pattern；

The network structure that the SegCNN models are made of 15 layers of convolutional layer, 4 layers of down-sampling layer；The convolution kernel of each convolutional layer Size is respectively：First layer is 13 × 13, and the second layer is 11 × 11 with third layer, and the 4th layer is 5 × 5, remaining each layer is 3 × 3； The step-length of convolutional layer is respectively：First convolutional layer is 2, remaining is all 1；The size of down-sampling layer is all 3 × 3, and step-length is all It is 2；

3rd step：It is applied to all lesion images, i.e., 20000 chosen to the 1st step using the SegCNN models that the 2nd step obtains It opens image automatically to be divided, then establishes a figure and cut model, the focal area that SegCNN models obtain is carried out automatic Refinement segmentation, finally obtains ROI, i.e., all good pernicious lesions；

4th, the CNN models for establishing second convolutional neural networks framework understand good pernicious focus characteristic automatically, remember this CNN mould Type is RecCNN models；

The network structure that the RecCNN models are made of 6 layers of convolutional layer, 4 layers of down-sampling layer, 3 layers of full articulamentum；3 connect entirely The neuron node number for connecing layer is respectively 4096,4096,1；The size of the convolution kernel of each convolutional layer is respectively：First layer for 13 × 13, the second layer is 11 × 11 with third layer, and the 4th layer is 5 × 5, remaining each layer is 3 × 3；The step-length of convolutional layer is respectively：First A convolutional layer is 2, remaining is all 1；The size of down-sampling layer is all 3 × 3, and step-length is all 2；

The ROI that three SegCNN models of process are partitioned into automatically is divided into p groups, for training RecCNN models；The p is not Positive integer less than 2；

5th, p-1 groups data in process four are selected and make training set, training set is used to train RecCNN models, data one group remaining Make test set, test set is used to test trained RecCNN models；

RecCNN models are trained using training set, it, can be to all diseases split automatically for understanding medical image features Stove extracted region feature；

Wherein, the i refers to i-th of sample；The j refers to jth class；Described 1 refers to the 1st class；The m represents to share m sample This, m value ranges are arbitrary positive integer；The c represents that these samples can be divided into c classes in total, and c value ranges are arbitrary just whole Number；It is describedIt is a matrix, is the parameter corresponding to a classification, i.e. weight and biasing per a line；The θ^T _jRefer to The transposition of the parameter vector of jth class, the θ^T ₁Refer to the transposition of the parameter vector of the 1st class, the θ_ijRefer to the of parameter matrix The element of i row jth；1 { } is an indicative function, i.e., when the value in braces is true, the result of the function is 1, Otherwise as a result 0；The λ is the parameter for balancing fidelity item and regular terms, and λ takes positive number here；The e represents Euler's numbers 2.718281828 e^xIt is exactly exponential function；The T is the transposition operator during representing matrix calculates；Log represents natural logrithm, I.e. using Euler's numbers as the logarithm at bottom；x⁽ⁱ⁾It is the i-th dimension of input vector；y⁽ⁱ⁾It is the i-th dimension of each sample label；

The classification number c of Softmax graders is equal to 5, i.e., represents echo characteristics, edge feature, structure feature, the calcium of lesion respectively Change feature, five category feature of aspect ratio features；Each class has different subclasses, and echo characteristics has four subclasses：High echo, etc. return Sound, low echo or extremely low echo, echoless；Edge feature has two subclasses：Finishing, not finishing；Structure feature has four subclasses：It is real Based on property, reality, based on capsule, capsule；Calcification feature has two subclasses：Microcalcification, without Microcalcification；Aspect ratio features have two sons Class：More than 1, less than or equal to 1；Then, it is special which class is the feature vector exported by stochastic gradient descent method be belonging respectively to The probability of the subclass of sign；

6th, repetitive process five, do p crosscheck, i.e., the p group data divided for process four are selected a different set of every time Data make test set, and remaining p-1 groups data make training set, until each group of data all made test set；

By p crosscheck, the weight and offset parameter of convolutional neural networks model RecCNN can be preserved every time, and according to Accuracy rate assessment result on test set；The calculation formula of accuracy rate isWherein AC represents accuracy rate, TN tables Show the correct sample number of classification, the sample number of FN presentation class mistakes；It finally takes in the highest primary crosscheck of accuracy rate Weight and offset parameter as the optimal parameter of RecCNN models, have obtained trained RecCNN models, i.e., have finally determined The assistant diagnosis system of medical image feature is understood based on deep learning method；

The lesion image for needing to understand is input to this auxiliary diagnosis based on deep learning method deciphering medical image feature System, you can obtain the feature of the lesion, and analyzed per category feature it, and then good evil can be diagnosed according to these features Venereal disease stove.

2. the assistant diagnosis system according to claim 1 that medical image features are understood based on deep learning method, special Sign is, the 2nd step and the 3rd step in the process three, specially：

Step A：In a convolutional layer, the feature maps of last layer carries out convolution, Ran Houtong by a convolution kernel that can learn As soon as crossing an activation primitive, output feature map can be obtained；Each output is one input of convolution nuclear convolution or combines multiple The value of convolution input：

Wherein, symbol * represents convolution operator；The l represents the number of plies；The i represents l-1 layers of i-th of neuron node；Institute State j-th of neuron node that j represents l layers；The M_iRepresent the set of the input maps of selection；It is describedRefer to l-1 layers Output, as l layers of input, the x¹ _jRefer to j-th of component of the 1st layer of output；The f is activation primitive, is taken here Sigmoid functionsAs activation primitive, e represents Euler's numbers 2.718281828, e^xIt is exactly exponential function；The k It is convolution operator, the k¹ _ijRefer to the element of (i, j) position of level 1 volume product core；The b is to bias, the b¹ _jRefer to the 1st J-th of component of layer biasing；Each output map can give an additional biasing b, but specifically export map for one, The convolution kernel that convolution each inputs maps is different；

Wherein, the l represents the number of plies；The j represents l layers of j-th of neuron node；The o represents each element multiplication；Institute The sensitivity that δ represents output neuron is stated, that is, biases the change rate of b, the δ¹ _jRefer to j-th of component of the 1st layer of sensitivity, institute State δ¹⁺¹ _jRefer to j-th of component of 1+1 layers of sensitivity；The s^l=W^lx^l-1+b¹, x^l-1Refer to l-1 layers of output, W is power Weight, b is biases, the s¹ _jRefer to the 1st layer of s^l=W^lx^l-1+b^lJ-th of component, the W¹Refer to the 1st layer of weight parameter, institute State b¹Refer to the 1st layer of biasing；The f is activation primitive, takes sigmoid functions hereAs activation primitive, e Represent Euler's numbers 2.718281828, e^xIt is exactly exponential function；F " (x) is the derived function of f (x)；

Wherein, the l represents the number of plies；The j represents l layers of j-th of neuron node；The b represents biasing, the b¹ _jRefer to J-th of component of the 1st layer of biasing；The δ represents the sensitivity of output neuron, that is, biases the change rate of b；The u, v table Show (u, v) position of output maps；(the δ¹ _j) u, v refer to the element of the 1st layer of sensitivity (u, v) position；The E is error letter Number, hereThe C represents the dimension of label；It is describedRepresent the h of n-th of sample corresponding label Dimension；It is describedRepresent h-th of output of the corresponding network output of n-th of sample；

Wherein, the W is weight parameter, and the Δ W refers to the knots modification of weight parameter；The W¹Refer to the 1st layer of weight ginseng Number；The E is error function, andThe C represents the dimension of label；It is describedRepresent n-th of sample The h dimensions of corresponding label；It is describedRepresent h-th of output of the corresponding network output of n-th of sample；The η is learning rate, i.e., Step-length；Since the weights much connected are shared, for a given weights, need have connection with the weights to all The connection of system seeks gradient to the point, then sums to these gradients：

Wherein, the l represents the number of plies；The i represents l layers of i-th of neuron node；The j represents l layers of j-th of nerve First node；B represents biasing, and the δ represents the sensitivity of output neuron, that is, biases the change rate of b；The u, v represent output (u, v) position of maps；(the δ¹ _j) u, v refer to the 1st layer of sensitivity (u, v) position element；The E is error function, This is hopedThe C represents the dimension of label；It is describedRepresent the h dimensions of n-th of sample corresponding label；Institute It statesRepresent h-th of output of the corresponding network output of n-th of sample；It is describedIt is convolution kernel；It is describedIt isIn Element when convolution withBy the patch of element multiplication, i.e., all areas in all pictures identical with convolution kernel size Domain block, the value of (u, v) position of output convolution map is by the patch and convolution kernel of last layer (u, v) positionBy element phase The result multiplied；

Step B：Down-sampling layer has N number of input maps, just there is N number of output maps, and only each output map becomes smaller, then has：

Wherein, the X¹ _jRefer to j-th of component of the 1st layer of output, the X^1-1 _jRefer to j-th point of 1-1 layers of output Amount；The f is activation primitive, takes sigmoid functions hereAs activation primitive, e represents Euler's numbers 2.718281828 e^xIt is exactly exponential function；It is describedRepresent the weights that each layer is shared；The down () represents a down-sampling Function；The b¹ _jRefer to j-th of component of the 1st layer of biasing；The all pixels of the block of different n × n of input picture are asked With export image in this way and all reduce n times on two dimensions, n value ranges are positive integer；Each output map corresponding one An a one's own weight parameter β and additivity biasing b；

By gradient descent method come undated parameter β and b：

Wherein, the f " (x) refers to the derivative of activation primitive f (x)；The o represents each element multiplication；The conv2 is two Tie up convolution operator；The rot180 is rotation 180 degree；It is described ' full ' refer to carry out complete convolution；The l represents the number of plies；Institute State i-th of neuron node that i represents l layers；The j represents l layers of j-th of neuron node；The b represents biasing, described b_jRefer to the jth component of offset parameter；The δ represents the sensitivity of output neuron, that is, biases the change rate of b, the δ¹ _jIt is Refer to j-th of component of the 1st layer of sensitivity, the δ¹⁺¹ _jRefer to j-th of component of 1+1 layers of sensitivity；The u, v represent output (u, v) position of maps；(the δ¹ _j) u, v refer to 1 layer of sensitivity (u, v) position element；The E is error function, table It is same as above up to formula, i.e.,The C represents the dimension of label；It is describedRepresent n-th sample corresponding label H is tieed up；It is describedRepresent h-th of output of the corresponding network output of n-th of sample；The β is weight parameter, the β_jRefer to J-th of component of weight parameter；The down () represents a down-sampling function；It is describedIt is l+1 layers of convolution kernel；Institute It statesJ-th of the neuron node of the output of l-1 layers for being；The s^l=Wlx^l-1+b^l, wherein W is weight parameter, and b is inclined It puts,It is s^lJ-th of component；

s.t.∑_iα_ij0≤α of=1, and_ij≤1.

Wherein, symbol * represents convolution operator；The l represents the number of plies；The i represents l layers of i-th of neuron node；It is described J represents l layers of j-th of neuron node；The f is activation primitive, takes sigmoid functions hereAs activation Function, e represent Euler's numbers 2.718281828, e^°It is exactly exponential function；It is describedBe l-1 layers output i-th of component, institute State X¹ _jRefer to j-th of component of the 1st layer of output；The N_inRepresent the map numbers of input；It is describedIt is convolution kernel；It is describedIt is Biasing；The α_ijRepresent l-1 layer when exporting map as l layers of input, l-1 layers obtain wherein the i-th of j-th of output map The weights of a input map or contribution；

(2) focal area is automatically identified using the feature combination Softmax extracted in step (1), exports the probability graph of segmentation, Determine the model divided automatically；As soon as specific Softmax identification process is exactly given sample, a probability value is exported, it should What probability value represented is that this sample belongs to several probability of classification, and loss function is：

Wherein, the i refers to i-th of sample；The j refers to jth class；Described 1 refers to the 1st class；The m represents to share m sample This, m value ranges are arbitrary positive integer；The c represents that these samples can be divided into c classes in total, and c value ranges are arbitrary just whole Number；It is describedIt is a matrix, is the parameter corresponding to a classification, i.e. weight and biasing per a line；The θ^T _jRefer to The transposition of the parameter vector of jth class, the θ^T ₁Refer to the transposition of the parameter vector of the 1st class, the θ_ijRefer to the of parameter matrix The element of i row jth；1 { } is an indicative function, i.e., when the value in braces is true, the result of the function is 1, Otherwise as a result 0；The λ is the parameter for balancing fidelity item and regular terms, and λ takes positive number here；The J (θ) refers to system Loss function；The e represents Euler's numbers 2.718281828, e^xIt is exactly exponential function；The T is turned during representing matrix calculates Put operator；Log represents natural logrithm, i.e., using Euler's numbers as the logarithm at bottom；x⁽ⁱ⁾It is the i-th dimension of input vector；y⁽ⁱ⁾It is each The i-th dimension of sample label；Then it is solved using gradient：

Wherein, the θ^T _j、i、j、c、l、θ^T ₁It is respectively identical meaning with what is represented in above-mentioned loss function J (θ)；The m represents to share m sample；It is describedIt is a square Battle array is the parameter corresponding to a classification, i.e. weight and biasing per a line；The θ_jRefer to the parameter corresponding to jth class；It is described 1 { } is an indicative function, i.e., when the value in braces is true, the result of the function is 1, otherwise as a result 0；Institute It is the parameter for balancing fidelity item and regular terms to state λ, and λ takes positive number here；The J (θ) refers to the loss function of system； It is J (θ) derived function；The e represents Euler's numbers 2.718281828, e^xIt is exactly exponential function；The T is during representing matrix calculates Transposition operator；Log represents natural logrithm, i.e., using Euler's numbers as the logarithm at bottom；x⁽ⁱ⁾It is the i-th dimension of input vector；y⁽ⁱ⁾It is The i-th dimension of each sample label；

(3) using the medical image of the automatic divided ownerships of SegCNN, that is, focal area and non-focal area is distinguished, finds lesion The boundary in region, and the lesion shape being partitioned into is refined, it is the segmentation refined using the method that figure is cut here, tool Body is exactly：Remember I：X ∈ V → R is are defined on regionOn 2D ultrasound image datas, S be V in all pixels point collection It closes, Nx is the 6- neighborhood point sets of pixel x；Assuming that 1_x∈ { 0,1 } is the label of pixel x, wherein 0 and 1 represents this respectively Pixel belongs to background and prospect；Then need the tally set 1={ l of the energy functional below searching minimization_x, x ∈ S },

Wherein,Parameter lambda is used for adjusting data penalty term E_D(l) and boundary penalty term E_B (l) balance between, λ value ranges are arbitrary real number；The V refers to the regional extent of image；Area item D_x(l_x) for describing The similarity of pixel x and prospect or background；Edge detection function B_xy(x, y) features discontinuous between pixel x and y Property, andβ is constant term, and the I (x) refers to the gray value at pixel x on image, described I (y) refers to the gray value at pixel y on image；Next, define a gray threshold function：

Wherein, the ζ refers to pixel minimum gradation value in focal area, and the η refers to the maximum gray scale of pixel in focal area Value；The gray value interval of lesion thus can be roughly estimated from initial focal areaDefinition is by one group of feature point The part characterization item that cloth is formed, the feature of selection have the gray value I (x) of image, improved local binary patternsAnd part Gray variance VAR_{P, r}；These features are combined into a union featureτ, P, r are normal Number；Here

Wherein I_p(p=0,1 ..., P-1) is corresponding to be generally evenly distributed in using c ∈ Ω as the center of circle, and r is P point on the circle of radius Gray value, I_cIt is the gray value of circle centre position；The I_mRefer to using c ∈ Ω as the center of circle, r is P gray values on the circle of radius Mean value, the sign refers to sign function, and when x is more than 0, sign (x) is more than 0, and otherwise sign (x) is less than 0；H (x) is Heaviside functions, i.e.,

NoteAccumulation histogram for ith features of the pixel x in local neighborhood 0 (x)；It is that ith feature is initializing area Average accumulated histogram in domain, variance are denoted asThen it locally characterizes item and can be defined asW¹ () is one-dimensional L1Wasserstein distances；The segmentation probability graph L (x) of focal area that last combination S egCNN is obtained, Gray threshold function F (x) and part characterization P (x), obtains data item expression formula D_x(l_x) be：

D_x(l_x)=max (- R (x), 0) l_x+ max (R (x), 0) (1-l_x)

Hereγ is normal number；The max refers to be maximized；From And the figure for having obtained that focal area can be carried out refinement segmentation cuts model, cutting model using this figure can be to by SegCNN mould The focal area that type obtains carries out refinement segmentation.