CN116958701A - Network abnormal flow detection method based on improved VGG16 and image enhancement - Google Patents
Network abnormal flow detection method based on improved VGG16 and image enhancement Download PDFInfo
- Publication number
- CN116958701A CN116958701A CN202310956833.4A CN202310956833A CN116958701A CN 116958701 A CN116958701 A CN 116958701A CN 202310956833 A CN202310956833 A CN 202310956833A CN 116958701 A CN116958701 A CN 116958701A
- Authority
- CN
- China
- Prior art keywords
- network
- layer
- feature
- vgg16
- detection method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 53
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 22
- 238000000605 extraction Methods 0.000 claims abstract description 19
- 238000010606 normalization Methods 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 12
- 238000012549 training Methods 0.000 claims abstract description 8
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 238000011897 real-time detection Methods 0.000 claims abstract description 4
- 239000013598 vector Substances 0.000 claims description 15
- 238000012216 screening Methods 0.000 claims description 13
- 238000011176 pooling Methods 0.000 claims description 8
- 238000000034 method Methods 0.000 claims description 7
- 230000003014 reinforcing effect Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 2
- 230000009466 transformation Effects 0.000 claims 1
- 230000004927 fusion Effects 0.000 abstract description 8
- 230000006870 function Effects 0.000 description 6
- 238000012795 verification Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000005856 abnormality Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009901 attention process Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of network security, and discloses a network abnormal flow detection method based on improved VGG16 and image enhancement, which comprises the following steps: the one-dimensional network flow data acquired in a preset time period is converted into a two-dimensional gray scale map through normalization processing; preprocessing the two-dimensional gray scale map by an ImageDataGenerator to obtain enhanced sample image data; constructing an anomaly detection model through an improved VGG16 network fused with a convolution attention module, and training the anomaly detection model by utilizing the enhanced sample image data; and applying the anomaly detection model to real-time detection of network anomaly traffic. In summary, the convolution attention module is integrated into the VGG16 network, so that full extraction of self-feature fusion of the network from shallow to deep is effectively realized, the accuracy of a detection result is further improved, and image data after image data enhancement processing is used as input of an anomaly detection model, so that the generalization capability of the anomaly detection model can be effectively enhanced.
Description
Technical Field
The invention belongs to the technical field of network security, and particularly relates to a network abnormal flow detection method based on improved VGG16 and image enhancement.
Background
With the development of technology, various applications of network communication are becoming more and more popular in people's life, and demands for network communication are also increasing, so security of network communication is also increasing.
The conventional network defense detection system mostly adopts expert rules or machine learning to detect abnormal traffic, wherein: expert rules have a certain artificial subjective factor, the detection accuracy is poor; machine learning often uses network models such as a Recurrent Neural Network (RNN) and a long and short time memory network (LSTM) to perform feature learning on abnormal data of one-dimensional time sequence signals, however, the depth of the models is shallow, and it is difficult to learn high-dimensional features in the data, so that the problem of low detection accuracy often occurs when detecting actual data of network traffic.
Disclosure of Invention
In view of the above, in order to solve the above-mentioned problems, an object of the present invention is to provide a network abnormal traffic detection method based on improved VGG16 and image enhancement.
In order to achieve the above purpose, the present invention provides the following technical solutions:
a network anomaly traffic detection method based on improved VGG16 and image enhancement, comprising:
the one-dimensional network flow data acquired in a preset time period is converted into a two-dimensional gray scale map through normalization processing;
preprocessing the two-dimensional gray scale map by an ImageDataGenerator to obtain enhanced sample image data;
constructing an anomaly detection model through an improved VGG16 network fused with a convolution attention module, and training the anomaly detection model by utilizing the enhanced sample image data;
and applying the anomaly detection model to real-time detection of network anomaly traffic.
Preferably, the one-dimensional network traffic data includes a normal time sequence traffic signal and an abnormal time sequence traffic signal.
Preferably, the normalization conversion formula is as follows:
where P (i) is a feature value after normalization, N (i) is a feature value before normalization, and Min (N) and Max (N) are the minimum and maximum values, respectively, of the feature values.
Preferably, the anomaly detection model comprises a feature extraction layer based on a VGG16 network, a feature enhancement layer based on a convolution attention module, and a feature classification layer based on a Softmax classifier.
Preferably, the feature extraction layer is used for extracting data features from the enhanced sample image data, and the feature extraction layer comprises five convolution layers and a pooling layer which are matched with each other.
Preferably, the feature enhancement layer is used for screening and fusing the data features extracted by the third layer, the fourth layer and the fifth layer in the feature extraction layer to obtain new enhancement features.
Preferably, the step of obtaining new reinforcing features comprises:
taking the data features extracted by the convolution layer of the third layer in the feature extraction layer as a first input;
taking the data features extracted by the convolution layer in the fourth layer in the feature extraction layer as a second input;
screening the first input and the second input respectively to obtain a first characteristic vector and a second characteristic vector;
and firstly splicing and fusing the first feature vector and the second feature vector by using an Add function to obtain a fused feature, and then fusing the fused feature with the data feature extracted by the convolution layer at the fifth layer in the feature extraction layer to obtain a new reinforced feature.
Preferably, when the first input and the second input are screened, screening is performed based on the channel attention and the spatial attention of the convolution attention module.
Preferably, the screening formula based on the channel attention is as follows:
M c (F) =σ (MLP (AvgPooling (F))+mlp (MaxPooling (F))); wherein sigma is a sigmoid function; f is input and F.epsilon.R C×H×W ；M c (F) To filter the output channel eigenvector and M c (F)∈R C×1×1 。
Preferably, the screening formula based on the spatial attention is as follows:
M s (F)＝σ(f 7×7 ([AvgPooling(F)；MaxPooling(F)]) A) is provided; wherein sigma is a sigmoid function; f is input, and F.epsilon.R C×H×W ；M s (F) To filter out the output space feature vector, and M s (F)∈R 1×H×W 。
Compared with the prior art, the invention has the following beneficial effects:
according to the network abnormal flow detection method, the VGG16 network is improved in a mode of being integrated with the convolution attention module, and an abnormal detection model is built on the basis of the improved VGG16 network, so that full extraction of self-feature fusion of the network from shallow to deep is realized, and the accuracy of a detection result is effectively ensured; the one-dimensional network flow signal normalization processing of each network node is converted into a two-dimensional gray image, the two-dimensional gray image is enhanced based on an image data generator image processing technology, and the image characteristics after the enhancement processing are used as the input of an anomaly detection model, so that the generalization capability of the model is effectively enhanced.
Drawings
FIG. 1 is a flow chart of a method for detecting network abnormal traffic according to the present invention;
FIG. 2 is a block diagram of an anomaly detection model of the present invention;
FIG. 3 is a block diagram of a convolution attention module of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As shown in fig. 1, the network abnormal traffic detection method based on improved VGG16 and image enhancement provided by the invention comprises the following steps:
s1, continuously acquiring one-dimensional network flow data in a preset time period;
specifically, data acquisition starts from 0:00 until 23:55, wherein every 5min is a sampling point, n samples are acquired in total, each sample is a signal segment containing 288 sampling points, and the acquired one-dimensional network traffic data comprises a normal time sequence traffic signal and an abnormal time sequence traffic signal.
S2, carrying out normalization processing on the one-dimensional network flow data and converting the one-dimensional network flow data into a two-dimensional gray scale map;
the normalization processing formula is:where P (i) is a feature value after normalization, N (i) is a feature value before normalization, and Min (N) and Max (N) are the minimum and maximum values, respectively, of the feature values.
S3, dividing the two-dimensional gray scale image obtained by conversion into a training set and a verification set;
as shown in the following table, it is assumed that 1000 data samples are included in one-dimensional network traffic data of a normal time sequence traffic signal and an abnormal time sequence traffic signal, respectively, and a training set and a verification set are randomly divided according to a ratio of 4:1;
s4, preprocessing the two-dimensional gray scale map through an ImageDataGenerator to obtain enhanced sample image data.
S5, constructing an abnormality detection model through an improved VGG16 network fused with a convolution attention module, and training the abnormality detection model by utilizing the enhanced sample image data (training set);
as shown in fig. 2, the anomaly detection model includes a feature extraction layer based on the VGG16 network, a feature enhancement layer based on the convolution attention module, and a feature classification layer based on the Softmax classifier;
specific:
a) The feature extraction layer takes a VGG16 network as a main network structure, and specifically comprises five convolution layers and a pooling layer which are matched with each other;
the convolution layer (conv 3-64/conv3-128/conv3-256/conv3-512/conv 3-512) carries out convolution operation on the input data, extracts the local characteristics of the input data, takes the output as the input of the next layer, and each convolution kernel has the same size. Is provided withMapping of the output characteristics for the kth layer mth convolution kernel,>is a weight matrix>For the local input feature of the kth layer c-th convolution operation, # denotes the convolution operation, # is->For bias, the convolution layer operation can be described by the following formula:
pooling layer (maxpool) reduced transfusionThe size of the input features expands the sensing field, realizes translational invariance, rotational invariance and scale invariance of input, and can reduce the number of network parameters. Is provided withThe jth pooling value mapped for the kth layer ith input,/th layer of k>For the local input feature of the jth pooling operation of the kth layer, w is the width of the pooling area, the pooling layer working process can be described by the following formula:
b) The feature enhancement layer takes a convolution attention module as a feature screening basis, takes data features extracted by the convolution layers of the third layer and the fourth layer in the feature extraction layer as (first/second) input, and screens to obtain (first/second) feature vectors;
as shown in fig. 3, the convolution attention module consists of channel attention and spatial attention, whereby the filtering is performed from the channel dimension and the spatial dimension, respectively, when performing feature filtering:
screening for channel dimension features: let the input feature map be F, F ε R C×H×W The method comprises the steps of carrying out a first treatment on the surface of the Let the channel feature vector be M c (F) And M is c (F)∈R C×1×1 . The specific channel attention procedure can be described by the following formula:
M c (F)＝σ(MLP(AvgPooling(F))+MLP(MaxPooling(F)))；
regarding spatial dimension feature screening: let the input feature map be F, F ε R C×H×W The method comprises the steps of carrying out a first treatment on the surface of the Set a space feature vector M s (F)，M s (F)∈R 1×H×W . The specific spatial attention process can be described by the following formula:
M s (F)＝σ(f 7×7 ([AvgPooling(F)；MaxPooling(F)]))；
the sigma is a sigmoid function, and f 7×7 Represented as a convolution kernel size of 7 x 7 in the feature enhancement layer.
And splicing and fusing the screened (first/second) feature vectors from shallow to deep by using an Add function to obtain fusion features, and then fusing the fusion features with data features extracted by the convolution layer at the fifth layer in the feature extraction layer to obtain new reinforcement features. Specifically, based on feature fusion of Add function, let feature one be X' = [ X ] 1 ,x 2 ,x 3 ,…,x n ]Feature two is Y' = [ Y ] 1 ,y 2 ,y 3 ,…,y n ]New feature F after fusion 1 Add (X ', Y') =x '+ iY', where i represents a fusion coefficient.
c) And inputting the reinforced features combined with the features of the different layers into a feature classification layer based on a Softmax classifier, so as to obtain classification results, and completing training of the anomaly detection model.
S6, inputting a verification set into the trained abnormal detection model, outputting a classification detection result (normal or abnormal binary classification label) by the abnormal detection model through a feature classification layer (Softmax classifier), and judging the detection accuracy of the trained abnormal detection model based on the verification set, so that the detection accuracy of the abnormal detection model is effectively ensured.
S7, applying the anomaly detection model to network anomaly traffic real-time detection;
collecting one-dimensional network flow data in a real-time network state, and converting the one-dimensional network flow data into a two-dimensional gray scale map through normalization processing;
performing image enhancement pretreatment on the two-dimensional gray scale image through an ImageDataGenerator;
and inputting the image data subjected to the enhanced pretreatment into the anomaly detection model, and correspondingly outputting to obtain a detection result about the real-time network state.
In summary, the one-dimensional network time sequence flow signal is converted into the two-dimensional gray level diagram, then image enhancement processing is carried out based on an image data generator image processing technology, and then full extraction of self-feature fusion from shallow to deep is realized on an enhanced image through an improved VGG16 model fused with a convolution attention module, so that network abnormal flow detection with high precision and high efficiency is realized.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (10)
1. The network abnormal flow detection method based on the improved VGG16 and the image enhancement is characterized by comprising the following steps:
the one-dimensional network flow data acquired in a preset time period is converted into a two-dimensional gray scale map through normalization processing;
preprocessing the two-dimensional gray scale map by an ImageDataGenerator to obtain enhanced sample image data;
constructing an anomaly detection model through an improved VGG16 network fused with a convolution attention module, and training the anomaly detection model by utilizing the enhanced sample image data;
and applying the anomaly detection model to real-time detection of network anomaly traffic.
2. The network anomaly traffic detection method based on improved VGG16 and image enhancement of claim 1, wherein: the one-dimensional network traffic data includes a normal timing traffic signal and an abnormal timing traffic signal.
3. The network anomaly traffic detection method based on improved VGG16 and image enhancement of claim 1, wherein the normalization process transformation formula is as follows:
wherein P (i) is a characteristic value after normalization and N (i) is a characteristic value before normalizationThe sign values, min (N) and Max (N), are the minimum and maximum values, respectively, of the feature values.
4. The network anomaly traffic detection method based on improved VGG16 and image enhancement of claim 1, wherein: the anomaly detection model includes a feature extraction layer based on a VGG16 network, a feature enhancement layer based on a convolution attention module, and a feature classification layer based on a Softmax classifier.
5. The network anomaly traffic detection method based on improved VGG16 and image enhancement of claim 4, wherein: the feature extraction layer is used for extracting data features from the enhanced sample image data, and comprises five convolution layers and a pooling layer which are matched with each other.
6. The network anomaly traffic detection method based on improved VGG16 and image enhancement of claim 5, wherein: the characteristic reinforcing layer is used for screening and fusing the data characteristics extracted by the third layer, the fourth layer and the fifth layer in the characteristic extracting layer so as to obtain new reinforcing characteristics.
7. The method for detecting network anomaly traffic based on improved VGG16 and image enhancement of claim 6, wherein the step of obtaining new enhancement features comprises:
taking the data features extracted by the convolution layer of the third layer in the feature extraction layer as a first input;
taking the data features extracted by the convolution layer in the fourth layer in the feature extraction layer as a second input;
screening the first input and the second input respectively to obtain a first feature vector and a second feature vector;
and firstly splicing and fusing the first feature vector and the second feature vector by using an Add function to obtain a fused feature, and then fusing the fused feature with the data feature extracted by the convolution layer at the fifth layer in the feature extraction layer to obtain a new reinforced feature.
8. The network anomaly traffic detection method based on improved VGG16 and image enhancement of claim 7, wherein: and when the first input and the second input are screened, screening is respectively carried out based on the channel attention and the space attention of the convolution attention module.
9. The network anomaly traffic detection method based on improved VGG16 and image enhancement of claim 8, wherein: the screening formula based on the channel attention is as follows:
M c (F) =σ (MLP (AvgPooling (F))+mlp (MaxPooling (F))); wherein sigma is s igmoid function
A number; f is input, and F.epsilon.R C×H×W ；M c (F) To filter the output channel characteristic vector, and M c (F)∈R C×1×1 。
10. The network anomaly traffic detection method based on improved VGG16 and image enhancement of claim 8, wherein: the screening formula based on the spatial attention is as follows:
M s (F)＝σ(f 7×7 ([AvgPooling(F)；MaxPooling(F)]) A) is provided; wherein sigma is an s igmoid function; f is input, and F.epsilon.R C×H×W ；M s (F) To filter out the output space feature vector, and M s (F)∈R 1×H×W 。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310956833.4A CN116958701A (en) | 2023-08-01 | 2023-08-01 | Network abnormal flow detection method based on improved VGG16 and image enhancement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310956833.4A CN116958701A (en) | 2023-08-01 | 2023-08-01 | Network abnormal flow detection method based on improved VGG16 and image enhancement |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116958701A true CN116958701A (en) | 2023-10-27 |
Family
ID=88452769
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310956833.4A Pending CN116958701A (en) | 2023-08-01 | 2023-08-01 | Network abnormal flow detection method based on improved VGG16 and image enhancement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116958701A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118115797A (en) * | 2024-03-07 | 2024-05-31 | 浙江省交通运输科学研究院 | Bridge structure health monitoring data anomaly detection method based on deep learning |
-
2023
- 2023-08-01 CN CN202310956833.4A patent/CN116958701A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118115797A (en) * | 2024-03-07 | 2024-05-31 | 浙江省交通运输科学研究院 | Bridge structure health monitoring data anomaly detection method based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113191215B (en) | Rolling bearing fault diagnosis method integrating attention mechanism and twin network structure | |
| CN110287849B (en) | Lightweight depth network image target detection method suitable for raspberry pi | |
| CN112991354B (en) | High-resolution remote sensing image semantic segmentation method based on deep learning | |
| CN108734208B (en) | Multi-source heterogeneous data fusion system based on multi-mode deep migration learning mechanism | |
| CN111814661B (en) | Human body behavior recognition method based on residual error-circulating neural network | |
| CN109993100B (en) | Method for realizing facial expression recognition based on deep feature clustering | |
| CN106897738A (en) | A kind of pedestrian detection method based on semi-supervised learning | |
| CN112766229B (en) | Human face point cloud image intelligent identification system and method based on attention mechanism | |
| CN112950780B (en) | Intelligent network map generation method and system based on remote sensing image | |
| CN114818774A (en) | Intelligent gearbox fault diagnosis method based on multi-channel self-calibration convolutional neural network | |
| CN112766283B (en) | Two-phase flow pattern identification method based on multi-scale convolution network | |
| CN111161224A (en) | Casting internal defect grading evaluation system and method based on deep learning | |
| CN112488963A (en) | Method for enhancing crop disease data | |
| CN111967361A (en) | Emotion detection method based on baby expression recognition and crying | |
| CN116011507A (en) | Rare fault diagnosis method for fusion element learning and graph neural network | |
| CN116863237A (en) | Image classification method and device based on category related feature reconstruction | |
| CN113435276A (en) | Underwater sound target identification method based on antagonistic residual error network | |
| CN116958701A (en) | Network abnormal flow detection method based on improved VGG16 and image enhancement | |
| CN116152678A (en) | Marine disaster-bearing body identification method based on twin neural network under small sample condition | |
| CN117593666B (en) | Geomagnetic station data prediction method and system for aurora image | |
| CN118051831B (en) | Underwater sound target identification method based on CNN-transducer cooperative network model | |
| CN114295967A (en) | Analog circuit fault diagnosis method based on migration neural network | |
| CN105809200A (en) | Biologically-inspired image meaning information autonomous extraction method and device | |
| CN117710841A (en) | Small target detection method and device for aerial image of unmanned aerial vehicle | |
| CN117011219A (en) | Method, apparatus, device, storage medium and program product for detecting quality of article |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |