Disclosure of Invention
The technical problem to be solved by the embodiments of the present invention is to provide a method, a system and a mobile phone for identifying a thermal imaging image of skin pressure injury, wherein a machine learning method is used to analyze an image and a temperature index to complete the construction of an infrared imaging mode diagram of the pressure injury, so as to identify the thermal imaging image of the pressure injury.
In order to solve the technical problem, an embodiment of the present invention provides a method for identifying a thermal imaging image of a skin pressure injury, where the method includes the following steps:
s1, acquiring a plurality of original skin pressure injury thermal imaging images;
step S2, marking each original skin pressure injury thermal imaging image with a corresponding label, and further performing image segmentation processing on each original skin pressure injury thermal imaging image with the label to segment a target area image of each original skin pressure injury thermal imaging image with the label;
step S3, extracting image features of each segmented target area image to obtain feature data of each extracted target area image feature, and further preprocessing the feature data of each extracted target area image feature and establishing a data set;
step S4, training the preprocessed characteristic data in the data set based on a preset mechanical method, and constructing a skin pressure injury thermal imaging image recognition model;
and step S5, acquiring a skin thermal imaging image to be detected, importing the skin thermal imaging image to be detected into the constructed skin pressure damage thermal imaging image recognition model for detection, and determining whether the skin thermal imaging image to be detected is a skin pressure damage thermal imaging image.
Wherein, the step S2 specifically includes:
if the matching degree of a certain original skin pressure injury thermal imaging image and a preset skin pressure injury thermal imaging image is larger than a preset threshold value, marking a label as 1; otherwise, the label is marked as 0;
after all the original skin pressure damage thermal imaging image labels are marked, performing true color enhancement treatment on the original skin pressure damage thermal imaging image of each existing label; wherein, the true color enhancement processing is that the image color is kept unchanged, but the image brightness is enhanced;
the color image segmentation method based on the preset visual color clustering is used for carrying out image segmentation on each original skin pressure damage thermal imaging image subjected to true color enhancement processing, and segmenting a target area image of each original skin pressure damage thermal imaging image with a label.
The specific steps of performing true color enhancement processing on the original skin pressure injury thermal imaging image of each existing label comprise:
the R, G, B components in the original skin pressure injury thermal imaging image of each existing label are correspondingly converted into H, I, S components to be represented;
enhancing the converted I component in the original skin pressure injury thermal imaging image of each existing label by utilizing a gray scale linear transformation method;
and after the I component in each original skin pressure injury thermal imaging image subjected to gray scale linear transformation is enhanced, inversely converting the H, I, S component of each original skin pressure injury thermal imaging image subjected to gray scale linear transformation into a R, G, B component for representation, and obtaining each original skin pressure injury thermal imaging image subjected to true color enhancement treatment.
The color image segmentation method based on the preset visual color clustering performs image segmentation on each original skin pressure damage thermal imaging image subjected to true color enhancement processing, and the specific steps of segmenting the target area image of each original skin pressure damage thermal imaging image with a label include:
determining a gray threshold;
performing graying processing on each original skin pressure damage thermal imaging image subjected to true color enhancement processing to obtain a gray value of each pixel in each grayed original skin pressure damage thermal imaging image, and further screening out and reserving the pixels with the gray values equal to the gray threshold value in each grayed original skin pressure damage thermal imaging image to obtain a gray image formed by the reserved pixels in each grayed original skin pressure damage thermal imaging image;
after the obtained gray level images are all reversely converted into corresponding RGB images, the images are further all converted into images represented by H, I, S components;
converting each gray image into H, I, S components in H, I, S representation images by using a color clustering algorithm based on visual consistency, respectively performing clustering calculation, and further performing region merging and deleting operation to obtain each target region image represented by H, I, S components;
the obtained target area images represented by H, I, S components are respectively reversely converted into corresponding RGB images to be output as the target area images of the original skin pressure damage thermographic image of each existing label after being divided.
The specific step of "extracting image features of each segmented target region image" in step S3 includes:
and extracting the approximate entropy and the sample entropy of each segmented target area image to be used as main features, and further extracting the color histogram, the color moment, the energy, the contrast, the texture entropy, the texture correlation and the image local binarization features of each segmented target area image to be used as supplementary features by using a color statistical feature extraction method, a gray level co-occurrence matrix method and a local binarization method.
Wherein the "preset mechanical method" in the step S4 is a support vector machine algorithm or a BP neural network algorithm.
The embodiment of the invention also provides a system for identifying the skin pressure injury thermal imaging image, which comprises an acquisition unit, an image segmentation unit, an image feature extraction unit, an identification model construction unit and a result judgment unit; wherein,
the acquisition unit is used for acquiring a plurality of original skin pressure injury thermal imaging images;
the image segmentation unit is used for marking each original skin pressure damage thermal imaging image with a corresponding label, further performing image segmentation processing on each original skin pressure damage thermal imaging image with the label, and segmenting a target area image of each original skin pressure damage thermal imaging image with the label;
the image feature extraction unit is used for extracting image features of each segmented target area image to obtain feature data of each extracted target area image feature, and further preprocessing the feature data of each extracted target area image feature and establishing a data set;
the identification model construction unit is used for training the preprocessed characteristic data in the data set based on a preset mechanical method to construct a skin pressure injury thermal imaging image identification model;
and the result judging unit is used for acquiring a thermal imaging image of the skin to be detected, importing the thermal imaging image into the constructed thermal imaging image recognition model of the skin pressure damage for detection, and determining whether the thermal imaging image of the skin to be detected is the thermal imaging image of the skin pressure damage.
Wherein the image segmentation unit includes:
the image label marking module is used for marking a label as 1 if the matching degree of a certain original skin pressure injury thermal imaging image and a preset skin pressure injury thermal imaging image is greater than a preset threshold value; otherwise, the label is marked as 0;
the image color processing module is used for performing true color enhancement processing on the original skin pressure damage thermal imaging image of each existing label after all the original skin pressure damage thermal imaging image labels are marked; if the true color enhancement processing is that the image color is kept unchanged, the image brightness is enhanced;
and the image segmentation module is used for carrying out image segmentation on each original skin pressure damage thermal imaging image subjected to true color enhancement processing based on a preset color image segmentation method of visual color clustering, and segmenting a target area image of each original skin pressure damage thermal imaging image with a label.
The embodiment of the invention also provides a mobile phone for identifying the thermal imaging image of the skin pressure injury, which comprises an acquisition unit, an image segmentation unit, an image feature extraction unit, an identification model construction unit and a result judgment unit; wherein,
the acquisition unit is used for acquiring a plurality of original skin pressure injury thermal imaging images;
the image segmentation unit is used for marking each original skin pressure damage thermal imaging image with a corresponding label, further performing image segmentation processing on each original skin pressure damage thermal imaging image with the label, and segmenting a target area image of each original skin pressure damage thermal imaging image with the label;
the image feature extraction unit is used for extracting image features of each segmented target area image to obtain feature data of each extracted target area image feature, and further preprocessing the feature data of each extracted target area image feature and establishing a data set;
the identification model construction unit is used for training the preprocessed characteristic data in the data set based on a preset mechanical method to construct a skin pressure injury thermal imaging image identification model;
and the result judging unit is used for acquiring a thermal imaging image of the skin to be detected, importing the thermal imaging image into the constructed thermal imaging image recognition model of the skin pressure damage for detection, and determining whether the thermal imaging image of the skin to be detected is the thermal imaging image of the skin pressure damage.
Wherein the image segmentation unit includes:
the image label marking module is used for marking a label as 1 if the matching degree of a certain original skin pressure injury thermal imaging image and a preset skin pressure injury thermal imaging image is greater than a preset threshold value; otherwise, the label is marked as 0;
the image color processing module is used for performing true color enhancement processing on the original skin pressure damage thermal imaging image of each existing label after all the original skin pressure damage thermal imaging image labels are marked; if the true color enhancement processing is that the image color is kept unchanged, the image brightness is enhanced;
and the image segmentation module is used for carrying out image segmentation on each original skin pressure damage thermal imaging image subjected to true color enhancement processing based on a preset color image segmentation method of visual color clustering, and segmenting a target area image of each original skin pressure damage thermal imaging image with a label.
The embodiment of the invention has the following beneficial effects:
according to the invention, after an original skin pressure damage thermal imaging image is collected and subjected to label marking, image processing and target area segmentation, the characteristic data of the target area image is extracted, and the image and temperature indexes are analyzed by using a machine learning method to complete the construction of the pressure damage infrared imaging mode diagram, so that the identification of the pressure damage thermal imaging image is realized, and the method is not only fast and convenient, but also high in accuracy, thereby solving the problem that whether the pressure damage thermal imaging image is identified in the prior art.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings.
As shown in fig. 1, a method for thermal imaging image recognition of a skin pressure injury is provided in an embodiment of the present invention, the method includes the following steps:
s1, acquiring a plurality of original skin pressure injury thermal imaging images;
step S2, marking each original skin pressure injury thermal imaging image with a corresponding label, and further performing image segmentation processing on each original skin pressure injury thermal imaging image with the label to segment a target area image of each original skin pressure injury thermal imaging image with the label;
step S3, extracting image features of each segmented target area image to obtain feature data of each extracted target area image feature, and further preprocessing the feature data of each extracted target area image feature and establishing a data set;
step S4, training the preprocessed characteristic data in the data set based on a preset mechanical method, and constructing a skin pressure injury thermal imaging image recognition model;
and step S5, acquiring a skin thermal imaging image to be detected, importing the skin thermal imaging image to be detected into the constructed skin pressure damage thermal imaging image recognition model for detection, and determining whether the skin thermal imaging image to be detected is a skin pressure damage thermal imaging image.
Specifically, in step S2, each original skin pressure injury thermal imaging image may be labeled by manual or automatic comparison of an image database, and used as a training category of a subsequent mechanical method to identify the attribution of the skin pressure injury thermal imaging image to be detected, i.e., whether the skin pressure injury thermal imaging image is a skin pressure injury thermal imaging image. It should be noted that the manual label marking is determined by the experience of the expert and recorded and stored by the computer, and the automatic comparison of the image database is automatically determined by the image similarity and recorded and stored by the computer.
Step S2 specifically includes the following steps:
step S21, if the matching degree of a certain original skin pressure injury thermal imaging image and a preset skin pressure injury thermal imaging image is larger than a preset threshold (if the matching similarity exceeds the threshold 90%), marking a label as 1; otherwise, the label is marked as 0;
step S22, after all the labels of the original skin pressure injury thermal imaging images are marked, carrying out true color enhancement processing on the original skin pressure injury thermal imaging images of each existing label; wherein, the true color enhancement processing is that the image color is kept unchanged, and the image brightness is enhanced;
and step S23, carrying out image segmentation processing on each original skin pressure injury thermal imaging image subjected to true color enhancement processing based on a preset color image segmentation method of visual color clustering, and segmenting a target area image of each original skin pressure injury thermal imaging image with a label.
In step S22, the specific steps of performing true color enhancement on the original skin pressure injury thermographic image of each existing label include:
(1) the R, G, B components in the original skin pressure injury thermal imaging image of each existing label are correspondingly converted into hue H, brightness I and saturation S components for representation;
(2) enhancing the converted I component in the original skin pressure injury thermal imaging image of each existing label by utilizing a gray scale linear transformation method;
(3) and after the I component in each original skin pressure injury thermal imaging image subjected to gray scale linear transformation is enhanced, inversely converting the H, I, S component of each original skin pressure injury thermal imaging image subjected to gray scale linear transformation into a R, G, B component for representation, and obtaining each original skin pressure injury thermal imaging image subjected to true color enhancement treatment.
In step S23, the image background is first removed by using an image segmentation algorithm based on a global threshold, the portion representing the skin in the thermal imaging image is retained, and then a color image segmentation method based on visual color clustering is used to segment the pressure injury target area, so as to obtain the target area image.
Therefore, the color image segmentation method based on the preset visual color clustering performs image segmentation on each original skin pressure damage thermal imaging image subjected to true color enhancement processing, and the specific steps of segmenting the target area image of each original skin pressure damage thermal imaging image with a label include:
(1) determining a gray threshold; wherein, if the gray value range is between the sum, the gray threshold is satisfied;
(2) performing graying processing on each original skin pressure damage thermal imaging image subjected to true color enhancement processing to obtain a gray value of each pixel in each grayed original skin pressure damage thermal imaging image, and further screening out and reserving the pixels with the gray values equal to the gray threshold value in each grayed original skin pressure damage thermal imaging image to obtain a gray image formed by the reserved pixels in each grayed original skin pressure damage thermal imaging image;
(3) after the obtained gray level images are all reversely converted into corresponding RGB images, the images are further all converted into images represented by H, I, S components;
(4) converting each gray image into H, I, S components in H, I, S representation images by using a color clustering algorithm based on visual consistency, respectively performing clustering calculation, and further performing region merging and deleting operation to obtain each target region image represented by H, I, S components;
(5) the obtained target area images represented by H, I, S components are respectively reversely converted into corresponding RGB images to be output as the target area images of the original skin pressure damage thermographic image of each existing label after being divided.
In step S3, the features of each image are extracted by using a plurality of feature extraction methods, which are more classical methods: the method comprises the steps of selecting a proper feature extraction method aiming at a thermal imaging image and collecting representative features.
Therefore, the specific steps of performing image feature extraction on each segmented target area image include:
and extracting the approximate entropy and the sample entropy of each segmented target area image to be used as main features, and further extracting the color histogram, the color moment, the energy, the contrast, the texture entropy, the texture correlation and the image local binarization features of each segmented target area image to be used as supplementary features by using a color statistical feature extraction method, a gray level co-occurrence matrix method and a local binarization method.
Thermography images of the skin in embodiments of the present invention may successfully distinguish between the features of the pressure lesion area and the rest of the image. According to the relevant literature, the entropy features of thermographic images are the main identifying features.
In step S4, for the collected features, a plurality of machine learning methods (such as support vector machine algorithm, BP neural network algorithm, etc.) may be selected to establish an image recognition model, and after comparison, a method with the optimal performance and accuracy is selected, and a stable and ideal model is obtained through a plurality of tests; secondly, collecting multiple characteristics, establishing an image recognition model by using the same machine learning algorithm (optimal algorithm), selecting the characteristics most representing the pressure damage image after characteristic screening, and obtaining the model with higher recognition degree after testing.
The training set trained using different machine learning algorithms is a digital matrix of the number of skin thermographic images x the features of the skin thermographic images, which are the features automatically learned by the computer in the project. Different machine learning methods are applied to calculate the training set to obtain a model for identifying the new skin pressure injury thermal imaging image (namely, the probability of whether the new image is the pressure injury image or not can be calculated), so that the aim of early warning the pressure injury risk is fulfilled.
In step S5, acquiring a thermal imaging image of the skin to be detected and importing the image into the constructed thermal imaging image recognition model for detecting the skin pressure damage, and if the output category is 1, determining that the thermal imaging image of the skin to be detected is the thermal imaging image of the skin pressure damage; otherwise, if the output category is 0, determining that the skin thermal imaging image to be detected is not the skin pressure damage thermal imaging image. It should be noted that the output category attribute is determined by the setting of the tag in step S2.
As shown in fig. 2, a system for skin pressure injury thermal imaging image recognition in an embodiment of the present invention includes an obtaining unit 110, an image segmentation unit 120, an image feature extraction unit 130, a recognition model construction unit 140, and a result determination unit 150; wherein,
the acquiring unit 110 is configured to acquire a plurality of original skin pressure injury thermal imaging images;
the image segmentation unit 120 is configured to mark each original skin pressure damage thermal imaging image with a corresponding label, and further perform image segmentation processing on each original skin pressure damage thermal imaging image with an existing label to segment a target area image of each original skin pressure damage thermal imaging image with an existing label;
the image feature extraction unit 130 is configured to perform image feature extraction on each segmented target area image to obtain feature data after feature extraction of each target area image, and further perform preprocessing on the feature data after feature extraction of each target area image and establish a data set;
the identification model construction unit 140 is configured to train the preprocessed feature data in the data set based on a preset mechanical method, and construct a skin pressure injury thermal imaging image identification model;
the result determining unit 150 is configured to obtain a thermal imaging image of the skin to be detected, introduce the thermal imaging image into the constructed thermal imaging image recognition model for detecting, and determine whether the thermal imaging image of the skin to be detected is a thermal imaging image of the skin pressure damage.
Wherein the image segmentation unit 120 includes:
an image tag marking module 1201, configured to mark a tag as 1 if a matching degree between a certain original skin pressure damage thermal imaging image and a preset skin pressure damage thermal imaging image is greater than a preset threshold; otherwise, the label is marked as 0;
the image color processing module 1202 is configured to perform true color enhancement on the original skin pressure damage thermal imaging image of each existing label after all the original skin pressure damage thermal imaging image labels are marked; if the true color enhancement processing is that the image color is kept unchanged, the image brightness is enhanced;
the image segmentation module 1203 is configured to perform image segmentation processing on each original skin pressure damage thermal imaging image subjected to true color enhancement processing based on a preset color image segmentation method for visual color clustering, and segment a target area image of each labeled original skin pressure damage thermal imaging image.
Fig. 3 and fig. 4 are application scene diagrams of a system for thermal imaging image recognition of skin pressure injury according to an embodiment of the present invention.
From a large amount of clinical care monitoring data, the position of the body part where the pressure injury is most likely to occur is in the sacral tail, so the sacral tail skin is selected for temperature measurement during measurement, and after thermal imaging images are obtained, the images are interpreted by experienced personnel and marked for each image, as shown in fig. 3.
Through an image processing algorithm, the sacrococcygeal region of the target region is automatically identified by a computer, and irrelevant information in the original image is removed. The lower graph is an ideal segmentation effect graph. The location of the stage 1 skin pressure injury (arrows shown in fig. 4) can be easily found from the segmented graphs B, C and D.
After the image segmentation processing is finished, the features of each image are extracted by using a plurality of feature extraction methods, and the more classical methods comprise the following steps: the method comprises the steps of selecting a proper feature extraction method aiming at a thermal imaging image and collecting representative features.
And after finishing the image feature extraction link, carrying out sorting and normalization operation on the obtained image data set. And then, selecting a proper classification algorithm to train the images to obtain a thermal imaging image recognition model with higher recognition degree. The current widely used algorithms include a support vector machine, a random forest, an artificial neural network and the like, and a proper algorithm is selected for training. When a sufficient number of images are acquired, an attempt is also made to build a model using deep learning. After the model is initially built, new image data will be continuously collected, and the model is continuously optimized and tested to increase the stability and accuracy of the model.
Finally, acquiring a skin thermal imaging image to be detected and importing the skin thermal imaging image to be detected into the constructed skin pressure damage thermal imaging image recognition model for detection, and if the output category is 1, determining that the skin thermal imaging image to be detected is the skin pressure damage thermal imaging image; otherwise, if the output category is 0, determining that the skin thermal imaging image to be detected is not the skin pressure damage thermal imaging image.
As shown in fig. 5, a mobile phone provided in the embodiment of the present invention is used for thermal imaging image recognition of skin pressure injury, and includes an obtaining unit 210, an image segmentation unit 220, an image feature extraction unit 230, a recognition model construction unit 240, and a result determination unit 250; wherein,
the acquiring unit 210 is configured to acquire a plurality of original skin pressure injury thermal imaging images;
the image segmentation unit 220 is configured to mark each original skin pressure damage thermal imaging image with a corresponding label, and further perform image segmentation processing on each original skin pressure damage thermal imaging image with an existing label to segment a target area image of each original skin pressure damage thermal imaging image with an existing label;
the image feature extraction unit 230 is configured to perform image feature extraction on each segmented target area image to obtain feature data after feature extraction of each target area image, and further perform preprocessing on the feature data after feature extraction of each target area image and establish a data set;
the identification model constructing unit 240 is configured to train the preprocessed feature data in the data set based on a preset mechanical method, and construct a skin pressure injury thermal imaging image identification model;
the result determining unit 250 is configured to obtain a thermal imaging image of the skin to be detected, import the thermal imaging image into the constructed thermal imaging image recognition model for detecting, and determine whether the thermal imaging image of the skin to be detected is a thermal imaging image of the skin pressure damage.
Wherein the image segmentation unit 220 includes:
the image label marking module 2201 is configured to mark a label as 1 if a matching degree of a certain original skin pressure injury thermal imaging image and a preset skin pressure injury thermal imaging image is greater than a preset threshold; otherwise, the label is marked as 0;
the image color processing module 2202 is used for performing true color enhancement processing on the original skin pressure damage thermal imaging image of each existing label after all the original skin pressure damage thermal imaging image labels are marked; if the true color enhancement processing is that the image color is kept unchanged, the image brightness is enhanced;
the image segmentation module 2203 is configured to perform image segmentation on each true-color enhanced original skin pressure damage thermal imaging image based on a preset color image segmentation method for visual color clustering, and segment a target area image of each labeled original skin pressure damage thermal imaging image.
The embodiment of the invention has the following beneficial effects:
according to the invention, after an original skin pressure damage thermal imaging image is collected and subjected to label marking, image processing and target area segmentation, the characteristic data of the target area image is extracted, and the image and temperature indexes are analyzed by using a machine learning method to complete the construction of the pressure damage infrared imaging mode diagram, so that the identification of the pressure damage thermal imaging image is realized, and the method is not only fast and convenient, but also high in accuracy, thereby solving the problem that whether the pressure damage thermal imaging image is identified in the prior art.
It should be noted that, in the above system embodiment, each included unit is only divided according to functional logic, but is not limited to the above division as long as the corresponding function can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It will be understood by those skilled in the art that all or part of the steps in the method for implementing the above embodiments may be implemented by relevant hardware instructed by a program, and the program may be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc.
While the invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not to be limited to the disclosed embodiment, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.