CN117770774A - Nail fold microcirculation image processing system, method and electronic equipment - Google Patents
Nail fold microcirculation image processing system, method and electronic equipment Download PDFInfo
- Publication number
- CN117770774A CN117770774A CN202311864010.5A CN202311864010A CN117770774A CN 117770774 A CN117770774 A CN 117770774A CN 202311864010 A CN202311864010 A CN 202311864010A CN 117770774 A CN117770774 A CN 117770774A
- Authority
- CN
- China
- Prior art keywords
- image
- blood vessel
- video
- fold
- nail fold
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000004089 microcirculation Effects 0.000 title claims abstract description 31
- 210000004204 blood vessel Anatomy 0.000 claims abstract description 116
- 238000001514 detection method Methods 0.000 claims abstract description 23
- 210000000265 leukocyte Anatomy 0.000 claims description 28
- 239000013598 vector Substances 0.000 claims description 19
- 230000011218 segmentation Effects 0.000 claims description 14
- 238000013527 convolutional neural network Methods 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 10
- 238000004422 calculation algorithm Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 8
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000013528 artificial neural network Methods 0.000 claims description 5
- 238000007635 classification algorithm Methods 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 5
- 238000003672 processing method Methods 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims description 5
- 230000000007 visual effect Effects 0.000 claims description 4
- 230000006870 function Effects 0.000 claims description 2
- 238000012544 monitoring process Methods 0.000 claims 2
- 238000010586 diagram Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000001228 spectrum Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 239000002131 composite material Substances 0.000 description 2
- 238000005314 correlation function Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 208000024172 Cardiovascular disease Diseases 0.000 description 1
- 208000025747 Rheumatic disease Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 210000004027 cell Anatomy 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 206010012601 diabetes mellitus Diseases 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a nail fold microcirculation image processing system, a method and electronic equipment, which comprises the following steps: the device comprises an external lens module, an image pre-judging module and an image processing module; the external lens module is used for shooting a nail fold image and adjusting the magnification; the image pre-judging module is used for sending a control instruction for shooting a first nail fold image to the external lens module, wherein the control instruction for shooting the first nail fold image carries the first magnification; the image processing module is used for receiving the third fold image and the nail fold video and analyzing the third fold image and the nail fold video. The invention can realize miniaturized collection of blood vessel detection, has simpler and more convenient operation, facilitates the detection of the first wall of the required crowd at any time and any place, automatically marks the blood vessel and detects the blood vessel information through the image processing module, and effectively improves the detection efficiency.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a system, a method and electronic equipment for processing a nail fold microcirculation image.
Background
The nail mirror detection mainly comprises the steps of shooting the skin at the nail, and then acquiring relevant information of blood vessels and leucocytes of the skin at the nail according to the shot nail fold image, so as to acquire relevant parameters such as blood vessel width and the like. The existing medical first-class image sensing instrument is high in price and complex in operation, and if a mobile terminal is used for shooting, the limitation of a fixed scene of a hospital can be eliminated. However, there is a problem in that an ideal nail fold microcirculation image cannot be obtained due to insufficient magnification of the mobile terminal.
Disclosure of Invention
In order to solve the existing technical problems, the embodiment of the invention provides a system, a method and electronic equipment for processing a plication microcirculation image.
In a first aspect, the embodiment of the invention provides the external lens module, which is installed on the mobile terminal and is connected with the lens module of the mobile terminal;
the external lens module is used for sending a nail fold microcirculation image processing system when the image pre-judging module is received, and comprises: the device comprises an external lens module, an image pre-judging module and an image processing module;
when a control instruction for shooting a first nail fold image is sent, shooting the first nail fold image under a first magnification to obtain the first nail fold image, and sending the shot first nail fold image to the image pre-judging module; when a control instruction for shooting a second nail fold image sent by the image pre-judging module is acquired, shooting to obtain the second nail fold image under a second magnification, and sending the shot second nail fold image to the image pre-judging module; when a control instruction for shooting a third plication image and a plication video sent by the image pre-judging module is acquired, shooting the third plication image and the plication video under a third magnification, and sending the shot third plication image and the shot plication video to the image processing module; wherein the second magnification is greater than the first magnification, and the third magnification is greater than the second magnification;
the image pre-judging module is used for sending a control instruction for shooting a first nail fold image to the external lens module, wherein the control instruction for shooting the first nail fold image carries the first magnification;
the method comprises the steps of receiving a first nail fold image sent by an external lens module, classifying the first nail fold image by using a convolutional neural network model obtained through training, and sending a control instruction for shooting a second nail fold image to the external lens module when an obtained classification result indicates that the first nail fold image has a blood vessel, wherein the control instruction for shooting the second nail fold image carries the second magnification;
the second nail fold image sent by the external lens module is received, the second nail fold image is classified by utilizing a convolutional neural network model obtained through training, when the obtained classification result indicates that the second nail fold image is in a normal state, a control instruction for shooting a third nail fold image and a nail fold video is sent to the external lens module, and the control instruction for shooting the third nail fold image and the nail fold video carries the third magnification;
the image processing module is used for receiving the third fold image and the nail fold video and analyzing the third fold image and the nail fold video.
In a second aspect, an embodiment of the present invention further provides a method for processing a nail fold microcirculation image, which is applied to the nail wall microcirculation image processing system, and the method includes:
receiving the third plicated image and the plicated video;
the third plicated image and the plicated video are analyzed.
In a third aspect, an embodiment of the present invention provides an electronic device including a bus, a transceiver, a memory, a processor, and a computer program stored on the memory and executable on the processor, the transceiver, the memory, and the processor being connected by the bus, the computer program implementing the steps in the plication microcirculation image processing method of any one of the second aspects when executed by the processor.
In a fourth aspect, an embodiment of the present invention further provides a computer readable storage medium having a computer program stored thereon, which when executed by a processor, implements the steps of the plication microcirculation image processing method according to any one of the second aspects.
In the solutions provided in the first to fourth aspects of the present application, a control instruction is sent to an external lens module through an image pre-judging module, the external lens module firstly shoots a first wall image according to the control instruction, if no blood vessel exists in the first wall image, adjusts the position of the external lens module and shoots a second wall image, and when the blood vessel exists in the second wall image, adjusts the magnification factor to shoot a third wall image and shoot a wall video, and analyzes the third wall image and the wall video through an image processing module to obtain relevant information of the blood vessel and leucocyte; compared with the prior art that the nail wall image can be shot only by a large nail mirror detection instrument in a hospital, the blood vessel detection can be realized by using the mobile terminal externally connected with the lens module, the detection is not limited by time and space, and the operation is simple and convenient; meanwhile, the image processing module automatically marks the blood vessel and detects blood vessel information, so that the detection efficiency is effectively improved.
In order to make the above objects, features and advantages of the present invention more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly describe the embodiments of the present invention or the technical solutions in the background art, the following description will describe the drawings that are required to be used in the embodiments of the present invention or the background art.
FIG. 1 is a schematic diagram showing the connection of the various modules of the plication microcirculation image processing system provided by an embodiment of the present invention;
fig. 2 is a schematic diagram showing an installation relationship between a rotating motor and a plurality of cameras in an external lens module according to an embodiment of the present invention;
FIG. 3 is a flow chart of a method for processing a plication microcirculation image according to an embodiment of the present invention;
fig. 4 shows a schematic diagram of an electronic device of the method for processing a plication microcirculation image according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention. The present invention will be further described in detail below with reference to the drawings and detailed description for the purpose of enabling those skilled in the art to better understand the aspects of the present invention.
The first-pass detection is an analysis of a first-wall microcirculation image, is one of main methods of non-invasive microcirculation detection, and is widely applied to diagnosis of rheumatic diseases, diabetes and cardiovascular diseases. Currently, the first-class mirror detection can only be realized by a first-class mirror detection instrument special for hospitals. The special first-lens detecting instrument has the phenomena of high price, limited functions, complex operation and the like. Meanwhile, the processing mode of the nail-wall image by the nail-mirror detecting instrument is not manually marked by medical staff, so that time and labor are consumed in the marking process, and the detecting efficiency is low.
Therefore, in order to solve the problems, the invention provides a mobile terminal-based nail wall microcirculation image processing system, so that the nail mirror detection is realized without being limited to medical places, the rapid nail mirror detection can be performed anytime and anywhere, and the operation is simple.
The implementation main body of the A-wall microcirculation image processing system provided by the embodiment of the invention is a mobile terminal.
In one embodiment, the mobile terminal includes, but is not limited to: cell phones and tablet computers.
Example 1
The embodiment of the invention provides a nail wall microcirculation image processing system, referring to a schematic connection diagram of each module of the nail fold microcirculation image processing system shown in fig. 1, the system comprises: an external lens module 100, an image pre-judging module 101 and an image processing module 102.
The external lens module is arranged on the mobile terminal and is connected with the lens module of the mobile terminal.
An external lens module 100, configured to, when receiving a control instruction for capturing a first nail fold image sent by the image pre-determining module, capture the first nail fold image under a first magnification, and send the captured first nail fold image to the image pre-determining module; when a control instruction for shooting a second nail fold image sent by the image pre-judging module is acquired, shooting to obtain the second nail fold image under a second magnification, and sending the shot second nail fold image to the image pre-judging module; when a control instruction for shooting a third plication image and a plication video sent by the image pre-judging module is acquired, shooting the third plication image and the plication video under a third magnification, and sending the shot third plication image and the shot plication video to the image processing module; wherein the second magnification is greater than the first magnification, and the third magnification is greater than the second magnification.
Specifically, referring to the schematic diagram of the installation relationship between the rotating motor and the plurality of cameras in the external lens module shown in fig. 2, the external lens module cannot exist independently and needs to be installed on the mobile terminal for use. The external lens module comprises a plurality of cameras, a singlechip, a rotating motor and an annular aperture, wherein the number of the cameras is different in magnification, the singlechip is connected with a mobile phone interface through a data line, and the brightness of the annular aperture is adjusted; preferably, the magnification of the imaging lens may be 20 times, 50 times, and 200 times. The cameras are connected with the rotating motor through the DuPont wire, so that the switching of the cameras can be realized. The annular aperture is connected with the rotating motor through the DuPont wire, and the mobile terminal can control the change of the illumination intensity of the annular aperture through controlling the output voltage.
In one embodiment, the external lens module performs three times of nail wall image shooting in total, only the camera with the first magnification is used for test shooting when shooting for the first time, whether a blood vessel exists at the shooting position or not is checked, if the first shot image does not exist, the second magnification shooting is performed at the moving position, when the second shot image does not exist, the magnification of the camera is switched after the blood vessel is shot for the second time, the second shot image is classified, if the second shot image is a qualified image, the third magnification shooting is performed, the magnification is further switched, and if the second shot image is a disqualified image, the external lens module is adjusted according to the disqualified type. In particular, the image classification comprises: image clarity and image ambiguity, wherein image ambiguity in turn includes image blur, image over-brightness, and image over-darkness. The focal length of the camera of the mobile terminal can be adjusted aiming at image blurring, and if the image is too bright or too dark, the light intensity of the annular aperture can be reduced or increased for adjustment. In particular, the first magnification of the camera used for capturing the first wall image is 20, the second magnification of the camera used for capturing the second wall image is 50, and the third magnification of the camera used for capturing the third wall image is 200. It should be noted that the camera needs to be gradually increased when the camera is switched, if a camera 200 times is used for shooting the first wall image, the search of the blood vessel area can be missed, and the central focus area of the image can not be gradually adjusted, so that the blood vessel area is easy to lose in the amplifying process.
An image pre-judging module 101, configured to send a control instruction for shooting a first plicated image to the external lens module, where the control instruction for shooting the first plicated image carries the first magnification; the method comprises the steps of receiving a first nail fold image sent by an external lens module, classifying the first nail fold image by utilizing a convolutional neural network model obtained through training, sending a control instruction for shooting a second nail fold image to the external lens module when an obtained classification result indicates that the first nail fold image has a blood vessel, wherein the control instruction for shooting the second nail fold image carries the second magnification factor, and particularly, when the obtained classification result indicates that the first nail fold image does not have the blood vessel, prompting information to a user to move the external lens module through a mobile terminal. The prompt information may be voice broadcast or screen text display, etc., and the voice broadcast and screen text display performed by using the mobile terminal are common knowledge in the art, so that the principle thereof will not be described in detail;
the image pre-judging module 101 receives the second nail fold image sent by the external lens module, classifies the second nail fold image by utilizing the convolutional neural network model obtained through training, and sends a control instruction for shooting the third nail fold image and the nail fold video to the external lens module when the obtained classification result indicates that the second nail fold image is in a normal state. Otherwise, when the second first wall image is in an abnormal state, the external lens module does not receive a control instruction for shooting the third wall image and the first wall video. Wherein, the abnormal state of the second first wall image means: image blurring, poor sharpness, over-bright or over-dark images, etc.
Specifically, the image pre-judging module can classify and identify the images shot by the external lens module, and when the identification result is a blurred, too bright or too dark image, a control instruction is sent to a singlechip in the external lens module to control the annular aperture to increase or decrease the light intensity. In particular, the exposure intensity and sharpness of the identification image are of the prior art and will not be described in detail here.
An image processing module 102 for receiving the third plication image and the plication video and analyzing the third plication image and the plication video.
Specifically, the image processing module may be stored in the mobile terminal or in the cloud server. When the image processing module analyzes the composite image (i.e., the third wall image and the first wall video), the image processing module performs the following steps (1) to (9):
(1) Preprocessing the third fold image and converting the third fold image into a feature map;
in the above step (1), the third wall image is converted into a feature map, and the following sub-step A1-sub-step A2 are performed:
a1, firstly, sending the third fold image to the image pre-judging module for classification;
a2, when the received classification result returned by the image pre-judging module indicates that the third fold image is in a normal state, performing brightness matrix and chromaticity matrix transformation on the light intensity color of the third fold image based on a basic image template to obtain the feature map.
Specifically, the basic image template in the step A2 is preset for people, the definition, brightness, chromaticity and the like of any image can be adjusted in advance according to requirements, and the adjusted image is pre-stored in the mobile terminal or the cloud server as the basic image template.
(2) Dividing the feature map into a plurality of areas, and extracting feature vectors of each area in the plurality of areas;
in the step (2), the feature vector means a blood vessel feature contained in each region.
Specifically: restoring the feature map to the original resolution, extracting the blood vessel features by utilizing a convolutional neural network model through up-sampling and down-sampling, outputting a binary segmentation map with the same size as the basic image template after extraction, and counting the number of selection frames in the map based on the binary segmentation map to determine the density of the blood vessel. In particular, the resolution of the picture is restored to the prior art, the principle of which is not repeated here.
(3) Classifying the feature vectors by using a neural network classification algorithm, determining the areas where the feature vectors are located, and selecting the area with the feature vectors from the areas as a candidate area; in particular, other algorithms may be used to classify the feature vectors, including but not limited to decision tree algorithms, support vector machine algorithms, or random forest algorithms. Therefore, the algorithm in this embodiment cannot be understood as only the neural network classification algorithm can classify the feature vector.
(4) Extracting a blood vessel image in the region to be selected, processing the region to be selected by a contour extraction method, and determining the length of a blood vessel in the blood vessel image;
(5) Selecting characteristic points located in the edge area in the blood vessel image as blood vessel key points, and obtaining the diameter of the blood vessel by calculating the distance between two blood vessel key points in the normal direction of the blood vessel wall;
in the step (5), the selection of the blood vessel key points refers to the crossing points between the top upper, lower, left and right key points and the blood vessel, and the key points can be obtained through labeling and convolutional neural network model training.
(6) Reading each frame of video image in the nail fold video, and synchronously acquiring the position information of the white blood cells contained in each frame of video image;
in the step (6), the location information of the white blood cells refers to the location of the white blood cells in each frame of the video image. Particularly, the white blood cells are obvious in the blood vessel chart, and the white blood cells in the blood vessel are distinguished according to the RGB color range by a simple color threshold dividing method, wherein the specific process of inquiring the white blood cells by the color threshold dividing method is the prior art and is not repeated here.
(7) Acquiring a blood vessel segmentation graph in each frame of video image by utilizing image processing, and calculating tangential field information and distance field information of blood vessels in the blood vessel segmentation graph;
in the above step (7), the tangential field information indicates a change in the direction of the blood vessel, and the distance field information indicates a distance between the blood vessel and the key point. In particular, the calculation of tangential field information and distance field information belongs to the prior art, and thus the detailed calculation process thereof is not repeated.
Further, after the tangential field information is obtained, the tangential field information can be optimized, bad connection is eliminated, and optimized tangential field information is obtained.
(8) Determining a distance between the leukocytes and flow rate information of the leukocytes in the blood vessel using the position information and distance field information in the leukocytes;
in the above step (8), the process of determining the distance between the white blood cells and the flow rate of the white blood cells using the position information and the distance field information of the white blood cells belongs to the existing process, and the detailed calculation process thereof will not be repeated.
(9) And outputting the distance between the white blood cells, the position information, the tangential field information and the flow velocity information of the white blood cells as a visual result.
Further, when the image processing module analyzes the composite image (the third wall image and the first wall video), the image processing module may further perform the following steps B1 to B2:
b1, detecting key points of each frame of video image in the nail fold video to obtain key points of the video frame image, and performing preliminary alignment by utilizing the key points of the video frame image and the feature map;
in the step B1, the preliminary alignment of the key points and the feature map is achieved by performing matrix division on the positions of the key points of the feature map and the positions of the key points in the video frame image to obtain an affine transformation matrix.
And B2, extracting a blood vessel image from the video frame image by utilizing a blood vessel region segmentation algorithm, aligning the extracted blood vessel image by using a phase correlation method, and returning to the step (6).
In the step B2, the object to be aligned by the phase correlation method may be a first frame image in the video frame, and the first frame image is used as a reference, and the blood vessel images in the remaining video frame images are aligned with the blood vessel images in the first frame image. The position difference between the two images represents the pixel offset of the two images in the horizontal and vertical directions, and the blood vessel image in the remaining video frame image is aligned with the blood vessel image in the first frame image by using the position difference between the blood vessel image in the remaining video frame image and the blood vessel image in the first frame image. Specifically, fourier spectrums of the first frame image and the remaining frame images may be obtained by fourier transform and multiplied to obtain a cross power spectrum, and the cross power spectrum may be inverse fourier transformed to obtain a cross correlation function. The peak value of the cross correlation function represents the offset position of the blood vessel image in the first frame image and any residual frame image, the offset position is the position difference of the blood vessel image in the first frame image and any residual frame image, and the position difference refers to the pixel offset of the first frame image and the residual frame image in the horizontal and vertical directions.
In summary, the present embodiment provides a first wall microcirculation image processing system, wherein an image pre-judging module sends a control instruction to an external lens module, the external lens module firstly shoots a first wall image according to the control instruction, if no blood vessel exists in the first wall image, adjusts the position of the external lens module and shoots a second wall image, and when the blood vessel exists in the second wall image, adjusts the magnification to shoot a third wall image and shoot a wall video, and the image processing module analyzes the third wall image and the wall video to obtain relevant information of the blood vessel and leucocyte; the mobile terminal externally connected with the lens module can be used for realizing blood vessel detection, and the detection is not limited by time and space, so that the operation is simple and convenient; meanwhile, the image processing module automatically marks the blood vessel and detects blood vessel information, so that the detection efficiency is effectively improved.
Example 2
The embodiment of the invention also provides a method for processing the nail fold microcirculation image, which is applied to the nail fold microcirculation image processing system of the embodiment 1, and is shown in a flow diagram of the method for processing the nail fold microcirculation image shown in fig. 3, and the method comprises the following steps:
step 200: receiving the third plicated image and the plicated video;
step 201: the third plicated image and the plicated video are analyzed.
Further, the step 204 includes:
s1: preprocessing the third fold image and converting the third fold image into a feature map;
s2: dividing the feature map into a plurality of areas, and extracting feature vectors of each area in the plurality of areas;
s3: classifying the feature vectors by using a neural network classification algorithm, determining the areas where the feature vectors are located, and selecting the area with the feature vectors from the areas as a candidate area;
s4: extracting a blood vessel image in the region to be selected, processing the region to be selected by a contour extraction method, and determining the length of a blood vessel in the blood vessel image;
s5: selecting characteristic points in the edge area in the blood vessel image as key points, and calculating the distance between two key points on the same horizontal line to obtain the diameter of the blood vessel;
s6: reading each frame of video image in the nail fold video, and synchronously acquiring the information of the white blood cells contained in each frame of video image;
s7: acquiring a blood vessel segmentation graph in each frame of video image by utilizing image processing, and calculating tangential field information and distance field information of blood vessels in the blood vessel segmentation graph;
s8: optimizing the tangential field information to eliminate bad connection;
s9: determining a distance between the leukocytes and flow rate information of the leukocytes in the blood vessel using the position information and distance field information in the leukocytes;
s10: and outputting the distance between the white blood cells, the position information, the tangential field information and the flow velocity information of the white blood cells as a visual result.
Still further, the step 204 further includes:
performing key point detection on each frame of video image in the nail fold video to obtain key points of the video frame image, and performing preliminary alignment by utilizing the key points of the video frame image and the feature map;
and extracting a blood vessel image from the video frame image by using a blood vessel region segmentation algorithm, aligning the extracted blood vessel image by using a phase correlation method, namely calculating the Fourier cross energy spectrum of the two images to obtain the position difference of the two images, and performing secondary alignment on the images.
Still further, the step S1 includes:
sending the third plicated image to the image pre-judgment module for classification;
and when the received classification result returned by the image pre-judging module indicates that the third fold image is in a normal state, performing brightness matrix and chromaticity matrix transformation on the light intensity color of the third fold image based on a basic image template to obtain the feature map.
In summary, the embodiment provides a method for processing a micro-circulation image of a first wall, wherein the image pre-judging module sends a control command to the external lens module, the external lens module firstly shoots a first wall image according to the control command, if no blood vessel exists in the first wall image, adjusts the position of the external lens module and shoots a second wall image, when the blood vessel exists in the second wall image, adjusts the magnification to shoot a third wall image and shoot a wall video, and the image processing module analyzes the third wall image and the wall video to obtain relevant information of the blood vessel and leucocyte; the mobile terminal externally connected with the lens module can be used for realizing blood vessel detection, and the detection is not limited by time and space, so that the operation is simple and convenient; meanwhile, the image processing module automatically marks the blood vessel and detects blood vessel information, so that the detection efficiency is effectively improved.
Example 3
The present embodiment proposes a computer-readable storage medium having a computer program stored thereon, which when executed by a processor performs the steps of the plication microcirculation image processing method described in embodiment 2 above. The specific implementation can be referred to method embodiment 2, and will not be described herein.
In addition, referring to the schematic structural diagram of an electronic device shown in fig. 4, the present embodiment further proposes an electronic device, which includes a bus 300, a processor 301, a transceiver 302, a bus interface 303, a memory 304, and a user interface 305. The electronic device includes a memory 304.
In this embodiment, the electronic device further includes: one or more programs stored on memory 304 and executable on processor 301, configured to be executed by the processor for performing steps (1) through (5) below:
(1) Connecting the external lens module with a lens module of the mobile terminal; starting the image pre-judging module and sending a control instruction for shooting a first nail fold image, wherein the control instruction of the first nail fold image carries a first magnification; the external lens module shoots and obtains a first nail fold image under a first magnification, and the shot first nail fold image is sent to the image pre-judging module;
(2) The image pre-judging module is used for sending a control instruction for shooting a second nail fold image again, the control instruction of the second nail fold image carries a second method multiple, the second nail fold image is shot and obtained under the second magnification, and the shot second nail fold image is sent to the image pre-judging module; when a control instruction for shooting a third plication image and a plication video, which is sent by the image pre-judging module, is acquired, shooting the third plication image and the plication video, and sending the shot third plication image and the shot plication video to the image processing module; wherein the second magnification is greater than the first magnification;
(3) The method comprises the steps of receiving a first nail fold image sent by an external lens module, classifying the first nail fold image by using a convolutional neural network model obtained through training, and sending a control instruction for shooting a second nail fold image to the external lens module when an obtained classification result indicates that the first nail fold image has a blood vessel, wherein the control instruction for shooting the second nail fold image carries the second magnification;
(4) Receiving a second nail fold image sent by the external lens module, classifying the second nail fold image by using a convolutional neural network model obtained through training, and sending a control instruction for shooting a third nail fold image and a nail fold video to the external lens module when the obtained classification result indicates that the second nail fold image is in a normal state;
(5) The image processing module is activated and receives the third plication image and the plication video and analyzes the third plication image and the plication video.
A transceiver 302 for receiving and transmitting data under the control of the processor 301.
Where bus architecture (represented by bus 300), bus 300 may comprise any number of interconnected buses and bridges, with bus 300 linking together various circuits, including one or more processors, as represented by processor 301, and memory, as represented by memory 304. Bus 300 may also link together various other circuits such as peripheral devices, voltage regulators, power management circuits, etc., as are well known in the art and, therefore, will not be described further herein. Bus interface 303 provides an interface between bus 300 and transceiver 302. The transceiver 302 may be one element or may be a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. For example: the transceiver 302 receives external data from other devices. The transceiver 302 is used to transmit the data processed by the processor 301 to other devices. Depending on the nature of the computing system, a user interface 305 may also be provided, such as a keypad, display, speaker, microphone, joystick.
The processor 301 is responsible for managing the bus 300 and general processing as described above for running the general operating system 3041. And memory 304 may be used to store data used by processor 301 in performing operations.
Alternatively, the processor 301 may be, but is not limited to: a central processing unit, a single chip microcomputer, a microprocessor or a programmable logic device.
It is to be appreciated that the memory 304 in embodiments of the present application can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM) which acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (Double Data Rate SDRAM), enhanced SDRAM (ESDRAM), synchronous DRAM (SLDRAM), and Direct RAM (DRRAM). The memory 304 of the system and method described in this embodiment is intended to comprise, without being limited to, these and any other suitable types of memory.
In some implementations, the memory 304 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof: an operating system 3041 and application programs 3042.
The operating system 3041 includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks. The application 3042 includes various application programs such as a Media Player (Media Player), a Browser (Browser), and the like for realizing various application services. A program implementing the method of the embodiment of the present application may be included in the application program 3042.
The foregoing is merely a specific implementation of the embodiment of the present invention, but the protection scope of the embodiment of the present invention is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the embodiment of the present invention, and the changes or substitutions are covered by the protection scope of the embodiment of the present invention. Therefore, the protection scope of the embodiments of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A plication microcirculation image processing system, comprising: the device comprises an external lens module, an image pre-judging module and an image processing module;
the external lens module is arranged on the mobile terminal and is connected with the lens module of the mobile terminal;
the external lens module is used for shooting under a first magnification to obtain a first nail fold image when receiving a control instruction for shooting the first nail fold image sent by the image pre-judging module, and sending the shot first nail fold image to the image pre-judging module; when a control instruction for shooting a second nail fold image sent by the image pre-judging module is acquired, shooting to obtain the second nail fold image under a second magnification, and sending the shot second nail fold image to the image pre-judging module; when a control instruction for shooting a third plication image and a plication video sent by the image pre-judging module is acquired, shooting the third plication image and the plication video under a third magnification, and sending the shot third plication image and the shot plication video to the image processing module; wherein the second magnification is greater than the first magnification, and the third magnification is greater than the second magnification;
the image pre-judging module is used for sending a control instruction for shooting a first nail fold image to the external lens module, wherein the control instruction for shooting the first nail fold image carries the first magnification;
receiving a first nail fold image sent by the external lens module, and classifying the first nail fold image by using a convolutional neural network model obtained by training:
when the obtained classification result indicates that the first nail fold image has a blood vessel, a control instruction for shooting a second nail fold image is sent to the external lens module, and the control instruction for shooting the second nail fold image carries the second magnification;
the second nail fold image sent by the external lens module is received, the second nail fold image is classified by utilizing a convolutional neural network model obtained through training, when the obtained classification result indicates that the second nail fold image is in a normal state, a control instruction for shooting a third nail fold image and a nail fold video is sent to the external lens module, and the control instruction for shooting the third nail fold image and the nail fold video carries the third magnification;
the image processing module is used for receiving the third fold image and the nail fold video and analyzing the third fold image and the nail fold video.
2. The system of claim 1 wherein the image processing module for analyzing the third plication image and the plication video comprises:
preprocessing the third fold image and converting the third fold image into a feature map;
dividing the feature map into a plurality of areas, and extracting feature vectors of each area in the plurality of areas;
classifying the feature vectors by using a neural network classification algorithm, determining the areas where the feature vectors are located, and selecting the area with the feature vectors from the areas as a candidate area;
extracting a blood vessel image in the region to be selected, processing the region to be selected by a contour extraction method, and determining the length of a blood vessel in the blood vessel image;
selecting a blood vessel key point in a blood vessel image, and obtaining the diameter of the blood vessel by calculating the distance between two blood vessel key points in the normal direction of the blood vessel wall;
and acquiring a blood vessel segmentation graph of each frame of video image in the wall video by using an image processing method, and calculating tangential field information and distance field information of blood vessels in the blood vessel segmentation graph.
3. The system of claim 2 wherein the image processing module for analyzing the third plication image and the plication video further comprises:
performing key point detection on each frame of video image in the nail fold video to obtain key points of the video frame image, and performing preliminary alignment on the key points in the video frame image by utilizing the blood vessel key points obtained by the feature map;
extracting a blood vessel image from the video frame image by using a blood vessel region segmentation algorithm, and aligning the extracted blood vessel image by using a phase correlation method;
determining the position information of the white blood cells and the flow velocity information of the white blood cells by utilizing a color threshold dividing method through the aligned blood vessel images;
and outputting the visual results of the position information of the white blood cells, the flow velocity information of the white blood cells, the tangential field information of the blood vessels, the distance field information of the blood vessels, the length of the blood vessels and the diameter of the blood vessels.
4. The system of claim 2 wherein the image processing module for preprocessing the third plicated image to convert the third plicated image into a feature map comprises:
sending the third plicated image to the image pre-judgment module for classification;
and when the received classification result returned by the image pre-judging module indicates that the third fold image is in a normal state, performing brightness matrix and chromaticity matrix transformation on the light intensity color of the third fold image based on a basic image template to obtain the feature map.
5. A method of plication microcirculation image processing for implementing the functions of an image processing module in a plication microcirculation image processing system according to any of claims 1-4, the method comprising:
receiving the third plicated image and the plicated video;
the third plicated image and the plicated video are analyzed.
6. The method of claim 5 wherein the analyzing the third plicated image and the plicated video comprises:
preprocessing the third fold image and converting the third fold image into a feature map;
dividing the feature map into a plurality of areas, and extracting feature vectors of each area in the plurality of areas;
classifying the feature vectors by using a neural network classification algorithm, determining the areas where the feature vectors are located, and selecting the area with the feature vectors from the areas as a candidate area;
extracting a blood vessel image in the region to be selected, processing the region to be selected by a contour extraction method, and determining the length of a blood vessel in the blood vessel image;
selecting a blood vessel key point in a blood vessel image, and obtaining the diameter of the blood vessel by calculating the distance between two blood vessel key points in the normal direction of the blood vessel wall;
and acquiring a blood vessel segmentation graph of each frame of video image in the wall video by using an image processing method, and calculating tangential field information and distance field information of blood vessels in the blood vessel segmentation graph.
7. The method of claim 5 wherein the analyzing the third plicated image and the plicated video further comprises:
performing key point detection on each frame of video image in the nail fold video to obtain key points of the video frame image, and performing preliminary alignment on the key points in the video frame image by utilizing the blood vessel key points obtained by the feature map;
extracting a blood vessel image from the video frame image by using a blood vessel region segmentation algorithm, and aligning the extracted blood vessel image by using a phase correlation method;
determining the position information of the white blood cells and the flow velocity information of the white blood cells by utilizing a color threshold dividing method through the aligned blood vessel images;
and outputting the visual results of the position information of the white blood cells, the flow velocity information of the white blood cells, the tangential field information of the blood vessels, the distance field information of the blood vessels, the length of the blood vessels and the diameter of the blood vessels.
8. The method of claim 6 wherein the preprocessing the third plicated image to convert the third plicated image to a feature map comprises:
sending the third plicated image to the image pre-judgment module for classification;
and when the received classification result returned by the image pre-judging module indicates that the third fold image is in a normal state, performing brightness matrix and chromaticity matrix transformation on the light intensity color of the third fold image based on a basic image template to obtain the feature map.
9. An electronic device comprising a bus, a transceiver, a memory, a processor, and a computer program stored on the memory and executable on the processor, the transceiver, the memory, and the processor being connected by the bus, wherein the computer program when executed by the processor implements the steps in the plication microcirculation monitoring method of any one of claims 5-8.
10. A computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor performs the steps in the plication microcirculation monitoring method of any one of claims 5 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311864010.5A CN117770774A (en) | 2023-12-29 | 2023-12-29 | Nail fold microcirculation image processing system, method and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311864010.5A CN117770774A (en) | 2023-12-29 | 2023-12-29 | Nail fold microcirculation image processing system, method and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117770774A true CN117770774A (en) | 2024-03-29 |
Family
ID=90394544
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311864010.5A Pending CN117770774A (en) | 2023-12-29 | 2023-12-29 | Nail fold microcirculation image processing system, method and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117770774A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118469957A (en) * | 2024-05-17 | 2024-08-09 | 北京信息科技大学 | Microscopic imaging-based nail fold microcirculation white blood cell counting method and device |
-
2023
- 2023-12-29 CN CN202311864010.5A patent/CN117770774A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118469957A (en) * | 2024-05-17 | 2024-08-09 | 北京信息科技大学 | Microscopic imaging-based nail fold microcirculation white blood cell counting method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020192483A1 (en) | Image display method and device | |
| JP7140580B2 (en) | stereo imager | |
| US9104914B1 (en) | Object detection with false positive filtering | |
| CN108229277B (en) | Gesture recognition method, gesture control method, multilayer neural network training method, device and electronic equipment | |
| WO2020207423A1 (en) | Skin type detection method, skin type grade classification method and skin type detection apparatus | |
| CN109167924B (en) | Video imaging method, system, device and storage medium based on hybrid camera | |
| WO2020078229A1 (en) | Target object identification method and apparatus, storage medium and electronic apparatus | |
| CN111027504A (en) | Face key point detection method, device, equipment and storage medium | |
| US9384386B2 (en) | Methods and systems for increasing facial recognition working rang through adaptive super-resolution | |
| CN109844804B (en) | Image detection method, device and terminal | |
| CN111915483B (en) | Image stitching method, device, computer equipment and storage medium | |
| CN111160202A (en) | AR equipment-based identity verification method, AR equipment-based identity verification device, AR equipment-based identity verification equipment and storage medium | |
| CN101983507A (en) | Automatic redeye detection | |
| CN117770774A (en) | Nail fold microcirculation image processing system, method and electronic equipment | |
| CN112651358A (en) | Target detection method and device, network camera and storage medium | |
| CN113158773B (en) | Training method and training device for living body detection model | |
| CN110674729A (en) | Method for identifying number of people based on heat energy estimation, computer device and computer readable storage medium | |
| CN110855957A (en) | Image processing method and device, storage medium and electronic equipment | |
| CN110929615B (en) | Image processing method, image processing apparatus, storage medium, and terminal device | |
| CN114332993A (en) | Face recognition method and device, electronic equipment and computer readable storage medium | |
| CN117994542A (en) | Foreign matter detection method, device and system | |
| CN112036277A (en) | Face recognition method, electronic equipment and computer readable storage medium | |
| JP3980464B2 (en) | Method for extracting nose position, program for causing computer to execute method for extracting nose position, and nose position extracting apparatus | |
| CN111353361A (en) | Face recognition method and device and electronic equipment | |
| KR102194511B1 (en) | Representative video frame determination system and method using same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |