CN115862821A - Construction method of intelligent operating room based on digital twins and related device - Google Patents
Construction method of intelligent operating room based on digital twins and related device Download PDFInfo
- Publication number
- CN115862821A CN115862821A CN202310122298.2A CN202310122298A CN115862821A CN 115862821 A CN115862821 A CN 115862821A CN 202310122298 A CN202310122298 A CN 202310122298A CN 115862821 A CN115862821 A CN 115862821A
- Authority
- CN
- China
- Prior art keywords
- digital twin
- point cloud
- information
- cloud data
- twin
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Processing Or Creating Images (AREA)
Abstract
The invention relates to the field of data processing, and discloses a construction method of an intelligent operating room based on digital twins and a related device, which are used for improving the accuracy of patient information processing. The method comprises the following steps: acquiring patient information of a target patient, and performing type matching on the patient information to obtain a corresponding information type; constructing a finite element model based on the information type to obtain a corresponding initial digital twin body; performing multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body; carrying out mesh division on the initial digital twin body based on multi-angle image information, and determining a plurality of corresponding twin body meshes; modifying the initial digital twins based on the twin grids to obtain target digital twins; and carrying out operation instruction setting on the target digital twin, and transmitting the target digital twin set by the operation instruction to the three-dimensional operation terminal.
Description
Technical Field
The invention relates to the field of data processing, in particular to a construction method of an intelligent operating room based on digital twins and a related device.
Background
In the traditional operation process, doctors often encounter various situations, for example, the vision can be blocked by surgical instruments, the characteristics of a patient at multiple angles are generally required to be checked in real time, errors can be caused due to the fact that the operation visual field is not clear, operation position points are difficult to find, and operation risks are increased.
Therefore, in the process of operating on a patient, a virtual simulation physical model is urgently needed to help a doctor find an operating position more accurately, so that the processing efficiency of the patient information is improved, and the accuracy of selecting the operating position in the process of operating on the patient in the operation process is further improved.
Disclosure of Invention
The invention provides a digital twin-based intelligent operating room construction method and a related device, which are used for improving the accuracy of patient data processing.
The invention provides a construction method of a digital twin-based intelligent operating room, which comprises the following steps: acquiring patient information of a target patient, and performing type matching on the patient information to obtain a corresponding information type; constructing a finite element model based on the information type to obtain a corresponding initial digital twin body; performing multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body; meshing the initial digital twin body based on the multi-angle image information to determine a plurality of corresponding twin body meshes; modifying the initial digital twins based on the twin meshes to obtain target digital twins; and carrying out operation instruction setting on the target digital twin body, and transmitting the target digital twin body subjected to the operation instruction setting to a three-dimensional operation terminal.
With reference to the first aspect, in a first implementation manner of the first aspect of the present invention, the performing finite element model construction based on the information type to obtain a corresponding initial digital twin includes: performing unit matching on the information types through a preset finite element model database to obtain a corresponding information unit set; performing parameter analysis on the information unit set to obtain unit parameter information corresponding to the information unit set; and carrying out model construction through the unit parameter information to obtain a corresponding initial digital twin body.
With reference to the first aspect, in a second implementation manner of the first aspect of the present invention, the performing a multi-angle scan on the initial digital twin to obtain multi-angle image information corresponding to the digital twin includes: determining the space position of the initial digital twin object based on a preset space coordinate system to obtain a corresponding space position coordinate; determining a corresponding scanning angle information set based on the spatial position coordinates; and performing multi-angle scanning on the initial digital twin body based on the scanning angle information set to obtain multi-angle image information corresponding to the digital twin body.
With reference to the second embodiment of the first aspect, in a third embodiment of the first aspect of the present invention, the mesh-dividing the initial digital twin based on the multi-angle image information and determining corresponding multiple twin meshes includes: performing key position point analysis on the initial digital twin through the space coordinate system, and determining a plurality of key position points corresponding to the initial digital twin; performing area expansion according to each key position point to obtain a plurality of corresponding expanded three-dimensional areas; performing image matching on the plurality of expanded three-dimensional areas and the multi-angle image information to obtain image information to be processed corresponding to each expanded three-dimensional area; and respectively performing sweeping division processing on the image information to be processed corresponding to each expanded three-dimensional area to obtain a plurality of corresponding twin body grids.
With reference to the first aspect, in a fourth embodiment of the first aspect of the present invention, the modifying the initial digital twin based on the plurality of twin meshes to obtain a target digital twin includes: carrying out virtual point cloud data mapping on the twin body grids to obtain corresponding virtual point cloud data sets; performing virtual semantic assignment on the virtual point cloud data set to obtain corresponding virtual semantic point cloud data; carrying out depth truth value correction on the virtual semantic point cloud data to obtain a target point cloud data set; and correcting the initial digital twin body through the target point cloud data set to obtain a target digital twin body.
With reference to the fourth embodiment of the first aspect, in a fifth embodiment of the first aspect of the present invention, the performing virtual point cloud data mapping on the plurality of twin body meshes to obtain corresponding virtual point cloud data sets includes: performing texture image conversion on the twin body meshes to obtain a plurality of corresponding twin body texture images; performing two-dimensional feature matching on the twin body texture images based on a preset feature database to obtain a target two-dimensional feature set corresponding to the twin body texture images; carrying out mapping relation matching based on the target two-dimensional feature set to obtain a corresponding point cloud mapping relation; and performing virtual point cloud data mapping on the twin body grids through the point cloud mapping relation to obtain a corresponding virtual point cloud data set.
With reference to the fourth implementation manner of the first aspect, in a sixth implementation manner of the first aspect of the present invention, the performing virtual semantic assignment on the virtual point cloud data set to obtain corresponding virtual semantic point cloud data includes: performing semantic matching on the virtual point cloud data set based on a preset point cloud semantic database to obtain semantic information corresponding to the virtual point cloud data set; and performing virtual semantic assignment on the virtual point cloud data set based on the semantic information to obtain corresponding virtual semantic point cloud data.
The invention provides a digital twin-based intelligent operating room construction device, which comprises:
the acquisition module is used for acquiring the patient information of a target patient and performing type matching on the patient information to obtain a corresponding information type;
the construction module is used for constructing a finite element model based on the information type to obtain a corresponding initial digital twin body;
the scanning module is used for carrying out multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body;
the dividing module is used for carrying out mesh division on the initial digital twin based on the multi-angle image information and determining a plurality of corresponding twin meshes;
the correcting module is used for correcting the initial digital twins based on the twin meshes to obtain target digital twins;
and the setting module is used for setting an operation instruction for the target digital twin and transmitting the target digital twin set by the operation instruction to the three-dimensional operation terminal.
The invention provides a digital twin-based intelligent operating room construction device, which comprises: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to cause the digital twin based smart operating room construction apparatus to perform the digital twin based smart operating room construction method described above.
A fourth aspect of the present invention provides a computer-readable storage medium having stored therein instructions, which when run on a computer, cause the computer to execute the above-described method of constructing a digital twin-based smart operating room.
According to the technical scheme provided by the invention, the patient information of a target patient is collected, and the type matching is carried out on the patient information to obtain the corresponding information type; constructing a finite element model based on the information type to obtain a corresponding initial digital twin body; performing multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body; carrying out mesh division on the initial digital twin body based on multi-angle image information, and determining a plurality of corresponding twin body meshes; modifying the initial digital twins based on the twin grids to obtain target digital twins; the method and the device have the advantages that the operation instruction is set for the target digital twin body, the target digital twin body set through the operation instruction is transmitted to the three-dimensional operation terminal, the target digital twin body is constructed through the virtual simulated finite element model, the operation instruction is set for the target digital twin body, and the target digital twin body set through the operation instruction is transmitted to the three-dimensional operation terminal, so that a doctor is helped to find an operation position more accurately, the processing efficiency of information of a patient is improved, and the accuracy of selecting the operation position when the doctor operates in the operation process is further improved.
Drawings
FIG. 1 is a schematic diagram of an embodiment of a method for constructing a digital twin-based intelligent operating room according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating meshing of an initial digital twin according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating an exemplary modification of an initial digital twin;
FIG. 4 is a schematic diagram of an embodiment of a digital twin-based intelligent operating room building apparatus according to the present invention;
FIG. 5 is a schematic diagram of an embodiment of an apparatus for constructing a digital twin-based intelligent operating room according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a method, a device, equipment and a storage medium for constructing an intelligent operating room based on digital twins, which are used for improving the accuracy of patient data processing. The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be implemented in other sequences than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
For ease of understanding, a specific flow of an embodiment of the present invention is described below, and referring to fig. 1, an embodiment of a method for constructing a digital twin-based intelligent operating room according to an embodiment of the present invention includes:
s101, collecting patient information of a target patient, and performing type matching on the patient information to obtain a corresponding information type;
it is understood that the execution subject of the present invention may be a construction device of a digital twin-based intelligent operating room, and may also be a terminal or a server, which is not limited herein. The embodiment of the present invention is described by taking a server as an execution subject.
Specifically, the server extracts entity words in patient information of a target patient and performs boundary checking on the extracted entity words, in this embodiment, a preset text information classifier is used for performing syntactic analysis on the patient information, entity words are marked out through an entity marking model in the preset text information classifier, and the marked entity words are extracted. After the patient information of the target patient is received, semantic analysis can be performed on the patient information of the target patient, and then entity words corresponding to the patient information of the target patient are obtained. Specifically, the type matching of the patient information includes entity word normalization processing and near word feature extraction, wherein in the entity word normalization processing, entity word normalization processing can be performed on each extracted entity word according to a normalization word library, for example, the number words, quantifier words and units are normalized in a standard manner, so that the work of later judgment and report generation is facilitated. In the extraction of the near-meaning word features, the extracted entity words can be subjected to near-meaning word feature extraction according to a near-meaning word library so as to ensure the consistency of information in the information acquisition process of patients and avoid the phenomenon of the repetition of the same patient information. After the entity word standardization processing is completed, clustering analysis is carried out on a plurality of entity words to obtain an entity class corresponding to a target patient, and then information type matching is carried out on the entity class to obtain an information type corresponding to patient information.
S102, constructing a finite element model based on the information type to obtain a corresponding initial digital twin body;
specifically, unit parameters and grid type variables are set based on information types, then combination optimization analysis is carried out by combining an intelligent optimization algorithm to obtain corresponding information unit sets, and then parameter analysis is carried out on the information unit sets to further construct an initial digital twin body. In the embodiment, different unit types are selected according to different information types, and the respective applicable simulation ranges of the unit types are different; aiming at the selection of the type and the size of the grid, the entity unit selects tetrahedrons or hexahedrons with different node numbers, whether the size of the grid is reasonable or not is judged by adjusting unit parameter information after the fineness of the grid is adjusted twice, if the change value of the unit parameter information accords with a preset threshold value, model construction is carried out according to the unit parameter information, and an initial digital twin body is obtained. The finite element model modification and analysis result data transfer is an optimization process of unit parameter information, a more accurate and reliable initial digital twin body is generated and load simulation analysis is carried out, and error comparison and analysis are further carried out on the unit parameter information.
S103, multi-angle scanning is carried out on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body;
determining coordinate values of a plurality of position points on the initial digital twin on the digital twin, wherein the initial digital twin is a digital twin model of a medical device entity in a smart operating room, and the medical device entity is any device entity in the smart operating room; based on a coordinate transformation matrix between the real space and the digital twin body, coordinate values of a plurality of position points on the initial digital twin body in the digital twin body are respectively subjected to coordinate transformation, and standard coordinate values of the plurality of position points on the initial digital twin body in the real space are obtained; determining the position points corresponding to the standard coordinate values of the plurality of position points on the initial digital twin body in the real space as a plurality of positioning standard points of the plurality of position points on the initial digital twin body in the real space, respectively installing a plurality of high-precision positioning devices at the plurality of standard positioning points, and acquiring image information corresponding to each positioning standard point according to the plurality of positioning standard points to obtain multi-angle image information.
S104, meshing the initial digital twin body based on the multi-angle image information, and determining a plurality of corresponding twin body meshes;
specifically, the initial digital twin body may be divided into a plurality of mesh regions by mesh division based on multi-angle image information, the plurality of mesh regions are a plurality of twin body meshes, a digital twin device is provided in the initial digital twin body, a twin body mesh where the digital twin device is located in the initial digital twin body is a corresponding twin body mesh in the initial digital twin body, and each twin body mesh corresponds to a central control center, which is mainly used for taking corresponding processing measures when the smart operating room device corresponding to the twin body mesh is in an abnormal working state. Specifically, when it is recognized that the smart operating room device is in an abnormal working state, the corresponding twin mesh in the initial digital twin can be searched and determined according to the device identification number corresponding to the smart operating room device, and the corresponding twin mesh in the initial digital twin can also be determined through an image processing technology.
S105, modifying the initial digital twin based on the twin grids to obtain a target digital twin;
specifically, after the virtual point cloud data are mapped to the twin texture image according to the preset feature database, it is equivalent to simultaneously map the virtual point cloud data of each twin mesh to the area occupied by the initial digital twin in the twin texture image. Therefore, the target point cloud data is point cloud data in an area covered by the initial digital twin corresponding to each twin mesh on the twin texture image after the initial point cloud is projected on the twin texture image. Determining each twin body grid in the virtual point cloud data according to a preset mapping relation and the initial digital twin body; and acquiring a target point cloud covered by the initial digital twin, and mapping the virtual point cloud data into the twin texture image according to the mapping relation so as to map each twin grid to the occupied area of the initial digital twin in the twin texture image, thereby acquiring the target digital twin.
And S106, carrying out operation instruction setting on the target digital twin body, and transmitting the target digital twin body set through the operation instruction to the three-dimensional operation terminal.
Specifically, according to an operation instruction mapped on the target digital twin body, a corresponding position of the operation instruction on the target digital twin body is determined, and an operation position is obtained; according to the corresponding position of the operation position on the target digital twin body, the real operation corresponding to the three-dimensional operation terminal of the operation position on the target digital twin body is determined, the preset proportion is provided between the equipment and the components on the target digital twin body and the equipment or the components in the real operation space, and the doctor can find the operation position more accurately through the virtual simulation physical model.
In the embodiment of the invention, the patient information of a target patient is collected, and the type matching is carried out on the patient information to obtain the corresponding information type; constructing a finite element model based on the information type to obtain a corresponding initial digital twin body; performing multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body; carrying out mesh division on the initial digital twin body based on multi-angle image information, and determining a plurality of corresponding twin body meshes; modifying the initial digital twins based on the twin grids to obtain target digital twins; the method and the device have the advantages that the operation instruction is set for the target digital twin body, the target digital twin body set through the operation instruction is transmitted to the three-dimensional operation terminal, the target digital twin body is constructed through the virtual simulated finite element model, the operation instruction is set for the target digital twin body, and the target digital twin body set through the operation instruction is transmitted to the three-dimensional operation terminal, so that a doctor is helped to find an operation position more accurately, the processing efficiency of information of a patient is improved, and the accuracy of selecting the operation position when the doctor operates in the operation process is further improved.
In a specific embodiment, the process of executing step S102 may specifically include the following steps:
(1) Performing unit matching on the information types through a preset finite element model database to obtain a corresponding information unit set;
(2) Performing parameter analysis on the information unit set to obtain unit parameter information corresponding to the information unit set;
(3) And carrying out model construction through unit parameter information to obtain the corresponding initial digital twin body.
Specifically, unit matching is carried out on information types through a preset finite element model database to obtain a corresponding information unit set, parameter analysis is carried out on the information unit set to obtain unit parameter information corresponding to the information unit set, model construction is carried out through the unit parameter information to obtain a corresponding initial digital twin body, specifically, unit parameters and grid type variables are set based on the information types, then combination optimization analysis is carried out through an intelligent optimization algorithm to obtain a corresponding information unit set, and parameter analysis is carried out on the information unit set to further construct the initial digital twin body.
In a specific embodiment, the process of executing step S103 may specifically include the following steps:
(1) Determining the space position of the initial digital twin object based on a preset space coordinate system to obtain a corresponding space position coordinate;
(2) Determining a corresponding scanning angle information set based on the spatial position coordinates;
(3) And performing multi-angle scanning on the initial digital twin body based on the scanning angle information set to obtain multi-angle image information corresponding to the digital twin body.
Specifically, images of a medical equipment entity are shot at a preset position and in a preset direction of a space coordinate system, wherein the medical equipment entity is any equipment entity in an intelligent operating room; determining the spatial position of the medical equipment entity based on the image of the medical equipment entity, and constructing a spatial position coordinate based on the spatial position of the medical equipment entity; in the space position coordinates, taking a preset position and a preset direction as a visual field origin, adjusting the position and the angle of an initial digital twin body for multiple times, and intercepting an image of the initial digital twin body every time of adjustment to obtain a plurality of target digital twin body matching templates, wherein the initial digital twin body is a target digital twin body of a medical equipment entity; respectively determining the matching degrees of the twin texture image matching template and the multiple target digital twin matching templates based on a preset image template matching algorithm to obtain multiple matching degrees; and determining the position and the angle of the initial digital twin corresponding to the maximum matching degree in the matching degrees as the position and the angle of the initial digital twin in the digital twin to obtain a scanning angle information set. The position of an intersection point of a medical equipment entity and the ground of the intelligent operating room in the multi-angle image information obtained through shooting is converted to a digital twin body to serve as a grid origin, grid lines are drawn on the ground of the digital twin body along the x axis and the y axis of the digital twin body with the grid origin as the center, the distance unit length between adjacent grid lines and the number of the grid lines are set, and the position and the angle of an initial digital twin body are adjusted for multiple times to obtain the multi-angle image information corresponding to the digital twin body.
In a specific embodiment, as shown in fig. 2, the process of executing step S104 may specifically include the following steps:
s201, performing key position point analysis on the initial digital twin body through a space coordinate system, and determining a plurality of key position points corresponding to the initial digital twin body;
s202, performing area expansion according to each key position point to obtain a plurality of corresponding expanded three-dimensional areas;
s203, carrying out image matching on the plurality of expanded three-dimensional areas and the multi-angle image information to obtain image information to be processed corresponding to each expanded three-dimensional area;
s204, respectively performing sweeping division processing on the image information to be processed corresponding to each expanded three-dimensional area to obtain a plurality of corresponding twin body grids.
Specifically, the key position points of the initial digital twin body are analyzed through a space coordinate system, a plurality of key position points corresponding to the initial digital twin body are determined, specifically, the initial digital twin body is rapidly identified through a mean shift segmentation algorithm, and a plurality of key position points are effectively located. And then, carrying out region expansion according to each key position point to obtain a plurality of corresponding expanded three-dimensional regions, wherein a smart operating room corresponding to the initial digital twin is divided into a plurality of regions according to a set size, and each region is numbered, so that basic attributes of all the regions in the smart operating room can be counted, and then a plurality of expanded three-dimensional regions are generated based on the plurality of regions and according to a grid number upper limit value determined by the number of the regions. Further, carrying out image matching on a plurality of expanded three-dimensional areas and multi-angle image information to obtain image information to be processed corresponding to each expanded three-dimensional area; the image information to be processed corresponding to each expanded three-dimensional area is respectively swept and divided to obtain a plurality of corresponding twin body grids, the twin body grids which are convenient for each user to manage relevant intelligent operating room equipment and timely process abnormal states can be reasonably divided through the scheme of the embodiment, and finally the plurality of corresponding twin body grids are obtained.
In a specific embodiment, as shown in fig. 3, the process of executing step S105 may specifically include the following steps:
s301, performing virtual point cloud data mapping on the twin body grids to obtain corresponding virtual point cloud data sets;
s302, performing virtual semantic assignment on the virtual point cloud data set to obtain corresponding virtual semantic point cloud data;
s303, carrying out depth truth value correction on the virtual semantic point cloud data to obtain a target point cloud data set;
and S304, correcting the initial digital twin body through the target point cloud data set to obtain a target digital twin body.
Specifically, the target point cloud data is a set of point cloud data in a region on the target digital twin that matches the position of the initial digital twin. That is, the target point cloud data refers to a set of points covering the initial digital twin. The points in the target point cloud data are all from the virtual point cloud data. Specifically, virtual point cloud data mapping is carried out on a plurality of twin body grids to obtain a corresponding virtual point cloud data set; and performing virtual semantic assignment on the virtual point cloud data set to obtain corresponding virtual semantic point cloud data, and acquiring the virtual point cloud data set of the virtual point cloud data on the target digital twin. The virtual point cloud data set comprises coordinates of all points in the virtual point cloud data on the target digital twin; the initial point cloud can represent the three-dimensional coordinates of any point in the virtual point cloud data, and the target digital twin body also has a corresponding image coordinate system, so that after the virtual point cloud data is mapped onto the target digital twin body, the coordinates of each point in the virtual point cloud data mapped onto the target digital twin body can be obtained, and the set of the coordinates of the points is regarded as the virtual point cloud data set. And performing virtual semantic assignment on the virtual point cloud data set to obtain corresponding virtual semantic point cloud data, performing depth truth value correction on the virtual semantic point cloud data to obtain a target point cloud data set, mapping the virtual point cloud data into the twin body texture image to substantially map the spatial position of each point in the virtual point cloud data to the virtual semantic point cloud data in the target digital twin body, wherein the virtual semantic point cloud data is used for labeling an initial digital twin body. Accordingly, since the initial digital twins have been identified from the target digital twins in an object identification related manner and assigned with the virtual semantic point cloud data, respectively, determining the target point cloud data in the target digital twins that matches the initial digital twins includes: acquiring point cloud data of each twin grid from the virtual point cloud data; according to the virtual point cloud data set, corresponding the point cloud data of each twin grid, namely the spatial position of each twin grid, to the virtual semantic point cloud data to obtain target point cloud data; the virtual semantic point cloud data is used for labeling an initial digital twin in the target digital twin. And correcting the initial digital twins through the target point cloud data set to obtain target digital twins.
In a specific embodiment, the process of executing step S301 may specifically include the following steps:
(1) Performing texture image conversion on the twin body meshes to obtain a plurality of corresponding twin body texture images;
(2) Performing two-dimensional feature matching on the twin body texture images based on a preset feature database to obtain a target two-dimensional feature set corresponding to the twin body texture images;
(3) Matching a mapping relation based on the target two-dimensional feature set to obtain a corresponding point cloud mapping relation;
(4) And carrying out virtual point cloud data mapping on the twin body grids through a point cloud mapping relation to obtain a corresponding virtual point cloud data set.
Specifically, firstly, texture image conversion is carried out on a plurality of twin body meshes to obtain a plurality of corresponding twin body texture images; performing two-dimensional feature matching on the twin body texture images based on a preset feature database to obtain a target two-dimensional feature set corresponding to the twin body texture images; then, matching a mapping relation based on the target two-dimensional feature set to obtain a corresponding point cloud mapping relation; and finally, carrying out virtual point cloud data mapping on the twin body grids through a point cloud mapping relation to obtain a corresponding virtual point cloud data set. Carrying out mapping relation matching on a target two-dimensional feature set to obtain candidate point cloud data of the target two-dimensional feature set, wherein the candidate point cloud data comprises a point cloud image and a dense depth point cloud data set corresponding to the point cloud image; determining a reference point cloud mapping relation of the image to be positioned based on the point cloud mapping relation of the candidate point cloud data, and determining the same characteristic points between the point cloud image and the virtual point cloud data; and in the dense depth point cloud data set corresponding to the point cloud image, determining a target point cloud mapping relation of the image to be positioned according to the three-dimensional coordinate points corresponding to the feature points, and performing virtual point cloud data mapping on a plurality of twin body grids through the point cloud mapping relation to obtain a corresponding virtual point cloud data set.
In a specific embodiment, the process of executing step S302 may specifically include the following steps:
(1) Performing semantic matching on the virtual point cloud data set based on a preset point cloud semantic database to obtain semantic information corresponding to the virtual point cloud data set;
(2) And performing virtual semantic assignment on the virtual point cloud data set based on the semantic information to obtain corresponding virtual semantic point cloud data.
Specifically, firstly, according to a point cloud semantic database of the three-dimensional point cloud, discrete points containing semantic information are marked in a virtual point cloud data set, and then the discrete points containing the semantic information are connected into polygons and classified to obtain the semantic information corresponding to the virtual point cloud data set. And after semantic information corresponding to the virtual point cloud data set is obtained, generating virtual semantic point cloud data according to the semantic information. The virtual semantic point cloud data carries out virtual semantic assignment on the virtual point cloud data set through semantic information corresponding to the virtual point cloud data set, the semantic information corresponding to the virtual point cloud data set refers to a connection relation between medical equipment and an intelligent operating room in the embodiment, KD trees are established for all the virtual point cloud data sets, the KD trees are respectively searched for the head point and the tail point of the ith medical equipment entity, the a-th medical equipment entity and the b-th medical equipment entity to which the nearest points belong are found out and used as the forward and backward association of the ith medical equipment entity, and all the medical equipment entities are traversed to obtain the complete connection relation.
In the above description of the method for constructing the intelligent operating room based on the digital twin according to the embodiment of the present invention, referring to fig. 4, a device for constructing the intelligent operating room based on the digital twin according to the embodiment of the present invention is described below, and an embodiment of the device for constructing the intelligent operating room based on the digital twin according to the embodiment of the present invention includes:
an obtaining module 401, configured to collect patient information of a target patient, and perform type matching on the patient information to obtain a corresponding information type;
a construction module 402, configured to perform finite element model construction based on the information type to obtain a corresponding initial digital twin;
a scanning module 403, configured to perform multi-angle scanning on the initial digital twin to obtain multi-angle image information corresponding to the digital twin;
a dividing module 404, configured to perform mesh division on the initial digital twin based on the multi-angle image information, and determine a plurality of corresponding twin meshes;
a modification module 405, configured to modify the initial digital twin based on the multiple twin meshes to obtain a target digital twin;
and the setting module 406 is configured to perform operation instruction setting on the target digital twin, and transmit the target digital twin set by the operation instruction to the three-dimensional operation terminal.
Acquiring patient information of a target patient through the cooperative cooperation of the components, and performing type matching on the patient information to obtain a corresponding information type; constructing a finite element model based on the information type to obtain a corresponding initial digital twin body; performing multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body; carrying out mesh division on the initial digital twin body based on multi-angle image information, and determining a plurality of corresponding twin body meshes; modifying the initial digital twins based on the twin grids to obtain target digital twins; the method and the device have the advantages that the operation instruction is set for the target digital twin body, the target digital twin body set through the operation instruction is transmitted to the three-dimensional operation terminal, the target digital twin body is constructed through the virtual simulated finite element model, the operation instruction is set for the target digital twin body, and the target digital twin body set through the operation instruction is transmitted to the three-dimensional operation terminal, so that a doctor is helped to find an operation position more accurately, the processing efficiency of information of a patient is improved, and the accuracy of selecting the operation position when the doctor operates in the operation process is further improved.
Fig. 5 is a schematic structural diagram of a digital twin-based intelligent operating room construction apparatus 500 according to an embodiment of the present invention, which may generate relatively large differences due to different configurations or performances, and may include one or more processors (CPUs) 510 (e.g., one or more processors) and a memory 520, and one or more storage media 530 (e.g., one or more mass storage devices) storing applications 533 or data 532. Memory 520 and storage media 530 may be, among other things, transient or persistent storage. The program stored on the storage medium 530 may include one or more modules (not shown), each of which may include a series of instructions operating on the digital twin based smart operating room construction apparatus 500. Still further, the processor 510 may be configured to communicate with the storage medium 530 to execute a series of instruction operations in the storage medium 530 on the digital twin-based smart operating room construction device 500.
The digital twin-based smart operating room construction apparatus 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input-output interfaces 560, and/or one or more operating systems 531, such as Windows server, mac OS X, unix, linux, freeBSD, and the like. It will be understood by those skilled in the art that the construction apparatus configuration of the digital twin based smart operating room shown in fig. 5 does not constitute a limitation of the construction apparatus of the digital twin based smart operating room, and may include more or less components than those shown, or some components in combination, or a different arrangement of components.
The invention also provides a construction device of the intelligent operating room based on the digital twin, which comprises a memory and a processor, wherein the memory stores computer readable instructions, and when the computer readable instructions are executed by the processor, the processor executes the steps of the construction method of the intelligent operating room based on the digital twin in the above embodiments.
The present invention also provides a computer readable storage medium, which may be a non-volatile computer readable storage medium, which may also be a volatile computer readable storage medium, having stored therein instructions, which, when run on a computer, cause the computer to perform the steps of the digital twin-based intelligent operating room construction method.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention, which is substantially or partly contributed by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media that can store program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A construction method of a digital twin-based intelligent operating room is characterized by comprising the following steps:
acquiring patient information of a target patient, and performing type matching on the patient information to obtain a corresponding information type;
constructing a finite element model based on the information type to obtain a corresponding initial digital twin body;
performing multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body;
meshing the initial digital twin body based on the multi-angle image information to determine a plurality of corresponding twin body meshes;
modifying the initial digital twins based on the twin meshes to obtain target digital twins;
and setting an operation instruction for the target digital twin, and transmitting the target digital twin set by the operation instruction to a three-dimensional operation terminal.
2. The method for constructing a digital twin-based smart operating room according to claim 1, wherein the finite element model construction based on the information type to obtain a corresponding initial digital twin comprises:
performing unit matching on the information types through a preset finite element model database to obtain a corresponding information unit set;
performing parameter analysis on the information unit set to obtain unit parameter information corresponding to the information unit set;
and carrying out model construction through the unit parameter information to obtain a corresponding initial digital twin body.
3. The method for constructing a digital twin-based smart operating room as claimed in claim 1, wherein the multi-angle scanning of the initial digital twin to obtain multi-angle image information corresponding to the digital twin comprises:
determining the space position of the initial digital twin object based on a preset space coordinate system to obtain a corresponding space position coordinate;
determining a corresponding scanning angle information set based on the spatial position coordinates;
and performing multi-angle scanning on the initial digital twin body based on the scanning angle information set to obtain multi-angle image information corresponding to the digital twin body.
4. The method for constructing a digital twin-based smart operating room according to claim 3, wherein the mesh-dividing the initial digital twin based on the multi-angle image information to determine a plurality of corresponding twin meshes comprises:
performing key position point analysis on the initial digital twin through the space coordinate system, and determining a plurality of key position points corresponding to the initial digital twin;
performing area expansion according to each key position point to obtain a plurality of corresponding expanded three-dimensional areas;
performing image matching on the plurality of expanded three-dimensional areas and the multi-angle image information to obtain image information to be processed corresponding to each expanded three-dimensional area;
and respectively performing sweeping division processing on the image information to be processed corresponding to each expanded three-dimensional area to obtain a plurality of corresponding twin body grids.
5. The method of claim 1, wherein the modifying the initial digital twin based on the plurality of twin meshes to obtain a target digital twin comprises:
carrying out virtual point cloud data mapping on the twin body grids to obtain corresponding virtual point cloud data sets;
performing virtual semantic assignment on the virtual point cloud data set to obtain corresponding virtual semantic point cloud data;
carrying out depth truth value correction on the virtual semantic point cloud data to obtain a target point cloud data set;
and correcting the initial digital twin body through the target point cloud data set to obtain a target digital twin body.
6. The method for constructing a digital twin-based smart operating room according to claim 5, wherein the virtual point cloud data mapping of the twin meshes to obtain corresponding virtual point cloud data sets comprises:
performing texture image conversion on the twin body grids to obtain a plurality of corresponding twin body texture images;
performing two-dimensional feature matching on the twin body texture images based on a preset feature database to obtain a target two-dimensional feature set corresponding to the twin body texture images;
carrying out mapping relation matching based on the target two-dimensional feature set to obtain a corresponding point cloud mapping relation;
and performing virtual point cloud data mapping on the twin body grids through the point cloud mapping relation to obtain a corresponding virtual point cloud data set.
7. The method for constructing a digital twin-based intelligent operating room according to claim 5, wherein the virtual semantic assignment of the virtual point cloud data set to obtain corresponding virtual semantic point cloud data comprises:
performing semantic matching on the virtual point cloud data set based on a preset point cloud semantic database to obtain semantic information corresponding to the virtual point cloud data set;
and performing virtual semantic assignment on the virtual point cloud data set based on the semantic information to obtain corresponding virtual semantic point cloud data.
8. A digital twin-based intelligent operating room building device, which is characterized by comprising:
the acquisition module is used for acquiring the patient information of a target patient and performing type matching on the patient information to obtain a corresponding information type;
the construction module is used for constructing a finite element model based on the information type to obtain a corresponding initial digital twin body;
the scanning module is used for carrying out multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body;
the dividing module is used for carrying out mesh division on the initial digital twin body based on the multi-angle image information and determining a plurality of corresponding twin body meshes;
the correcting module is used for correcting the initial digital twin based on the twin grids to obtain a target digital twin;
and the setting module is used for setting an operation instruction for the target digital twin and transmitting the target digital twin set by the operation instruction to the three-dimensional operation terminal.
9. A digital twin-based smart operating room building apparatus, comprising: a memory and at least one processor, the memory having instructions stored therein;
the at least one processor invokes the instructions in the memory to cause the digital twin based smart operating room construction device to perform the digital twin based smart operating room construction method of any of claims 1-7.
10. A computer readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, implement the method of constructing a digital twins-based smart operating room as claimed in any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310122298.2A CN115862821B (en) | 2023-02-16 | 2023-02-16 | Digital twinning-based intelligent operating room construction method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310122298.2A CN115862821B (en) | 2023-02-16 | 2023-02-16 | Digital twinning-based intelligent operating room construction method and related device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115862821A true CN115862821A (en) | 2023-03-28 |
| CN115862821B CN115862821B (en) | 2023-05-05 |
Family
ID=85658160
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310122298.2A Active CN115862821B (en) | 2023-02-16 | 2023-02-16 | Digital twinning-based intelligent operating room construction method and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115862821B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116228991A (en) * | 2023-05-08 | 2023-06-06 | 河北光之翼信息技术股份有限公司 | Coordinate conversion method and device, electronic equipment and storage medium |
| CN116912430A (en) * | 2023-09-14 | 2023-10-20 | 北京唯迈医疗设备有限公司 | Device for constructing three-dimensional digital twin system of remote intervention operating room |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190087544A1 (en) * | 2017-09-21 | 2019-03-21 | General Electric Company | Surgery Digital Twin |
| CN110709938A (en) * | 2017-06-28 | 2020-01-17 | 通用电气公司 | Method and system for generating a digital twin of patients |
| CN110853746A (en) * | 2019-10-14 | 2020-02-28 | 边缘智能研究院南京有限公司 | Intelligent hospital system based on digital twin and construction method thereof |
| CN112002401A (en) * | 2020-08-25 | 2020-11-27 | 上海至数企业发展有限公司 | Space environment monitoring method and system based on digital twin and storage medium |
| CN112380704A (en) * | 2020-11-16 | 2021-02-19 | 北京航空航天大学 | Digital twin model correction method and system based on machine vision |
| US20210202098A1 (en) * | 2018-08-08 | 2021-07-01 | Hc1.Com Inc. | Creation and management of digital twins of healthcare patients |
| CN113379860A (en) * | 2021-04-25 | 2021-09-10 | 上海索骥信息科技有限公司 | Medical image data generation method and system with marker based on digital object |
| CN114224528A (en) * | 2021-10-26 | 2022-03-25 | 上海交通大学医学院附属第九人民医院 | Oral digital twin model system based on virtual reality interaction and establishment method |
| CN114386150A (en) * | 2022-01-13 | 2022-04-22 | 上海理工大学 | Digital and intelligent building applying 6G digital twins |
| US20220165410A1 (en) * | 2020-11-20 | 2022-05-26 | Siemens Healthcare Gmbh | Remote communication with a medical technology facility with the aid of a digital twin |
| CN115054367A (en) * | 2022-06-20 | 2022-09-16 | 上海市胸科医院 | Focus positioning method and device based on mixed reality and electronic equipment |
-
2023
- 2023-02-16 CN CN202310122298.2A patent/CN115862821B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110709938A (en) * | 2017-06-28 | 2020-01-17 | 通用电气公司 | Method and system for generating a digital twin of patients |
| US20190087544A1 (en) * | 2017-09-21 | 2019-03-21 | General Electric Company | Surgery Digital Twin |
| US20210202098A1 (en) * | 2018-08-08 | 2021-07-01 | Hc1.Com Inc. | Creation and management of digital twins of healthcare patients |
| CN110853746A (en) * | 2019-10-14 | 2020-02-28 | 边缘智能研究院南京有限公司 | Intelligent hospital system based on digital twin and construction method thereof |
| CN112002401A (en) * | 2020-08-25 | 2020-11-27 | 上海至数企业发展有限公司 | Space environment monitoring method and system based on digital twin and storage medium |
| CN112380704A (en) * | 2020-11-16 | 2021-02-19 | 北京航空航天大学 | Digital twin model correction method and system based on machine vision |
| US20220165410A1 (en) * | 2020-11-20 | 2022-05-26 | Siemens Healthcare Gmbh | Remote communication with a medical technology facility with the aid of a digital twin |
| CN113379860A (en) * | 2021-04-25 | 2021-09-10 | 上海索骥信息科技有限公司 | Medical image data generation method and system with marker based on digital object |
| CN114224528A (en) * | 2021-10-26 | 2022-03-25 | 上海交通大学医学院附属第九人民医院 | Oral digital twin model system based on virtual reality interaction and establishment method |
| CN114386150A (en) * | 2022-01-13 | 2022-04-22 | 上海理工大学 | Digital and intelligent building applying 6G digital twins |
| CN115054367A (en) * | 2022-06-20 | 2022-09-16 | 上海市胸科医院 | Focus positioning method and device based on mixed reality and electronic equipment |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116228991A (en) * | 2023-05-08 | 2023-06-06 | 河北光之翼信息技术股份有限公司 | Coordinate conversion method and device, electronic equipment and storage medium |
| CN116228991B (en) * | 2023-05-08 | 2023-07-14 | 河北光之翼信息技术股份有限公司 | Coordinate conversion method and device, electronic equipment and storage medium |
| CN116912430A (en) * | 2023-09-14 | 2023-10-20 | 北京唯迈医疗设备有限公司 | Device for constructing three-dimensional digital twin system of remote intervention operating room |
| CN116912430B (en) * | 2023-09-14 | 2023-12-15 | 北京唯迈医疗设备有限公司 | Device for constructing three-dimensional digital twin system of remote intervention operating room |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115862821B (en) | 2023-05-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115862821A (en) | Construction method of intelligent operating room based on digital twins and related device | |
| CN108537876A (en) | Three-dimensional rebuilding method, device, equipment based on depth camera and storage medium | |
| JP5251080B2 (en) | Object recognition method | |
| CN102542035A (en) | Polygonal rasterisation parallel conversion method based on scanning line method | |
| CN108763827B (en) | Transmission tower finite element model establishing method and device | |
| CN107193923B (en) | Method and system for quickly superposing vectors in two-dimensional geographic space | |
| CN111707279A (en) | Matching evaluation method, medium, terminal and device of laser point cloud and map | |
| CN116596755A (en) | Method, device, equipment and storage medium for splicing point cloud data | |
| US7333104B2 (en) | Method and program of converting three-dimensional shape data into cell internal data | |
| CN109559374B (en) | Efficient mapping system based on point cloud data | |
| EP1507220A1 (en) | Boundary data inside/outside judgment method and program thereof | |
| CN115937466B (en) | GIS-fused three-dimensional model generation method, system and storage medium | |
| CN109360215B (en) | Method, device and equipment for searching outer contour of three-dimensional model and storage medium | |
| JP2018124798A (en) | Image search device and image search program | |
| JP2018010455A (en) | Three-dimensional cad data creation system and creation method | |
| CN115859524A (en) | Cylinder Boolean difference calculation method based on STL model | |
| CN116228158A (en) | Big data platform plan management method and related device based on constructional engineering | |
| CN114357253A (en) | Data visualization method and device, electronic equipment and storage medium | |
| CN117058358B (en) | Scene boundary detection method and mobile platform | |
| CN114485664B (en) | Boundary simplifying method, path planning method, device, equipment and system | |
| CN113902910B (en) | Vision measurement method and system | |
| CN115830249B (en) | Three-dimensional scanning method, three-dimensional measuring method, three-dimensional scanning system and electronic device | |
| CN110704900B (en) | Method for placing connection node between keel column model and wall keel model and product | |
| CN118840397A (en) | Component construction state identification method based on point cloud | |
| CN117671639A (en) | Semantic extraction method, device, equipment and robot for semantic grid map |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |