WO2024199664A1 - Système et procédé de superposition d'images de données d'imagerie ultrasonore - Google Patents

Système et procédé de superposition d'images de données d'imagerie ultrasonore Download PDF

Info

Publication number
WO2024199664A1
WO2024199664A1 PCT/EP2023/058363 EP2023058363W WO2024199664A1 WO 2024199664 A1 WO2024199664 A1 WO 2024199664A1 EP 2023058363 W EP2023058363 W EP 2023058363W WO 2024199664 A1 WO2024199664 A1 WO 2024199664A1
Authority
WO
WIPO (PCT)
Prior art keywords
registration
image data
ultrasound
image
computer
Prior art date
Application number
PCT/EP2023/058363
Other languages
English (en)
Inventor
Daniel MENA
Wei Wei
Original Assignee
Brainlab Ag
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Brainlab Ag filed Critical Brainlab Ag
Priority to PCT/EP2023/058363 priority Critical patent/WO2024199664A1/fr
Publication of WO2024199664A1 publication Critical patent/WO2024199664A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/38—Registration of image sequences
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10132—Ultrasound image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10132—Ultrasound image
    • G06T2207/10136—3D ultrasound image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing

Definitions

  • the present invention relates to a, particularly computer-implemented, method and a system for image registration of ultrasound image data, a corresponding computer program product and computer-readable medium.
  • Medical procedures often may involve positioning, guiding or navigating instruments during a medical procedure supported by three-dimensional image data, such as 3D ultrasound (US) or CT or MRI image data, obtained by performing imaging prior to the procedure (i.e., pre-procedural image data).
  • the 3D image data may then be registered at some point, e.g., at the beginning of a procedure. To that end, pointers or markers may be used or surface image data, e.g. from a surface scanner, may be used. Thus, the 3D image data is registered with respect to the subject.
  • the registration may become to some degree invalid over time, for example due to movement, breathing, and/or deformation of soft tissue.
  • Soft tissue is particularly challenging since the tissues are not fixed to a rigid structure like a bone that could be used as a reference point.
  • any static image registration done beforehand will become invalid over time.
  • the initially determined image data position cannot, as such, be used reliably throughout the process of interacting with the tissue, particularly for steps of the medical procedure that require guiding and/or navigating instruments by means of image data.
  • Monitoring the tissue by means of CT or MRI over an extended period of time may be cumbersome, impede the interaction with tissue, or may not be feasible at all, e.g., due to associated high radiation dose.
  • Ultrasound imaging provides an imaging method that, in principle, can be more easily used throughout the interaction to acquire real-time data. It is generally not harmful and ultrasound probes are easy to handle and allow for increased flexibility. Moreover, generating ultrasound images is fast and the machine itself does not take up much space, so the imaging can be repeated multiple times throughout the procedure.
  • co-aligning initial three-dimensional ultrasound or CT or MRI image data and subsequently acquired two-dimensional or three-dimensional ultrasound image data may be an option.
  • Processing of the signal data to reconstruct an image volume and perform the registration requires a large amount of computing resources. This may also lead to a time-delay in the reconstruction of the image that may not be desirable, for example, when relying on the images for guiding or navigating instruments.
  • the present invention can be used, for example, by being employed by or incorporated in surgical navigation systems.
  • navigation systems are Curve® Navigation or Kick® Navigation, but can also be used for various other purposes than surgical navigation systems, e.g., in any systems that involve obtaining positional data from ultrasound images to improve their accuracy.
  • a system and a computer-implemented method for image registration of ultrasound image data comprises a) receiving first ultrasound image data of a subject; b) performing, using a first registration method, a first image registration of the first ultrasound image data with three-dimensional, e.g. ultrasound or CT or MRI, image data of a subject, the three-dimensional image data acquired prior to the first ultrasound image data; c) receiving second ultrasound image data of the subject, the second ultrasound image data acquired after the first ultrasound image data; and d) performing, using a second registration method, a second image registration of the second ultrasound image data with the three-dimensional image data, e.g. ultrasound or CT or MRI image data, so as to provide an updated image registration, wherein the first registration method has higher registration accuracy than the second registration method.
  • a first registration method e.g. ultrasound or CT or MRI
  • the present invention provides a method and a system for image registration of ultrasound image data, a computer program product, and computer-readable medium according to the independent claims. Preferred embodiments are laid down in the dependent claims. Unless specified otherwise, any method according to the present disclosure may be a computer-implemented method.
  • The, particularly computer-implemented, method for image registration of ultrasound image data comprises a) receiving first ultrasound image data of a subject; b) performing, using a first registration method, a first image registration of the first ultrasound image data with three-dimensional (3D) image data, e.g.
  • ultrasound or CT or MRI image data of a subject, the three-dimensional image data acquired prior to the first ultrasound image data; c) receiving second ultrasound image data of the subject, the second ultrasound image data acquired after the first ultrasound image data; and d) performing, using a second registration method, a second image registration of the second ultrasound image data with the three-dimensional (3D) image data, e.g. ultrasound or CT or MRI image data, so as to provide an updated image registration, wherein the first registration method has higher registration accuracy than the second registration method.
  • a second registration method e.g. ultrasound or CT or MRI image data
  • the first ultrasound image data may be 2D or 3D ultrasound image data.
  • the second ultrasound image data may be 2D or 3D ultrasound image data.
  • the first ultrasound image data and second ultrasound image data may be understood as being representative of different times or stages of the same procedure.
  • the subject may, for example, be a patient undergoing the procedure.
  • the method may comprise loading a 3D image data set having been acquired prior to the ultrasound image data being registered to the 3D image data set, particularly, prior to the medical procedure.
  • Segmentation data representative of a segmentation carried out on the 3D image data may be included in or accompany the 3D image data set. This may advantageously be used for object-based registration.
  • the 3D image data may be referred to as pre-procedural 3D image data.
  • the 3D image data may comprise planning data for a medical procedure.
  • the first, second, and subsequent ultrasound image data of the present disclosure may be referred to as intra-procedural image data.
  • the second and subsequent ultrasound image data may be obtained, in particular continuously, over an extended period of time during the procedure. This may allow for improved monitoring and providing improved guidance and navigation during the medical procedure.
  • the first image registration may, for example, be the initial image registration of the procedure.
  • the second image registration may be a subsequent image registration.
  • the first registration method having a higher registration accuracy than the second registration method may be understood as, when a change has occurred with respect to the preceding registration, the first registration method yielding higher registration accuracy after the change than the first registration method would yield. Accordingly, the first registration method may also be referred to as high-accuracy or high-quality registration method herein.
  • the first registration method can be an elastic registration which can compensate a breath motion.
  • the second registration method can be rigid registration yielding high speed.
  • the first registration method may be a registration method requiring more computational resources and/or more time than the second registration method. This may be referred to as the second registration method being faster than the first registration method, or, short, the second registration method being a fast registration method.
  • the first registration may be considered to be a high-accuracy registration.
  • the second registration may be considered a fast registration.
  • the first registration method ensures that initial image registration has high accuracy.
  • the second registration method allows for a reliable, yet relatively fast and resourcesaving registration and real-time applications like positioning and/or navigation.
  • the low accuracy of a registration method may be due to using an image registration that puts more weight on speed than on accuracy or by using a low amount of information, e.g., low pixel or voxel density or low number of structural elements, as an example.
  • the real-time imaging capabilities of ultrasound are leveraged, for example to show the current situation as it is, by fusing it with a (pre-procedural) 3D image dataset that has high image quality and, for example, where procedure planning was done, for example CT or MRI images.
  • the method may allow for a high-quality registration to be done and then maintained over the course of the procedure with relatively low effort and, accordingly, without significant delays.
  • the method may allow for compensating for movement and/or deformation, for example caused by breathing or tissue manipulation.
  • the claimed steps may be carried out repeatedly, for example in cyclical fashion, over the course of the procedure. Thus, particularly good results can be achieved even over an extended period of time.
  • the method may comprise, after the second image registration, for subsequent ultrasound image data, e) continuing performing image registration using the second registration method, and f) subsequently, performing image registration using the first registration method.
  • the second registration method may be used for continued registration of the second and subsequent ultrasound image data.
  • continuous registration may, particularly, be understood as entailing multiple successive registrations.
  • the method may revert/return to using the first registration method for subsequent ultrasound image data.
  • a lower-accuracy (fast) registration method may be used, and, after some time, the method may revert/return to performing the high-accuracy registration method. Reverting to the first registration method may, thus, be referred to as high-quality or high-accuracy reregistration.
  • Continuing image registration using the second (fast) registration method may allow for avoiding time delays in the image registration, thereby improving real-time characteristics.
  • the potentially delaying first registration method may be applied only as needed, for example. Reverting to the first (high-accuracy) registration method after some time allows for ensuring that overall quality can remain high. It will be explained in detail below what may cause or trigger reverting/returning to the first registration method.
  • the steps e) and f) may be repeated any number of times, e.g., throughout the procedure, which may allow upholding quality over an extended period of time, e.g., throughout the procedure.
  • the method may comprise alternating between performing image registration using the first registration method and image registration using the second registration method for subsequent ultrasound image data.
  • alternating may be understood broadly as describing a sequence.
  • the second registration method is performed for an extended period of time and/or multiple frames prior to reverting/returning to the first registration method.
  • the first registration method is used for a first period of time or a first number of frames and the second registration method is used for a second period of time or a second number of frames prior to reverting to the first registration method, the first and second periods being different or the numbers of frames being different.
  • the method according to the present disclosure may comprise image registration by means of a cyclical image registration method comprising the first registration method and the second registration method.
  • the method of the present disclosure may default to continuing image registration of subsequent ultrasound image data by means of the second registration method until a trigger condition is met, which may trigger switching to the first registration method.
  • a trigger condition may comprise a user-activated trigger being activated and/or a predetermined time having elapsed and/or a predetermined frame-number having been exceeded since starting image registration using the second registration method and/or a predetermined degree of deterioration of registration accuracy being detected. Details will be provided below.
  • switching from the image registration using the second registration method to perform the image registration using the first registration method, in particular step f), may be initiated by a user and/or may be triggered automatically.
  • a user may initiate the switching by means of a/the user-activated trigger, which may, for example, be any suitable hardware or software feature.
  • a/the user-activated trigger may, for example, be any suitable hardware or software feature.
  • a hardware button or a GUI element, e.g., button rendered on a display device.
  • the hardware or software feature may particularly be incorporated in the ultrasound transducer or ultrasound probe.
  • a user may, for example, trigger the switching when a visualization looks suspicious or when performing steps of the procedure that might cause or have caused registration issues.
  • Automatically triggering the switching may, for example, comprise automatically triggering the switching in response to a predetermined time having elapsed and/or a predetermined frame-number having been exceeded since starting image registration using the second registration method and/or in response to detecting a predetermined degree of deterioration of registration accuracy.
  • switching from the image registration using the second registration method to the image registration using the first registration method, in particular step f), may be triggered automatically after performing the second image registration for a predetermined time or a predetermined number of frames, and/or may be triggered automatically in response to determining that a deviation of the updated image registration from a reference image registration exceeds a predetermined threshold.
  • the reference image registration may be the image registration obtained in step b) or the image registration as obtained from the most recent image registration using the first registration method.
  • a high-accuracy registration e.g. the initial one or the most recent one, may be used as a reference image registration.
  • the threshold may be selected/predefined so as to be suitable for the specific circumstances at hand, e.g., the type of hardware and software being employed and/or the overall accuracy requirements, which may depend on the procedure, for example.
  • a number of frames or an overall time during which image registration using the first registration method is performed may be lower than a number of frames or an overall time during which image registration using the second registration method is performed.
  • the second registration method may be the default or predominant image registration method.
  • the first registration method may serve for initial image registration and for corrective image registration, i.e., may be seen as an auxiliary image registration method.
  • the first registration method may be an elastic registration method. This allows for a particularly high accuracy.
  • the second registration method may be a rigid registration method. This allows for fast registration with lower computing requirements, i.e., is particularly suitable for real-time registration without delays.
  • the first registration method may comprise a first rigid registration and the second registration method may comprise a second rigid registration, wherein the first rigid registration has a larger search range than the second rigid registration.
  • the second rigid registration may be a registration in one or more local regions.
  • the second registration method may allow for fast registration.
  • the first registration method may comprise a first elastic registration and the second registration method may comprise a second elastic registration, wherein the first elastic registration is higher accuracy than the second elastic registration.
  • the second elastic registration may comprise that an elastic deformation is estimated with last N frames of the first elastic registration, e.g. using a Kalman filter.
  • At least N registrations by means of the first registration method are carried out.
  • Each may yield deformed 3D CT or MRI image data aligned with ultrasound image data used as input for the respective registration, for example.
  • Synthetic deformed data may be predicted from the last N accurate registrations, the prediction, for example, being done using a Kalman filter.
  • the first registration method may be a 3D ultrasound registration method and/or the second registration method may be a 2D ultrasound registration method.
  • a 3D ultrasound registration method may be a registration method using 3D ultrasound image data as an input.
  • a 2D ultrasound registration method may be a registration method using 2D ultrasound image data as an input.
  • the 2D ultrasound registration method may comprise registering 2D ultrasound image data obtained, particularly continuously, by a 2D ultrasound transducer and the 3D ultrasound registration method may comprise registering 3D ultrasound image data obtained by a 2D or 3D ultrasound transducer. Any of the known suitable transducers may be used to that end.
  • a 2D transducer may be used to generate 3D ultrasound image data via a sweep for carrying out the 3D ultrasound registration, e.g., for the first ultrasound registration method, and (e.g. live) 2D slices from that same 2D transducer may be used for the 2D ultrasound registration, e.g. for the second ultrasound registration method.
  • the 2D ultrasound registration method may comprise registering only selected slices, e.g., central slices, extracted from 3D ultrasound image data obtained by a 3D ultrasound transducer.
  • the 3D ultrasound registration method may comprise registering 3D ultrasound image data obtained by the 3D ultrasound transducer.
  • the first registration method may, for example, employ deep learning, for example, so as to carry out 2D ultrasound slice to 3D, e.g. ultrasound or CT or MRI, image data registration.
  • deep learning may allow for handling the potentially large amount of data.
  • the second image registration may be performed simultaneously with the ultrasound acquisition, particularly continuously. That is, the second registration method may yield real time or near-real time image registration with respect to the ultrasound acquisition.
  • an updated registration may be carried out to provide real-time registration.
  • updated registration may be provided at a frame rate of at least 5 fps, in particular at least 10 fps.
  • Providing registration at a specific frame rate may refer to updating the registration at said frame rate.
  • the method may comprise an inter-frame elastic volume prediction.
  • Such a prediction allows for reducing a registration error caused by non-rigid motion.
  • Known methods may be used to that end.
  • the method of the present disclosure may comprise obtaining 3D, for example ultrasound or CT or MRI, image data of the subject and/or obtaining, subsequently to obtaining the 3D image data, the first ultrasound image data and/or the second ultrasound image data and/or the subsequent ultrasound image data.
  • the method may comprise image acquisition of 3D CT or MRI image data, e.g., by means of a CT or MRI scanner, or 3D ultrasound image data acquisition by means of an ultrasound transducer, particularly acquired pre-procedurally.
  • the method of the present disclosure may comprise first, second and/or subsequent ultrasound image data acquisition, e.g., by means of a 2D or 3D ultrasound transducer.
  • the present disclosure also provides a system for image registration of ultrasound image data, the system configured to perform any of the methods described in the present disclosure.
  • the system may comprise a processing unit (which may also be referred to as a processor) that is configured to, in particular automatically, perform and/or control one or more of, in particular all, of the method steps described herein, unless specified otherwise, e.g., unless it is specified that the steps are performed by the user.
  • the system may further comprise a CT or MRI scanner configured to acquire the CT or MRI image data and/or an ultrasound transducer configured to acquire ultrasound image data of the 3D image data and/or the first ultrasound image data and/or the second ultrasound image data and/or the subsequent ultrasound image data.
  • the ultrasound transducer may be a 2D or a 3D ultrasound transducer, for example.
  • the processing unit may be part of a computer.
  • the system in particular the computer, may also comprise at least one memory unit or configured to access a memory unit.
  • the memory unit may be comprised in the computer-readable medium according to the invention.
  • the present disclosure also provides a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out and/or control any of the methods described herein.
  • the present disclosure also provides a computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out and/or control any of the methods described herein.
  • the present disclosure also provides use of a system as described above for guidance and/or navigation of instruments, e.g., by visual and/or audio and/or haptic output providing guidance for a practitioner and/or for generating control data for automatically guiding and/or navigating the instruments.
  • the invention does not involve or in particular comprise or encompass an invasive step which would represent a substantial physical interference with the body requiring professional medical expertise to be carried out and entailing a substantial health risk even when carried out with the required professional care and expertise.
  • the invention does not comprise a step of positioning a medical implant in order to fasten it to an anatomical structure or a step of fastening the medical implant to the anatomical structure or a step of preparing the anatomical structure for having the medical implant fastened to it. More particularly, the invention does not involve or in particular comprise or encompass any surgical or therapeutic activity. The invention is instead directed as applicable to guidance and/or navigation of an ultrasound probe and/or an instrument used for the procedure. For this reason alone, no surgical or therapeutic activity and in particular no surgical or therapeutic step is necessitated or implied by carrying out the invention.
  • the method in accordance with the invention is for example a computer implemented method.
  • all the steps or merely some of the steps (i.e. less than the total number of steps) of the method in accordance with the invention can be executed by a computer (for example, at least one computer).
  • An embodiment of the computer implemented method is a use of the computer for performing a data processing method.
  • An embodiment of the computer implemented method is a method concerning the operation of the computer such that the computer is operated to perform one, more or all steps of the method.
  • the computer for example comprises at least one processor and for example at least one memory in order to (technically) process the data, for example electronically and/or optically.
  • the processor being for example made of a substance or composition which is a semiconductor, for example at least partly n- and/or p-doped semiconductor, for example at least one of II-, III-, IV-, V-, Vl-sem iconductor material, for example (doped) silicon and/or gallium arsenide.
  • the calculating or determining steps described are for example performed by a computer. Determining steps or calculating steps are for example steps of determining data within the framework of the technical method, for example within the framework of a program.
  • a computer is for example any kind of data processing device, for example electronic data processing device.
  • a computer can be a device which is generally thought of as such, for example desktop PCs, notebooks, netbooks, etc., but can also be any programmable apparatus, such as for example a mobile phone or an embedded processor.
  • a computer can for example comprise a system (network) of "sub-computers", wherein each sub-computer represents a computer in its own right.
  • the term "computer” includes a cloud computer, for example a cloud server.
  • the term "cloud computer” includes a cloud computer system which for example comprises a system of at least one cloud computer and for example a plurality of operatively interconnected cloud computers such as a server farm.
  • Such a cloud computer is preferably connected to a wide area network such as the world wide web (WWW) and located in a so-called cloud of computers which are all connected to the world wide web.
  • WWW world wide web
  • Such an infrastructure is used for "cloud computing", which describes computation, software, data access and storage services which do not require the end user to know the physical location and/or configuration of the computer delivering a specific service.
  • the term "cloud” is used in this respect as a metaphor for the Internet (world wide web).
  • the cloud provides computing infrastructure as a service (laaS).
  • the cloud computer can function as a virtual host for an operating system and/or data processing application which is used to execute the method of the invention.
  • the cloud computer is for example an elastic compute cloud (EC2) as provided by Amazon Web ServicesTM.
  • a computer for example comprises interfaces in order to receive or output data and/or perform an analogue-to-digital conversion.
  • the data are for example data which represent physical properties and/or which are generated from technical signals.
  • the technical signals are for example generated by means of (technical) detection devices (such as for example devices for detecting marker devices) and/or (technical) analytical devices (such as for example devices for performing (medical) imaging methods), wherein the technical signals are for example electrical or optical signals.
  • the technical signals for example represent the data received or outputted by the computer.
  • the computer is preferably operatively coupled to a display device which allows information outputted by the computer to be displayed, for example to a user.
  • a display device is a virtual reality device or an augmented reality device (also referred to as virtual reality glasses or augmented reality glasses) which can be used as "goggles" for navigating.
  • augmented reality glasses is Google Glass (a trademark of Google, Inc.).
  • An augmented reality device or a virtual reality device can be used both to input information into the computer by user interaction and to display information outputted by the computer.
  • Another example of a display device would be a standard computer monitor comprising for example a liquid crystal display operatively coupled to the computer for receiving display control data from the computer for generating signals used to display image information content on the display device.
  • a specific embodiment of such a computer monitor is a digital lightbox.
  • An example of such a digital lightbox is Buzz®, a product of Brainlab AG.
  • the monitor may also be the monitor of a portable, for example handheld, device such as a smart phone or personal digital assistant or digital media player.
  • the invention also relates to a computer program product comprising instructions which, when the program is executed by (also referred to as running on) a computer, cause the computer to carry out or (also referred to as performing) one or more or all of the method steps described herein and/or to a computer-readable medium, also referred to as program storage medium, on which the program may be stored (in particular in a non-transitory form).
  • a computer comprising said program storage medium and/or to a (physical, for example electrical, for example technically generated) signal wave, for example a digital signal wave, carrying information which represents the program, for example the aforementioned program, which for example comprises code means which are adapted to perform any or all of the method steps described herein.
  • computer program elements can be embodied by hardware and/or software (this includes firmware, resident software, micro-code, etc.).
  • computer program elements can take the form of a computer program product which can be embodied by a computer-usable, for example computer-readable data storage medium comprising computer-usable, for example computer-readable program instructions, "code” or a "computer program” embodied in said data storage medium for use on or in connection with the instructionexecuting system.
  • Such a system can be a computer; a computer can be a data processing device comprising means for executing the computer program elements and/or the program in accordance with the invention, for example a data processing device comprising a digital processor (central processing unit or CPU) which executes the computer program elements, and optionally a volatile memory (for example a random access memory or RAM) for storing data used for and/or produced by executing the computer program elements.
  • a computer-usable, for example computer-readable data storage medium can be any data storage medium which can include, store, communicate, propagate or transport the program for use on or in connection with the instruction-executing system, apparatus or device.
  • the computer-usable, for example computer-readable data storage medium can for example be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, apparatus or device or a medium of propagation such as for example the Internet.
  • the computer-usable or computer-readable data storage medium could even for example be paper or another suitable medium onto which the program is printed, since the program could be electronically captured, for example by optically scanning the paper or other suitable medium, and then compiled, interpreted or otherwise processed in a suitable manner.
  • the data storage medium is preferably a non-volatile data storage medium.
  • the computer program product and any software and/or hardware described here form the various means for performing the functions of the invention in the example embodiments.
  • the computer and/or data processing device can for example include a guidance information device which includes means for outputting guidance information.
  • the guidance information can be outputted, for example to a user, visually by a visual indicating means (for example, a monitor and/or a lamp) and/or acoustically by an acoustic indicating means (for example, a loudspeaker and/or a digital speech output device) and/or tactilely by a tactile indicating means (for example, a vibrating element or a vibration element incorporated into an instrument).
  • a computer is a technical computer which for example comprises technical, for example tangible components, for example mechanical and/or electronic components. Any device mentioned as such in this document is a technical and for example tangible device.
  • acquiring data for example encompasses (within the framework of a computer implemented method) the scenario in which the data are determined by the computer implemented method or program.
  • Determining data for example encompasses measuring physical quantities and transforming the measured values into data, for example digital data, and/or computing (and e.g. outputting) the data by means of a computer and for example within the framework of the method in accordance with the invention.
  • the meaning of "acquiring data” also for example encompasses the scenario in which the data are received or retrieved by (e.g. input to) the computer implemented method or program, for example from another program, a previous method step or a data storage medium, for example for further processing by the computer implemented method or program.
  • the expression “acquiring data” can therefore also for example mean waiting to receive data and/or receiving the data.
  • the received data can for example be inputted via an interface.
  • the expression "acquiring data” can also mean that the computer implemented method or program performs steps in order to (actively) receive or retrieve the data from a data source, for instance a data storage medium (such as for example a ROM, RAM, database, hard drive, etc.), or via the interface (for instance, from another computer or a network).
  • the data acquired by the disclosed method or device, respectively may be acquired from a database located in a data storage device which is operably to a computer for data transfer between the database and the computer, for example from the database to the computer.
  • the computer acquires the data for use as an input for steps of determining data.
  • the determined data can be output again to the same or another database to be stored for later use.
  • the database or database used for implementing the disclosed method can be located on network data storage device or a network server (for example, a cloud data storage device or a cloud server) or a local data storage device (such as a mass storage device operably connected to at least one computer executing the disclosed method).
  • the data can be made "ready for use” by performing an additional step before the acquiring step. In accordance with this additional step, the data are generated in order to be acquired.
  • the data are for example detected or captured (for example by an analytical device). Alternatively, or additionally, the data are inputted in accordance with the additional step, for instance via interfaces.
  • the data generated can for example be inputted (for instance into the computer).
  • the data can also be provided by performing the additional step of storing the data in a data storage medium (such as for example a ROM, RAM, CD and/or hard drive), such that they are ready for use within the framework of the method or program in accordance with the invention.
  • a data storage medium such as for example a ROM, RAM, CD and/or hard drive
  • the n-dimensional image of a tissue is registered when the spatial location of each point of an actual object within a space, for example one or more vessels in a tissue, is assigned an image data point of an image (CT, MR, etc.) stored in a navigation system.
  • CT computed tomography
  • MR magnetic resonance
  • Image registration is the process of transforming different sets of data into one coordinate system.
  • the data can be multiple photographs and/or data from different sensors, different times or different viewpoints. It is used in computer vision, medical imaging and in compiling and analyzing images and data from satellites. Registration is necessary in order to be able to compare or integrate the data obtained from these different measurements.
  • a landmark is a defined element of an anatomical body part which is always identical or recurs with a high degree of similarity in the same anatomical body part of multiple patients.
  • Typical landmarks are for example the epicondyles of a femoral bone or the tips of the transverse processes and/or dorsal process of a vertebra.
  • the points (main points or auxiliary points) can represent such landmarks.
  • a landmark which lies on (for example on the surface of) a characteristic anatomical structure of the body part can also represent said structure.
  • the landmark can represent the anatomical structure as a whole or only a point or part of it.
  • a landmark can also for example lie on the anatomical structure, which is for example a prominent structure.
  • an example of such an anatomical structure is the posterior aspect of the iliac crest.
  • Another example of a landmark is one defined by the rim of the acetabulum, for instance by the centre of said rim.
  • a landmark represents the bottom or deepest point of an acetabulum, which is derived from a multitude of detection points.
  • one landmark can for example represent a multitude of detection points.
  • a landmark can represent an anatomical characteristic which is defined on the basis of a characteristic structure of the body part.
  • a landmark can also represent an anatomical characteristic defined by a relative movement of two body parts, such as the rotational centre of the femur when moved relative to the acetabulum.
  • Determining the position is referred to as referencing if it implies informing a navigation system of said position in a reference system of the navigation system.
  • Atlas / Atlas segmentation Preferably, atlas data is acquired which describes (for example defines, more particularly represents and/or is) a general three-dimensional shape of the anatomical body part.
  • the atlas data therefore represents an atlas of the anatomical body part.
  • An atlas typically consists of a plurality of generic models of objects, wherein the generic models of the objects together form a complex structure.
  • the atlas constitutes a statistical model of a body (for example, a part of the body) which has been generated from anatomic information gathered from a plurality of human bodies, for example from medical image data containing images of such human bodies.
  • the atlas data therefore represents the result of a statistical analysis of such medical image data for a plurality of human bodies.
  • the atlas data comprises image information (for example, positional image information) which can be matched (for example by applying an elastic or rigid image fusion algorithm) for example to image information (for example, positional image information) contained in medical image data so as to for example compare the atlas data to the medical image data in order to determine the position of anatomical structures in the medical image data which correspond to anatomical structures defined by the atlas data.
  • image information for example, positional image information
  • the atlas data comprises image information (for example, positional image information) which can be matched (for example by applying an elastic or rigid image fusion algorithm) for example to image information (for example, positional image information) contained in medical image data so as to for example compare the atlas data to the medical image data in order to determine the position of anatomical structures in the medical image data which correspond to anatomical structures defined by the atlas data.
  • the human bodies the anatomy of which serves as an input for generating the atlas data, advantageously share a common feature such as at least one of gender, age, ethnicity, body measurements (e.g. size and/or mass) and pathologic state.
  • the anatomic information describes for example the anatomy of the human bodies and is extracted for example from medical image information about the human bodies.
  • the atlas of a femur for example, can comprise the head, the neck, the body, the greater trochanter, the lesser trochanter and the lower extremity as objects which together make up the complete structure.
  • the atlas of a brain can comprise the telencephalon, the cerebellum, the diencephalon, the pons, the mesencephalon and the medulla as the objects which together make up the complex structure.
  • One application of such an atlas is in the segmentation of medical images, in which the atlas is matched to medical image data, and the image data are compared with the matched atlas in order to assign a point (a pixel or voxel) of the image data to an object of the matched atlas, thereby segmenting the image data into objects.
  • imaging methods are used to generate image data (for example, two- dimensional or three-dimensional image data) of anatomical structures (such as soft tissues, bones, organs, etc.).
  • image data for example, two- dimensional or three-dimensional image data
  • medical imaging methods is understood to mean (advantageously apparatus-based) imaging methods (for example so-called medical imaging modalities and/or radiological imaging methods) such as for instance computed tomography (CT) and cone beam computed tomography (CBCT, such as volumetric CBCT), x-ray tomography, magnetic resonance tomography (MRT or MRI), conventional x-ray, sonography and/or ultrasound examinations, and positron emission tomography.
  • CT computed tomography
  • CBCT cone beam computed tomography
  • MRT or MRI magnetic resonance tomography
  • sonography and/or ultrasound examinations
  • positron emission tomography positron emission tomography
  • the medical imaging methods are performed by the analytical devices.
  • medical imaging modalities applied by medical imaging methods are: X-ray radiography, magnetic resonance imaging, medical ultrasonography or ultrasound, endoscopy, elastography, tactile imaging, thermography, medical photography and nuclear medicine functional imaging techniques as positron emission tomography (PET) and Single-photon emission computed tomography (SPECT), as mentioned by Wikipedia.
  • PET positron emission tomography
  • SPECT Single-photon emission computed tomography
  • the image data thus generated is also termed “medical imaging data”.
  • Analytical devices for example are used to generate the image data in apparatus-based imaging methods.
  • Mapping describes a transformation (for example, linear transformation) of an element (for example, a pixel or voxel), for example the position of an element, of a first data set in a first coordinate system to an element (for example, a pixel or voxel), for example the position of an element, of a second data set in a second coordinate system (which may have a basis which is different from the basis of the first coordinate system).
  • the mapping is determined by comparing (for example, matching) the color values (for example grey values) of the respective elements by means of an elastic or rigid fusion algorithm.
  • the mapping is embodied for example by a transformation matrix (such as a matrix defining an affine transformation).
  • Fig. 1 illustrates a method for image registration of ultrasound image data according to the present disclosure
  • Fig. 2 is a schematic illustration of the system according to the present disclosure
  • Fig. 3 is a schematic illustration of a registration method according to the present disclosure.
  • Fig. 1 illustrates the basic steps of a computer-implemented method for image registration of ultrasound image data according to the present disclosure.
  • the method comprises, in step S11 , receiving first ultrasound image data of a subject.
  • the method further comprises, in step S12, performing, using a first registration method, a first image registration of the first ultrasound image data with three- dimensional (e.g., ultrasound or CT or MRI) image data of a subject, the three- dimensional image data acquired prior to the first ultrasound image data.
  • the 3D image data may be pre-procedural 3D image data.
  • the method further comprises, in step S13, receiving second ultrasound image data of the subject, the second ultrasound image data acquired after the first ultrasound image data.
  • an initial image registration may be carried out using ultrasound image data obtained at the beginning of the procedure using the first registration method. Thereby, a high-accuracy initial registration is obtained.
  • the second registration method may be employed. Thereby, a fast registration may be provided during the procedure so as to yield real-time representation of the current procedural situation, as will be explained in detail below.
  • step S14 performing, using a second registration method, a second image registration of the second ultrasound image data with the three-dimensional (e.g., ultrasound or CT or MRI) image data, so as to provide an updated image registration.
  • a second registration method performing, using a second registration method, a second image registration of the second ultrasound image data with the three-dimensional (e.g., ultrasound or CT or MRI) image data, so as to provide an updated image registration.
  • the first registration method has higher registration accuracy than the second registration method.
  • the second method on account of being a lower-accuracy method, may be considered to be a faster registration method, e.g. allowing for realtime applications.
  • the method comprises, after the second image registration, for subsequent ultrasound image data, in an optional step S15, continuing performing image registration using the second registration method and subsequently, in an optional step S16, performing image registration using the first registration method.
  • monitoring throughout the medical procedure may be carried out by repeatedly carrying out image registration using the second (fast) image registration method and occasionally reverting to the first (high-accuracy) image registration method.
  • the method may comprise alternating between performing image registration using the first registration method and image registration using the second registration method for subsequent ultrasound image data. Throughout the procedure, predominantly image registration using the second registration method may be carried out, interrupted repeatedly by registrations using the first registration method.
  • the second registration method may, thus, be the default registration method
  • the first registration method may be an auxiliary, e.g. corrective, image registration method.
  • Step S17 illustrates the optional acquisition of the first, second and/or subsequent ultrasound image data input into the respective registration methods.
  • Fig. 2 is a schematic illustration of a system 1 for image registration of ultrasound image data according to the present disclosure.
  • the system is configured to carry out the method of the present disclosure, for example as described in the context of Fig. 1.
  • the system comprises a processing unit 2 configured to carry out the method of the present disclosure, for example as described in the context of Fig. 1 .
  • the processing unit which may be part of a computer.
  • the system may further optionally comprise an electronic data storage device (such as a hard disc) 3.
  • the system may optionally also comprise an ultrasound transducer 4 (such as a 2D or 3D ultrasonic transducer) configured to acquire the first ultrasound image data and/or the second ultrasound image data and/or the subsequent ultrasound image data.
  • the system may optionally also comprise one or more instruments 5 to be used during the procedure.
  • a subject 6 is shown for sake of illustration, but it is to be understood, that the subject is not part of the system.
  • the system may, optionally, comprise a display device 7, e.g. for guidance during the procedure, and/or a user interface 8 for receiving user input, which may be external to or integrated in the display device.
  • the system may also optionally comprise a CT or MRI scanner 9 configured to obtain the CT or MRI image data.
  • the computer in particular the processing unit, may be configured to perform one or more, in particular all, of the steps of the methods according to the present disclosure, particularly the method of the claims or the above-described methods.
  • Fig. 3 is a schematic illustration of a registration method according to the present disclosure, wherein 3D ultrasound data is used as first, second, and subsequent ultrasound image data.
  • the first registration method also referred to hereinbelow as high-quality registration, in this example, is an elastic registration.
  • the second registration method also referred to as fast registration hereinbelow, in this example, is a rigid registration.
  • two rigid registrations having different search ranges may be employed.
  • the high-quality registration is performed.
  • fast registration is performed.
  • the fast registration e.g. rigid, registration
  • the high-quality e.g. elastic, registration
  • X may be selected according to the situation at hand, and may, for example, be empirically or semi-empirically determined.
  • a registration at 10 fps (frames per second) may be obtained without delays, while overall maintaining good quality.
  • a practitioner may also trigger high-quality re-registration when identifying a quality issue or a quality issue may be automatically identified and trigger high-quality re-registration (for example, additionally to the re-registrations performed every X-loops).
  • a possible application for the methods of the present disclosure is the registering of intra-procedural ultrasound image data (obtained during the procedure) to preprocedural 3D CT or MRI or ultrasound image data sets (obtained prior to the procedure), e.g., in the abdominal region.
  • This can be challenging due to breathing motion and deformations of soft tissue.
  • Continuous registration is desired to compensate for this and provide “digital updates” in essentially real-time to the preprocedural data sets using the live ultrasound, but it is challenging to achieve with sufficient accuracy and speed simultaneously.
  • the method of the present disclosure leverages real-time imaging capabilities of ultrasound to show the real-life situation as it is by registering it with a pre-procedural dataset that has high image quality and where procedure planning was done.
  • Existing methods often lack accuracy, as registration is done at the beginning of a procedure, e.g. using a pointer or device such as surface scanner on the skin/bone of the patient, and is rarely repeated during the procedure itself.
  • a 2D ultrasound transducer may acquire 2D ultrasound image data used to reconstruct a 3D volume from the 2D ultrasound image data or a 3D ultrasound transducer may be used to obtain a 3D volume. Data obtained accordingly may then get aligned with 3D pre-procedural image data.
  • the real-life situation may be different to the time point at which the registration was done. Additionally doing registration in soft tissue (such as for the abdominal space) is particularly challenging for this method since the tissues are not fixed to a rigid structure like a bone that could be used as a reference point. Similarly, the real-life situation after the registration can change as soon as the registration ends. After the medical practitioner has carried out procedural steps, some structures might have moved since the initial registration. Thus, one-time registrations might be incorrect after a short period of time. Due to radiation from CT/MRI devices, it is undesirable to do continuous imaging during the steps carried out by the medical practitioner.
  • the method of the present disclosure proposes, as an example, a method having a slower initial registration of ultrasound data to 3D MRI/CT/US image data with a high- accuracy method, followed by, particularly multiple, fast/rapid registrations, which are generally less computationally demanding. These may be seen as update registrations.
  • the slower registration may be repeated to maintain accuracy.
  • real-time registrations can be provided, particularly in a continuous manner while ensuring overall accuracy.
  • the method of the present disclosure may be understood as a two-step cyclical method for image registration of ultrasound image data.
  • the two registration methods i.e., first and second registration methods or fast and high accuracy methods, are alternately carried out, thereby obtaining a cyclical method.
  • the first registration method may comprise elastic registration
  • the second registration method may comprise rigid registration
  • the first registration method may comprise a high precision, large search range rigid registration
  • the second registration method may comprise a less demanding rigid registrations in local regions.
  • the first registration method may comprise a high-accuracy elastic registration
  • the second registration method may comprise fast registration where the elastic deformation is estimated with last N frames of the accurate registration (e.g. elastic registration), for example using a Kalman filter.
  • the first registration method may comprise registration of 3D ultrasound image data
  • the second registration method may comprise the registration of 2D ultrasound image data.
  • this may entail obtaining, as input for the first registration method, 3D ultrasound image data from a 2D ultrasound transducer swipe using a 2D transducer, and subsequently to the first registration, carrying out continuous registrations using 2D image data from the 2D transducer as input.
  • the method of the present disclosure may entail obtaining ultrasound image data via a 3D ultrasound transducer and using the full data as input for the first registration method, and only central slice(s), as input for the second registration method.
  • Deep Learning methods can be used, for example to perform registration of a 2D ultrasound slice to the 3D (e.g. ultrasound, CT or MRI) image data volume registration.
  • any of the above pairs of first and second registration methods may be repeated cyclically, automatically, e.g. after certain number of iterations or in response to detecting quality deterioration, and/or initiated through user interaction, i.e. button on GUI or ultrasound probe.
  • switching back to the first registration method i.e. the high-quality reregistration
  • may be initiated or triggered via user interaction e.g. a button on transducer or screen
  • user interaction e.g. a button on transducer or screen
  • fast registrations e.g. rigid registrations
  • deviation above a certain threshold from an original high-quality registration is determined, among others.
  • the method of the present disclosure allows for a good registration to be done and then maintained throughout the procedure. Registration can be repeated quickly and in a cyclical fashion to compensate for movement and deformations from breathing, procedure-related manipulation, etc.
  • a pre-procedural 3D dataset may be loaded. For example, this may entail loading a high quality image dataset where segmentation and planning was done. Intra-procedural ultrasound data is acquired. Such ultrasound data provides a good view of the real-life/physical situation as it is at that time. An initial high-quality registration is done using the image data from the preceding steps. Thus, good quality is provided after first/original registration.
  • the pre- procedural dataset and planning may be transferred into the current real-life situation in a high-quality way.
  • Live ultrasound image data may then be continuously acquired intra- procedurally. This may provide data in real-time. Said data may be used to maintain a registration of the real-time situation to the pre- procedural dataset and plan.
  • the live ultrasound data (from the preceding step) may be used for rapid registrations to 3D image data (of the 3D dataset from the first step) in real-time. It may be used to maintain the registration accurately as the medical procedure is happening.
  • a high-quality re-registration is triggered, for example in a cyclic manner. For example, this allows to compensate for large motion/deformations in a streamlined manner.
  • the method may entail ensuring that the registration is/remains correct.
  • an inter-frame elastic volume prediction may be used. This may allow to reduce an error caused by non-rigid motion.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)

Abstract

L'invention concerne un système et un procédé mis en œuvre par ordinateur pour la superposition d'images de données d'imagerie ultrasonore. Le procédé de superposition d'images de données d'imagerie ultrasonore comprend a) la réception de premières données d'imagerie ultrasonore d'un sujet ; b) la réalisation, à l'aide d'un premier procédé de superposition, d'une première superposition d'images des premières données d'imagerie ultrasonore avec des données d'imagerie tridimensionnelle, en particulier ultrasonore (US) ou TDM ou IRM, d'un sujet, les données d'imagerie tridimensionnelle étant acquises avant les premières données d'imagerie ultrasonore ; c) la réception de secondes données d'imagerie ultrasonore du sujet, les secondes données d'imagerie ultrasonore étant acquises après les premières données d'imagerie ultrasonore ; et d) la réalisation, à l'aide d'un second procédé de superposition, d'une seconde superposition d'images des secondes données d'imagerie ultrasonore avec les données d'imagerie tridimensionnelle, de façon à assurer une superposition d'images mise à jour, le premier procédé de superposition ayant une précision de superposition supérieure à celle du second procédé de superposition.
PCT/EP2023/058363 2023-03-30 2023-03-30 Système et procédé de superposition d'images de données d'imagerie ultrasonore WO2024199664A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/EP2023/058363 WO2024199664A1 (fr) 2023-03-30 2023-03-30 Système et procédé de superposition d'images de données d'imagerie ultrasonore

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2023/058363 WO2024199664A1 (fr) 2023-03-30 2023-03-30 Système et procédé de superposition d'images de données d'imagerie ultrasonore

Publications (1)

Publication Number Publication Date
WO2024199664A1 true WO2024199664A1 (fr) 2024-10-03

Family

ID=86007297

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2023/058363 WO2024199664A1 (fr) 2023-03-30 2023-03-30 Système et procédé de superposition d'images de données d'imagerie ultrasonore

Country Status (1)

Country Link
WO (1) WO2024199664A1 (fr)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200107884A1 (en) * 2018-10-03 2020-04-09 Canon Medical Systems Corporation Medical image processing apparatus and method

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200107884A1 (en) * 2018-10-03 2020-04-09 Canon Medical Systems Corporation Medical image processing apparatus and method

Similar Documents

Publication Publication Date Title
EP3217884B1 (fr) Procédé et appareil de détermination ou de prédiction de la position d'une cible
US10603120B2 (en) Optimized semi-robotic alignment workflow
EP3143599A1 (fr) Affichage dépendant d'une indication d'une image médicale
US10507064B1 (en) Microscope tracking based on video analysis
US20210343396A1 (en) Automatic setting of imaging parameters
US11495342B2 (en) Planning an external ventricle drainage placement
US11295462B2 (en) Determination of registration accuracy
JP7561819B2 (ja) 人体部分の撮像方法、コンピュータ、コンピュータ読み取り可能記憶媒体、コンピュータプログラム、および医療システム
US11877809B2 (en) Using a current workflow step for control of medical data processing
US11928828B2 (en) Deformity-weighted registration of medical images
WO2024199664A1 (fr) Système et procédé de superposition d'images de données d'imagerie ultrasonore
US20230237711A1 (en) Augmenting a medical image with an intelligent ruler
EP4409512A1 (fr) Système et procédé d'identification d'une région d'intérêt
EP4409594A1 (fr) Détermination de la hauteur de la colonne vertébrale à l'aide de la réalité augmentée
WO2024160375A1 (fr) Calcul d'une surface hybride à partir de données d'image médicale à l'aide de régions de confiance et d'un contrôle de plausibilité