Overall clinical NSA estimates
Minimally invasive and non-invasive image guided therapy can reduce surgical traumas and improve outcome for patients suffering from a wide variety of diseases. It may also reduce hospital stays and costs. Ultrasound is an important intraoperative imaging modality for guidance and monitoring of these therapeutic methods. Ultrasound has emerged as one of the main modalities for medical imaging in healthcare, the main reason being its ability to image soft tissue, blood flow, organ function and physiology with considerably improved image quality. Furthermore, ultrasound has the unique advantages of real time imaging, equipment portability, safety, and low costs. Ultrasound is now facing a paradigm shift in technology and clinical usability over the coming 10 years. The future potential will be released through exploration in knowledge and innovation deliveries in transducer arrays, ultrasound electronics, software beam forming, parallel imaging and compressed sensing, minimum diffractive wave imaging, model powered acquisition and new technology for a wide range of methods related to physiology, tissue properties and organ function in real time and on site. High-frequency ultrasound imaging makes it possible to obtain significantly improved spatial resolution, however, with limitations related to how deep into the tissue the imaging can be performed. In many image-guided surgery and therapy applications, ultrasound is performed with probes placed directly on the tissue and organ of interest (e.g. intravascular ultrasound, open chest cardiac surgery, esophagus probes for cardiac imaging, probes dedicated to surgery of pituitary gland). These applications limit the size of the ultrasound probe head and thus also the quality of the images. However, with miniaturization based on nanomaterials and nanoelectronics technology, significant improvements in image quality may be obtained. Furthermore, new ultrasound technology can greatly enhance the detection of contrast agents and drug carriers in the tissue. Integration of imaging with navigation technologies will ease image interpretation and further improve precision and accuracy of the therapeutic procedure. Ultrasound technology may also be used for therapeutic purposes. High intensity focused ultrasound (HIFU) for ablation of tumor tissue is already a commercial product. It has also been shown that ultrasound may improve the delivery and distribution of nanoparticles and local drug delivery by enhancing the local release, improving the penetration across the capillary wall and through the extracellular matrix as well as enhance the cellular uptake. The underlying mechanisms are cavitation, radiation force and heating. The ultrasound induced transient increase in porosity and permeability of cell membranes can potentially enhance drug uptake through tissue barriers (also the blood-brain barrier) and improve local drug delivery.
Therapeutic use of ultrasound will be addressed at the end of this chapter, which is mainly about guiding instruments into the body in a safe way using ultrasound, as well as the technological solutions involved to augment ultrasound in combination with other modalities and techniques. Ultrasound has been used to guide interventional instruments into the body for a long time. Different approaches have been used. From freehand 2D guidance, via “needle” guides mounted on conventional ultrasound probes to ultrasound-based navigation using tracking technology and 3D ultrasound (see figure 1). Surgical navigation will be the focus of this chapter and the analogy to GPS-navigation in a car is clear; instead of plotting the position of the car onto electronic maps of the terrain using satellites and GPS-receivers the position of important surgical instruments are shown on medical images of the patient using highly accurate tracking systems. Systems for image-guided surgery are now well established within many clinical disciplines. Surgical tools may be tracked by positioning systems and the surgeon may accurately navigate the tools into the patient with high precision based on image information only. Intraoperative imaging has shown to be important for obtaining improved tumor resection and increased survival for cancer patients undergoing surgery. Integration of intraoperative imaging with navigation technology, providing the surgeon with updated image information, is important to deal with tissue shifts and deformations that occur during surgery. MR, CT and ultrasound have been presented as alternative intraoperative imaging modalities showing complementary information and having different benefits and drawbacks. These intraoperative imaging modalities are reported to be useful for accurate navigation of surgical instruments, monitoring the progression of surgery and solving the shift problem. Intraoperative imaging has been used for updating preoperative images, which may be important for accurate guidance. In recent years ultrasound has gained increased attention as a useful intraoperative imaging modality (see figure 2), due to improved image quality and relatively low price. In addition, more integrated solutions, that makes the technology user friendly and flexible has been presented. In the evolution of the next generation of ultrasound-based multimodal navigation systems, advances in ultrasound imaging, registration algorithms, visualization and display techniques and navigation accuracy are important ingredients. We will therefore start by looking into the technology that is needed in order to make ultrasound-based navigation a reality and then show key applications of the navigation technology. Recent advances in ultrasound imaging will be useful also for intraoperative imaging. Furthermore, ultrasound needs to be integrated with tracking technology in order to make a navigation system with intraoperative imaging capabilities. In addition, such a system might be able to use preoperative CT/MR data, update these data to match the current patient anatomy using intraoperative ultrasound, extract important structures from the different datasets, present the available multimodal information to the surgeon in an optimal way and be able to track all the surgical tools. Last but not least we need to make sure that the navigation system is highly accurate so that we know that the navigation scene presented to the surgeon on the computer screen is a realistic representation of what’s really going on inside the patient.
2. Recent advances in ultrasound imaging
Sound in the human audible range have frequencies between 20 and 20 000 Hz. Ultrasound is defined as sound with frequencies above 20 kHz. In medical imaging, the ultrasound frequencies are usually between 2 and 40 MHz, with the highest frequencies currently used in intravascular ultrasound (IVUS).
The generation of an ultrasound image is based on transmission of sound pulses and receiving the echoes that have been reflected from tissue boundaries or scattered from smaller objects. In most conventional scanners today, a narrow ultrasound beam is transmitted from the ultrasound transducer. When the transmitted pressure pulse meets a hinder in form of a boundary between different soft tissues, or scatter points within the tissue with different acoustic properties, some of the energy of the transmitted sound pulse is echoed back to the transducer. This pulse-echo principle forms the basis of all ultrasound-imaging techniques, such as conventional brightness mode (B-mode) imaging of organs, imaging of blood flow using Doppler techniques and exploration of mechanical tissue properties using ultrasound elastography techniques.
2.1. Advances in ultrasound hardware and transducer technology
The ultrasound machines and ultrasound probes have gone through massive improvements in the last decade. The general increase in computer power is opening new possibilities for implementing sophisticated methods for beam forming. This may lead to higher resolution and better image quality than for existing solutions . The general trend with miniaturization of components has also strongly influenced the size of the ultrasound imaging systems. Small handheld ultrasound devices have been developed, which makes ultrasound an extremely portable imaging technology. One example of such a pocket sized ultrasound device is the Vscan from GE Healthcare (figure 3), which has been explored for use in echocardiography . The ultrasound transducer technology has made tremendous progress the last decade. The number of elements used by a transducer is increasing and the trend is to go from a single row of elements (1D) to multi-row arrays (1.25D / 1.5D) and 2D matrix arrays. The latter provides the possibility to perform 4D ultrasound imaging, in which a 3D ultrasound volume is acquired and displayed in real time. 4D ultrasound imaging may also be used for monitoring of treatment, e.g. radiofrequency ablation .
Ultrasound arrays today are mostly based on piezoelectric materials. The research activities in MUT (Micromachined Ultrasound Transducer) technology, and perhaps especially CMUT (capacitive MUT) transducers, pave the way for silicon-based arrays . This may introduce probes that are cheaper, more customizable and have higher frequencies and bandwidth compared to piezoelectric transducers. In combination with the everlasting trend of miniaturization, the CMUTs may in a long-term perspective allow complete ultrasound systems to be seamlessly integrated with surgical tools. It may very well be that the future surgical instrument has an ultrasound transducer integrated on the tip, and a display unit integrated in the handle.
2.2. Ultrasound elastography
The concept of ultrasound imaging of tissue strain or elasticity is often referred to as ultrasound elastography and the corresponding 2D images are frequently called elastograms. The imaging technique is often explained to be analogue to palpation, where the physician uses the fingers to apply a slight pressure in order to examine the stiffness of the tissue. If a organ is vibrating or excited, ultrasound elastography methods can in a similar fashion be used to map areas with differences in strain (figure 4).
The theoretical framework for the study of behavior of vibrating soft tissue was established in the early 1950ies. Von Gierke et al. published “Physics of vibrations in living tissues” in 1952 , for example. However, it was not until 30 years later that tissue movement was first measured for clinical purposes by using ultrasound in a study of tissue motion in the liver caused by vascular pulsation [6, 7]. In the late 1980ies, techniques for vibration elastography imaging, also known as vibration amplitude sonoelastography or simply sonoelasticity imaging was developed . In this technique a low frequency vibration (20-1000 Hz) is applied externally to the skin surface to investigate the subcutaneous structures. The internal motion of the tissue is investigated with a pulsed Doppler technique. Stiff tissue responds differently to the vibrations than softer tissue, and can therefore be distinguished in the real-time images.
In the early 1990ies, the development of compression elastography, also referred to as quasi-static elasticity imaging, begun. Ophir published a paper in 1991 where ultrasound radio frequency (RF) data before and after applying compression were compared and processed using cross-correlation to obtain the time-shifts of the echoes. This allowed the subsequent calculation of elastograms . The quasi-static elasticity imply that the force is applied for a sufficiently long time for the tissue strain to stabilize, and the resulting difference in echo travel time between ultrasound data acquired before and after compression can be calculated. The tissue may also be excited by applying forces at the surface (manually or by electromechanical devices) or by physiological processes within the organ, as for example the pulsation of the arteries. The generated elastograms are usually displayed as a color-coded overlay on the conventional ultrasound brightness mode image. The color mapping may cover a range of unit-less strain values as percentages from minimum (negative) strain to maximum (positive) strain. Alternatively, it may also be mapped from "soft" to "hard" tissue, thereby not quantifying the strain range displayed. Quasi-static elasticity imaging has been evaluated in a broad range of clinical applications. It has been reported used in diagnostics of tumors in for example breast, prostate, liver, the thyroid gland and in the brain (figure 4) [10-15]. Quasi-static elasticity imaging is an emerging ultrasound imaging modality, now becoming more and more available as an option on commercial ultrasound systems.
As previously explained, the elastography methods require that the tissue is excited. The tissue movement can be caused by physiological processes internally in the organ such as the pulsation of the arteries. The tissue can also be externally excited by manually pushing the tissue or by using an electromechanical vibrating device. An alternative approach is to use the acoustic radiation force of an ultrasonic focused beam to generate displacements in the tissue with subsequent detection of the mechanical properties. One example of such an approach is the Acoustic Radiation Force Impulse (ARFI) method developed at Duke University . In this technique, short duration acoustic pulses (push pulses) are used to generate small localized displacements deep in the tissue. These displacements are tracked by ultrasonic cross correlation, in a similar fashion as for the quasi-static elasticity imaging. The method has been investigated for imaging of focal liver lesions, prostate and breast [17-19].
Another example is the innovative Supersonic Shear Imaging (SSI) method developed by the research group at the Laboratoire Ondes et Acoustique . In SSI the acoustic radiation force is used to generate low-frequency shear waves (50-500 Hz) remotely in the tissue. The shear modulus of the tissue can be quantified by imaging the share wave propagation in the tissue by using ultrasound frame rates of several kHz. The method has been explored for diagnosis of liver fibrosis, breast lesions and cornea [21-23].
2.3. Nonlinear acoustics and contrast agents
In 1980, Carstensen and Muir published two papers describing the importance of nonlinear acoustics within the field of medical ultrasound imaging [26, 27]. These papers predicted and demonstrated nonlinear acoustical effects relevant for intensities and frequencies common in biomedical imaging. There has been an increasing interest with respect to nonlinear biomedical acoustics during the last 30 years. This interest was further escalated by the introduction of ultrasound contrast agents in the form of microbubbles and the study of these microbubbles was the main impetus for the introduction of the tissue harmonic imaging technique.
Nonlinear effects can be important in the forward wave propagation. The back-scattered pressure levels of the echoes are typically too low to induce any significant nonlinear effects. One source of nonlinear terms is produced by the deformation of tissue volume elements during compression and expansion with strongly curved phase fronts. It is, however, common to use transmit beams with relatively smooth phase fronts. Consequently, this nonlinear source is usually not the most dominant. The other important nonlinear source is nonlinear terms in the tissue elasticity and hence in the relation between acoustic pressure and tissue compression/expansion. Nonlinear terms in the tissue elasticity are responsible for the fact that the tissue becomes stiffer during compression and softer during expansion. The compression also increases the mass density of the tissue, but this effect is inferior to the increased stiffness and the propagation velocity and will therefore be pressure dependent and will increase with increasing compressions and thus with increasing pressure. The resulting distortion of the transmit pressure field produces harmonic components which today are utilized in tissue harmonic imaging, especially in transcutaneous cardiac and abdominal imaging to suppress multiple scattering [28-31].
Ultrasound imaging is based on several assumptions, and one important assumption is that multiple scattering is neglected. For many organs, this approximation is valid. However, for the body wall, where larger variations in material parameters often are found, this assumption can be inadequate. Interfaces between soft tissue components with significant differences in material parameters give so strong echoes from the transmitted acoustic pulses that multiple scattering can get significant amplitudes. Such multiple scatterings are usually termed pulse reverberations [32, 33]. These reverberations reduce the ratio of the strongest to the weakest scatterer that can be detected in the neighborhood of each other, defined as the contrast resolution in the image. Reduced contrast resolution is in particular a problem when imaging hypo-echoic structures such as the heart chambers, the lumen of large blood vessels, some atherosclerotic lesions, cysts, some tumors, the gallbladder as well as in fetal imaging. The contact interface between the ultrasound transducer itself and the soft tissue is also a strong reflector enhancing the problem with multiple scattering.
Ultrasound contrast agents are made as a suspension of gas microbubbles encapsulated in thin stabilizing shells made from lipid or albumin. Typical bubble size is in the 1-5 μm range and the contrast bubbles are intravenously injected to increase the scattering from blood, which is weak compared to the scattering from soft tissues. Commercially available contrast bubbles are stable and small enough to enable transpulmonary passage and the blood half-life is typically in the range of 1-10 minutes. Scattering from microbubbles occurring within a liquid is resonant through an interaction between a co-oscillating liquid mass around the bubble and the bubble compression elasticity  with typical resonance frequencies of 1-7 MHz. With adequately flexible shells, the gas bubble has a very high compliance relative to the surrounding blood and when driven by ultrasound pulses at frequencies below or around the bubble resonance frequency, large bubble radius excursions on the order of one micrometer is achieved due to mainly shear deformation and limited volume compression of the blood surrounding the bubble. This bubble radius displacement is then between one and two orders of magnitude larger than typical particle displacements obtained within soft tissues. The radius oscillation of a bubble may be obtained from the Rayleigh-Plesset equation [35, 36]:
where is the mass density of the surrounding liquid, is the bubble radius (where the time dependence has been omitted for convenience), is the pressure produced by the gas and the encapsulating shell, is the incident drive pressure and the dots represent differentiation with respect to time so that and represent the velocity and acceleration of the bubble wall, respectively. The terms on the left-hand side represent acceleration forces of the co-oscillation liquid mass whereas the terms on the right-hand side represent pressure terms due to gas and shell elasticity in addition to the drive pressure. The bubble pressure can be written
where the first term is the gas pressure and where is the polytropic exponent of the gas and is the equilibrium bubble radius. The second term is the pressure contribution from the encapsulating shell and is the ambient hydrostatic pressure. The fourth term accounts for surface tension due to the gas-liquid interface and the last term accounts for damping effects. When a contrast bubble is insonified by frequencies below or around its resonance frequency, the local nonlinear scattering from the contrast bubble is usually much larger than from soft tissues [37, 38]. This has resulted in several nonlinear ultrasound contrast agent detection techniques with the purpose to suppress the linear part of a received signal while maintaining as much as possible of the nonlinear part of a received signal. This is then used for low transmit pressure levels. The forward wave propagation is close to linear whereas the scattering from microbubbles can be highly nonlinear. Common techniques in use today are Pulse Inversion methods that detect even harmonic components [39, 40]. Amplitude Modulation methods are also in use , often in combination with Pulse Inversion methods [42-44].
The equations describing the bubble oscillations can be solved numerically. An example of a bubble with equilibrium radius of 2 μm is shown in figure 5. An incident drive pulse with center frequency around 2 MHz is displayed in the time and frequency domain in the upper panel. In the middle panel, the resulting bubble radius oscillation is depicted and in the lower panel, the resulting normalized far-field component of the scattered pressure from the bubble is displayed. It can bee seen that the response is highly nonlinear and several harmonic components are present in the scattered pressure from the bubble. This response is obtained with an incident drive pulse having a mechanical index equal to 0.07, which is very low compared to what is used for regular tissue imaging. At such low transmit pressure levels, the forward wave propagation will be close to linear and distortion of the transmit field due to nonlinear tissue elasticity will thus be very low. The harmonic components can then be used to differentiate bubble echoes from tissue echoes through Pulse Inversion and Amplitude Modulation pulsing schemes. In most clinical applications of ultrasound contrast agents, it is desirable to assess the micro-circulation or the tissue perfusion which cannot be done without the use of contrast agents and which often is related to various diseases. It is then necessary to obtain a strong suppression of the tissue signal for detection of the contrast bubble signal.
An example of the use of ultrasound contrast agents in relation to minimally invasive interventions is radiofrequency ablation of liver tumors where contrast-enhanced ultrasound is used for improved detection and imaging of the lesions, for planning and guidance of multiple needle electrodes and finally for immediate evaluation of the treatment .
SURF (Second order UltRasound Field) imaging is a nonlinear ultrasound imaging technique being developed in Trondheim [46-50]. It is based on transmission of dual frequency band pulse complexes consisting of a low frequency manipulation pulse and a high frequency imaging pulse that are co-propagating. Two transmit pulse complexes that may be used with the SURF technique are displayed in figure 6. With the use of conventional single frequency band transmit pulses, nonlinear effects are mainly restricted to the generation of harmonic components of the imaging pulse. With dual frequency band transmit pulses, other nonlinear effects also come into play. SURF imaging aims at further utilizing nonlinear acoustics for improved imaging of various tissues and ultrasound contrast agents.
For imaging of ultrasound microbubbles, conventional techniques relies on driving the bubble into strong nonlinear oscillations with the imaging pulse at relatively low mechanical indexes. This is typically feasible when the imaging frequency is below or around the bubble resonance frequency (as in the example of figure 5) and conventional contrast agents typically have resonance frequencies below 7 MHz. However, when the imaging frequency is above the bubble resonance frequency a much higher mechanical index is required to obtain significant nonlinear back-scattering from the bubble. At higher mechanical indexes the tissue will also respond nonlinearly and it then becomes difficult to differentiate the tissue signal from the bubble signal. For contrast imaging at high frequencies, such as 10 – 30 MHz, that can be used in minimal invasive interventions where the probe can be close to the object being imaged, conventional contrast imaging techniques often have limitations. The dual band SURF technique then has some advantages where the low frequency manipulation pulse can be tuned to match the bubble resonance frequency (typically around 2-3 MHz) whereas the high frequency imaging pulse can be optimized for the object being imaged and can for example be 20 MHz. The low frequency then manipulates the bubble oscillation and back-scattering which is interrogated by the high frequency pulse. The high frequency imaging pulse is hence decoupled from the resonance properties of the contrast bubbles.
3. Ultrasound-based navigation — Enabling technologies
State of the art ultrasound imaging is crucial for guiding interventions. But unlike freehand guidance and guidance based on ultrasound guides (figure 1) having optimal images on the ultrasound scanner is not enough to enable surgical navigation. In order to use ultrasound-based navigation to guide such procedures we usually have to:
Get the images out of the ultrasound scanner and into the navigation software in real-time.
Track the position and orientation of the ultrasound probe at all times.
Synchronize the image and tracking streams (temporal calibration) and find the transformation between the tracking sensor mounted on the ultrasound probe and the ultrasound scan plane (spatial calibration), which is the interesting part to track.
Reconstruct all the position tagged ultrasound frames from a conventional 2D ultrasound probe into a regular 3D volume that can be used in the same way as preoperative MR or CT is.
3.1. Streaming of ultrasound data
Convenient ultrasound-based navigation of surgical instruments requires real-time access to the ultrasound data in the navigation software (figure 7). This is required in order to tag the ultrasound frames with position and orientation data from the tracking system (alternatively the tracking data could be directed directly into the scanner and the ultrasound frames could be used off-line, e.g. to generate a 3D volume from the tagged 2D frames). The traditional way of getting real-time access to ultrasound frames is to connect the analog output (e.g., composite video, S-video) of the ultrasound scanner to a frame-grabbing card on the navigation computer. Using the analog output might affect the image quality due to the double digital-to-analog-to-digital conversion and no metadata (e.g. depth) follow the ultrasound images. Alternatively digital data can be streamed directly from the ultrasound scanner and into the navigation computer. Traditionally this has required some kind of research collaboration between the ultrasound manufacturer and the user but open ultrasound scanners are becoming available (e.g. the Ultrasonix scanner). These systems usually provide just a one-way streaming interface but two-way communication protocols where the scanner can be controlled (e.g. depth) by the navigation system exists making more integrated solutions possible (figure 7). Either way, the protocol (or interface / API) used is typically proprietary, although proposals for real-time standards are starting to emerge (e.g. OpenIGTLink, DICOM in surgery (WG24)). When the link between the ultrasound scanner and navigation system is digital, ultrasound data at different stages in the processing chain on the scanner can be transferred (e.g. scan-converted, scan-line and RF-data). Furthermore, a digital streaming interface will be required in order to use the real-time 3D scanners that are now becoming available also for navigation. It’s difficult to capture the 3D content in the scanner display using a frame grabber so the data needs to be transferred in real-time or tagged with a tracking reference on the ultrasound scanner.
3.2. Tracking of ultrasound probes
In order to use ultrasound to guide surgical procedures the ultrasound probe must be tracked. Several tracking technologies have been proposed over the years (mechanical, acoustical, optical and electromagnetic), but currently the most widely used solutions are optical or electromagnetic systems (see figure 8). Choosing the best tracking technology depends on the application at hand and the ultrasound probes used. If possible optical tracking systems should be preferred as magnetic tracking in the operating room can be challenging due to disturbances from metallic objects and the accuracy is close but not as good as optical systems under favorable conditions. For flexible us-probes or probes that are inserted into the body magnetic tracking is required as the transformation between the sensor and the scan plane must be rigid and optical tracking demands clear line of sight to the cameras. In addition the magnetic sensors are very small, crucial in order to be embedded in instruments and put into the body. When the ultrasound probe is tracked it becomes one of several tools and the streamed ultrasound data can either be shown in real time at the right spot in the patient or made into a 3D volume and shown together with other images to the surgeon. A brief description of the two main tracking technologies can be found below [51, 52]:
Optical tracking systems: The basic idea is to use one or more cameras with markers distributed on a rigid structure where the geometry is specified beforehand (figure 8A). At least three markers are necessary to determine the position and orientation of the rigid body in space. Additional markers allow a better camera visibility of the tracked object and improve the measurement accuracy. The markers can be infrared light-emitting diodes (active markers), infrared light reflectors (passive markers) or some kind of pattern (usually a checker board) that can be identified using visual light and image analysis.
Electromagnetic tracking systems: A receiver (sensor) is placed on the ultrasound probe and the system measures the induced electrical currents when the sensor is moved within a magnetic field generated by either an alternating current (AC) or direct current (DC) transmitter / generator (figure 8B). The AC and DC devices are both sensitive to some types of metallic objects placed too close to the transmitter or receiver, and to magnetic fields generated by power sources and devices such as cathode-ray tube monitors. Therefore, both types of electromagnetic systems are challenging to use in an environment such as an operating room, where various metallic objects are moved around in the field . The two metal related phenomena that influence the performance of electromagnetic tracking systems are ferromagnetism and eddy currents . Ferromagnetic materials (e.g., iron, steel) affect both AC and DC systems, because they change the homogeneity of the tracker-generated magnetic field, although the DC systems may be more sensitive to these effects. In contrast, the AC technology is more affected by the presence of conductors such as copper and aluminum because of distortions caused by eddy currents [53, 55]. DC systems minimize the eddy-current related distortions by sampling the field after eddy currents have decayed.
Comparisons between optical and magnetic tracking systems - pros and cons: The main advantages with optical tracking systems are their robustness and high accuracy and the challenges are line of sight problems and the relatively big sensor frames. For electromagnetic tracking system it’s basically the other way around.
3.3. Ultrasound probe calibration
After streaming ultrasound data into the navigation software and tracking the ultrasound probe, calibration is needed in order to integrate the image stream with the tracking stream. Ultrasound probe calibration is an important topic as this is the main error source for ultrasound-based navigation (see section on accuracy). Two types of calibration are necessary; temporal calibration to find the lag between the image and tracking streams and spatial calibration [56, 57] to find the transformation between the ultrasound scan plane and the tracking sensor mounted on the ultrasound-probe (see figure 9):
Temporal calibration (find the time lag between the image stream and the tracking stream, see figure 9A) : The most common way to do this is to move the ultrasound probe up and down in a water bath and extract some feature in the generated us-images (or correlate the images and measure the displacement). This gives us two sinus-like curves, one for the vertical position of the extracted feature in the images and one for the vertical component in the tracking data. The two curves are compared and one of them is fitted to the other to find the time lag between the two streams.
Spatial calibration (find the transformation between the image and the sensor, see figure 9B) : Considerable effort has been spent on probe calibration over the last decade, and it still seems to be a hot research topic. Maybe because it is a challenging task to make it accurate, especially if the same method / phantom is to be used for substantially different probes. It is not possible to measure this transform with a ruler because the orientation of the scan plane relative to the sensor frame is unknown, we do not know the origin of the us-plane inside the probe housing and magnetic sensors do not have a known origin. A commonly used approach for probe calibration is to acquire 2-D images of a phantom with known geometry and to identify distinct features in the images. Because the location of the same features are known in the global coordinate system, the probe calibration matrix can be found from a relatively simple matrix equation. The probe calibration methods reported in the literature mainly differ with respect to the phantom geometry, whereas the processing of the acquired data is more or less common for all methods. The majority of probe calibration methods can be categorized into one of three different classes: single- point or line; 2-D alignment; and freehand methods. The calibration matrix can be calculated as follows. Acquire the necessary amount of calibration images and find the coordinates of all the calibration points in each image. Next, we transform the corresponding physical points from global reference coordinates into sensor frame coordinates by using the inverse of the tracking matrix. The rigid body transformation that minimizes the mean Euclidian distance between the two homologous point sets will be the probe calibration matrix. The matrix can be calculated using a direct least squares error minimization technique .
3.4. 3D Ultrasound
It is difficult to guide an instrument into place using conventional 2D ultrasound only (freehand guidance): in order to know where the instrument is we need to see it in the ultrasound image and to reach the target we have to know where to go from there, a challenging hand-eye coordination task. It’s much more convenient to acquire a 3D ultrasound volume first and let the tracked instrument extract slices from the volume that can be annotated with the position and / or orientation of the instrument (see section on visualization).
3D ultrasound data can be acquired in different ways . A conventional 1D array probe (2D+t) can be moved over the area of interest, either by freehand motion or by a motor. If freehand movement is used all the ultrasound frames can be put together into a volume using tracking data (figure 10) or correlation. A motor inside the probe hosing or external to it can also be used to cover the ROI by tilt, translation or rotation of the 1D array (figure 11). Furthermore, with a 2D matrix probe the ultrasound beam can be steered in the elevation direction in addition to the lateral (azimuth) direction so that the ROI can be covered while the probe is standing still making real-time 3D ultrasound imaging possible (figure 12).
In practice the following methods are in use:
Freehand 3D ultrasound: This is still the most wildly used method (mainly because of its flexibility) and usually the method works in the following manner: Scan the area of interest using a conventional 2D probe that is tracked and reconstruct the position tagged ultrasound frames into a regular 3D volume that can be used in the same way as preoperative MR or CT. The ultrasound probe is usually tracked by optical or electromagnetic sensors, but other methods have been proposed. Furthermore, different methods exist to reconstruct all the 2D frames into a regular 3D volume. The methods can be categorized into tree main groups :
Voxel-based methods (VBM): VBM traverse each voxel in the target voxel grid and gather information from the input 2D images to be placed in the voxel. One or several pixels may contribute to the value of each voxel. The simplest method in this category is Voxel Nearest Neighbor (VNN), which traverses each voxel in the target volume and assigns the value of the nearest image pixel (see figure 10A).
Pixel-based methods (PBM): PBM usually consists of two steps: a Distribution Step (DS) where the input pixels are traversed and applied to one or several voxels and a Hole-Filling Step (HFS) where the voxels are traversed and empty voxels are being filled. The simplest method in this category is Pixel Nearest Neighbor (PNN) that runs through each pixel in all the 2D input images and assigns the pixel value to the nearest voxel in the target volume (see figure 10B).
Function based methods (FBM): FBM choose a particular function (like a polynomial) and determine the coefficients to make the functions pass through the input pixels. Afterwards, the function can be used to create a regular voxel array by evaluating the function at regular intervals (see figure 10C). These methods produce reconstructed volumes with the highest quality but are very computational intensive and are in limited use today.
Motorized (or mechanical) 3D ultrasound: Instead of using freehand movement of the ultrasound probe over the area of interest a motor can cover the same region by tilting (figure 11A), translating (figure 11B) or rotating (figure 11C) a conventional 1D ultrasound array. Motorized probes have existed for a long time and the motor can either be mounted inside the probe housing (easy to use but requires a specially build ultrasound probe) or be applied externally (more flexible as conventional probes can be used). Many of the benefits with freehand scanning also apply to motorized scanning, e.g. the possibility to use high frequency probes with higher spatial resolution, also in the elevation direction (1.25D/1.5D probes). Motorized scanning can use the same kind of reconstruction methods as freehand scanning but usually more optimized methods are used as the movement is known and the probe do not need to be tracked during the acquisition. Compared to freehand ultrasound the motorized probes are easier to use in an intraoperative setting, but on the other hand, they are not as flexible in general.
Real-time 3D ultrasound using 2D matrix probes [61-65]: Instead of using a conventional 1D array transducer that is moved by freehand or by a motor to sweep out the anatomy of interest, transducers with 2D phased arrays (figure 12A) that can generate 3D images in real time have been developed. Electronics is used to control and steer the ultrasound beam (figure 12B) and sweep out a volume shaped like a truncated pyramid (figure 12C). The main challenge with this technology is the large and heavy cable that would be required to connect all the elements in the array to a wire. Fortunately technological achievements in terms of multiplexing, sparse arrays and parallel processing over the last decade have made these systems commercially available. They are used extensively in echocardiology, which requires dynamic three-dimensional imaging of the heart and its valves.
3.5. Integrated ultrasound-based navigation solutions
Ultrasound and navigation can be integrated in different ways as we have seen. Complete systems can usually be categorized as follows:
Two-rack systems: Where the navigation computer with tracking system etc. and the ultrasound scanner are two separate systems. This is most common, especially in a research environment. The main reason for this is flexibility, in principle any ultrasound scanner with an analog output can be used together with a navigation system that is equipped with ultrasound-based navigation software. An example of such a configuration is our in house research system for us-based navigation called CustusX (figure 13A). The system is used for different clinical applications (e.g. neurosurgery and laparoscopy), each navigation rack is equipped with both optical and magnetic tracking and can be connected to a variety of ultrasound scannersusing analog and digital interfaces.
One-rack systems: Here the ultrasound scanner and the navigation computer have been integrated in the same system. These systems are more convenient to use in the operating room but are less flexible. Most commercial solutions belong to this category. Two variations exists:
An ultrasound scanner with navigation software integrated. The PercuNav system from Philips, an integrated solution for navigation and intraoperative imaging, is an example of this (figure 13B).
A navigation system with an ultrasound scanner integrated: The SonoWand system (Trondheim, Norway), where an ultrasound scanner has been embedded in the navigation rack, is an example of this (figure 13C). The system can be used in three distinct ways: 1) as a navigation system based on preoperative MR/CT data, 2) as a standalone ultrasound scanner and 3) as an ultrasound-based navigation system with intraoperative imaging capabilities, its main use.
4. Registration and segmentation in ultrasound-based navigation
Registration is the process of transforming an image into the coordinate system of a patient, or another image. After registration, the same anatomical features have the same coordinates in both the image and the patient, or in both images. Image-to-patient registration is one of the cornerstones of any navigation system, and is necessary for navigation using pre-operative images such as MR and/or CT. Image-to-image registration is useful to align pre-operative images before registration to the patient, and also to update the pre-operative images during surgery using for example intra-operative US. Only the latter involves US and will be the focus in this section, but image-to-patient registration is important for proper initialization of the MR/CT-to-US registration. The main motivation behind image-to-image registration is that different images contain different and complimentary information about the patient at a given point in time. When we bring the images into the same coordinate system and into the coordinate system of the patient, we can take advantage of more of the useful information in the different images. Such information can be the size and location of the surgical target, important blood vessels, critical structures that should be avoided etc. The registration method used in each case depends heavily on the type of images we want to register. The type of spatial transformation, how we measure the similarity between the images and how this measure is optimized are key components of any registration procedure.
4.1. Registration of preoperative images to the patient
Image-to-patient registration is a necessary and crucial step in order to use pre-operative images for guidance. Intraoperative ultrasound only shows a limited portion of the surgical field and might require some experience to appreciate. Preoperative data can therefore be used for overview and interpretation. In neurosurgery, for example, it is not possible to acquire ultrasound images before opening of the dura. Pre-operative images are therefore necessary for planning the craniotomy.
One of the most frequently used registration methods consists in using self-adhesive markers, also called fiducials. The fiducials are glued to the patient's skin before MR or CT imaging. The markers can be identified in the images and the corresponding markers can be identified on the patient using a tracked pointer once the patient is immobilized on the operating table (figure 14). A spatial transformation can then be computed transforming the image into the coordinate system of the patient. The surgeon can then point on the patient using a tracked pointer and see the corresponding location in the images on the computer screen. The use of markers for image-to-patient registration presents some limitations both for the patient and the hospital staff. First, fiducial based registration requires an imaging session shortly before surgery to minimize the risk for markers to fall off or be displaced. In many cases this imaging session comes in addition to an initial session needed for diagnosis. Any displacement of the fiducial markers between the imaging session and surgery will compromise the image-to-patient registration accuracy. The placement of fiducials also represents an inconvenience for patients and hospital staff in the preparations for the procedure.
In order to avoid the use of fiducial markers, natural anatomical landmarks can be used for patient registration. Typical features in the context of neurosurgery are the medial and lateral corners of the eyes, the nose and ears. Like fiducial based registration, an image-to-patient registration framework using natural anatomical landmarks requires identification of points in the pre-operative images. The typically used landmarks are almost coplanar, and they are all located in a relatively small area around the face and ears. This might compromise the registration accuracy in other parts of the head, and possibly close to the surgical target . A number of groups have presented surface matching techniques to address this issue. The skin surface of the patient is segmented from pre-operative data and registered to a set of surface points acquired in the operating room. Techniques to acquire surface points in the operating room include cameras [67, 68], laser surface scanners [69-71] and tracked pointers . The accuracy of the different methods has been evaluated and compared [71, 73-75]. Both landmarks and surface based registration alone are less accurate than fiducial based registration. Different approaches combining registration based on anatomical landmarks and alignment of surfaces have therefore been developed.
As surgery proceeds, tissue will shift and deform due to gravity, retraction, resection and administration of drugs. Consequently, the pre-operative images do not correspond to the patient anymore. In this case, intraoperative ultrasound can be used for direct guidance and to update the location of the pre-operative data according to the surgical reality at a given point in time.
4.2. Ultrasound-based update of preoperative data
As surgery proceeds the pre-operative images no longer reflect the reality and updated information is necessary for accurate navigation. Intra-operative ultrasound can be acquired when needed during the procedure and be used for direct guidance and resection control, but also as a registration target for pre-operative images in order to update their position. This is particularly important for images such as functional MRI (fMRI) and diffusion tensor imaging (DTI) in neurosurgery because the information contained in these images cannot be easy re-acquired during the procedure. By performing MR/CT-to-US registration, the information contained in the pre-operative images can be shifted to the correct position at any given point in time (figure 15). Registration of MR/CT to US is a challenging task due to differences in image appearance and noise characteristics. The existing methods can be divided into two main categories:
Intensity-based methods: These methods take the original images (MR/CT and B-mode US) as input, and the optimization of the registration parameters is computed from the image intensities, either directly or indirectly (blurring, gradients etc.). Some of the existing methods use well-known similarity measures such as mutual information and cross-correlation, while others have developed similarity measures particularly adapted to the registration of MR/CT and ultrasound [76-82].
Feature-based methods: These methods require segmentation or “enhancement” of particular features in the images to be registered. The registration algorithm will then align the corresponding features in each image. In MR/CT-to-US registration such feature might be the vascular tree [83-85]. Blood vessels are relatively easy to identify and segment in both MR angiography and Doppler ultrasound images, and are present in nearly any region of interest. A centerline or skeleton can be computed from the segmented vessels and be used for registration. The most commonly used method for feature-based registration is the iterative closest point algorithm (ICP) . In the case of vessel registration, all the points in the moving dataset are paired with the closest point in the fixed dataset. Based on these point correspondences, the registration parameters can be computed using the least squares method. The resulting transformation is then applied to the moving dataset and new point correspondences can be computed. The process is then iterated until convergence.
Several methods within the two main categories have been validated using retrospective clinical data [12, 14, 15]. So far no automatic method has been thoroughly validated intraoperatively (figure 15). The use of automatic registration methods in the operating room requires high quality data and straightforward, accurate, robust and fast image processing. With all this in place, image registration using intraoperative ultrasound will be able to correct the position of pre-operative data and thereby provide updated and reliable information about anatomy, pathology and function during surgery.
4.3. Motion correction using 4D ultrasound
Intensity based registration of ultrasound images can also be used to track the motion of an organ of interest. In the case of high-intensity focused ultrasound (HIFU or FUS) or radiotherapy, the organ can be imaged using 4D ultrasound (3D + time or real-time 3D) in order to monitor the temporal changes in anatomy during the imaging, planning and delivery of treatment. The consecutive 3D images can then be registered in order to estimate the organ motion (figure 16). The positioning of the HIFU or radiation beam can then be modified accordingly in order to hit the target at any point in time. We have validated automatic motion estimation from 4D ultrasound in the liver using a non-rigid registration algorithm and a group-wise optimization approach as part of an ongoing study to be published in the near future. The offline analysis was performed using a recently published non-rigid registration algorithm that was specifically designed for motion estimation from dynamic imaging data . The method registers the entire 4D sequence in a group-wise optimization fashion, thus avoiding a bias towards a specifically chosen reference time point. Both spatial and temporal smoothness of the transformations are enforced by using a 4D free-form B-spline deformation model. For the evaluation, three healthy volunteers were scanned over several breath cycles from three different positions and angles on the abdomen (nine 4D scans in total). A skilled physician performed the scanning and manually annotated well-defined anatomic landmarks for assessment of the automatic algorithm. Four engineers each annotated these points in all time frames, the mean of which was taken as a gold standard. The error of the automatic motion estimation method was compared with inter-observer variability. The registration method estimated liver motion better than the individual observers and had an error (75% percentile over all datasets) of 1 mm. We conclude that the methodology was able to accurately track the motion of the liver in the 4D ultrasound data. This methodology may be used intraoperatively to guide ablation of moving targets in the abdomen if the registration method can be run in real-time and the ultrasound probe can be made MR compatible (required for MR-guided HIFU).
4.4. Segmentation of ultrasound data
Fully automatic segmentation of structures from B-mode ultrasound images is a challenging task. The clarity and contrast of structure boundaries depend heavily on their orientation relative to the sound wave and the acoustic properties of the surrounding tissues. Consequently, the boundaries of interest are often broken or at least unclear in parts of the image volume. It is therefore necessary to use a priori knowledge about the shape and appearance of the structure of interest in order to obtain reliable segmentation results. This a priori knowledge can be obtained by manually segmenting the structure of interest in a set of training data. Then, shape and appearance statistics can be used to segment the structure in new datasets. Akbari et al.  and Zhan et al.  used this approach for segmentation of the prostate in 3D ultrasound images of the prostate, and Xie et al.  used a similar approach for segmentation of the kidneys from 2D ultrasound images. The disadvantage of this method is the requirement for a database of training data with manual segmentations. This method can also be difficult to employ if the shape and appearance of the structure is unknown or presents large variations such as tumors and other pathologies. Several groups have also presented segmentation algorithms for ultrasound images of bone surfaces, and particularly the spine [91-94]. In these cases, the purpose of the segmentation process is to extract the bone surface from intra-operative ultrasound images for registration to pre-operative CT images. The ultrasound images are filtered in order to highlight the bone surface and in some cases the characteristic shadow behind the bone surface can be used for segmentation purposes as shown by Yan et al. . They used backwards scan-line tracing to extract the bone surface from ultrasound images of the spine.
One of the great advantages of ultrasound is real time dynamic imaging. Methods based on shape and appearance statistics are in general not able to run fast enough to capture the dynamics of a moving organ such as the heart. Orderud et al.  proposed a method for real time segmentation of the beating heart. They fitted a set of control points of a model of the left ventricle to 4D ultrasound data (figure 17). The fitting process was run in real time using a state estimation approach and a Kalman filter. When the shape, appearance and localization of the structure are unknown semi-automatic or manual segmentation by an expert might be the only solution to obtain satisfactory results. Segmentation of Doppler ultrasound images, on the other hand is usually straightforward using simple thresholding methods. Vascular structures, however, often appear with a diameter that is to large in the Doppler ultrasound images causing neighboring vessels to be smeared together. Reliable segmentation of the vascular tree can therefore be challenging due to the spatial resolution of the images.
5. Ultrasound-based visualization and navigation
The amount of image data available for any given patient is increasing and may include pre-operative structural data such as CT and MRI (T1, T2, FLAIR, MR angiography etc.), pre-operative mapping of important gray (fMRI) and white matter (DTI), functional data from PET, intra-operative 3D ultrasound (B-mode and Doppler) in addition to images from microscopes, endoscopes and laparoscopes. All these sources of information are not equally important at all times during the procedure, and a selection of data has to be made in order to present only those images that are relevant for the surgeon at that particular point in time.
There are various ways to classify the different visualization techniques that exist. For medical visualization of 3D data from modalities like CT, MRI and US, it is common to refer to three approaches:
Slicing: Slicing means extracting a 2D plane from the 3D data and can further be classified according to how the 2D slice data are generated and how this information is displayed. The sequence of slices acquired by the modality and used to generate a regular image volume is often referred to as the raw or natural slices. From the reconstructed volume we can extract both orthogonal (figure 18A) and oblique (figure 18B) slices. Orthogonal slicing is often used in systems for pre- and postoperative visualization, as well as in intraoperative navigation systems, where the tip of the tracked instrument determines the three extracted slices. The slices can also be orthogonal relative to the tracked instrument or the surgeon’s view (i.e., oblique slicing relative to the volume axis or patient), and this is becoming an increasingly popular option in navigation systems. When a surgical tool cuts through multiple volumes several slices are generated. These slices can then be combined in different ways using various overlay and fusion techniques.
Direct volume rendering: Volume- and geometric rendering techniques are not easily distinguished. Often the two approaches can produce similar results, and in some cases one approach may be considered both a volume rendering and a geometric rendering technique. Still, the term volume rendering is used to describe a direct rendering process applied to 3D data where information exists throughout a 3D space instead of simply on 2D surfaces defined in (and often extracted from) such a 3D space. The two most common approaches to volume rendering are volumetric ray casting and 2D/3D texture mapping (figure 17 A, B, D, E, G). In ray casting, each pixel in the image is determined by sending a ray into the volume and evaluating the voxel data encountered along the ray using a specified ray function (maximum, isovalue, compositing). Using 2D texture mapping, polygons are generated along the axis of the volume that is most closely aligned with the viewing direction. The data is then mapped onto these quads and projected into a picture using standard graphics hardware.
Geometric surface rendering: The technique used to render the texture-mapped quads is essentially the same technique that is used to render geometric surface representations of relevant structures (figure 17 A-F). However, the geometric representations must first be extracted from the image information. While it is possible in some cases to extract a structure and generate a 3D model of it by directly using an isosurface extraction algorithm , the generation of an accurate geometric model from medical data often requires a segmentation step first. The most common surface representation is to use a lot of simple geometric primitives (e.g., triangles), though other possibilities exist. Furthermore, the surfaces can be made transparent so that it’s possible to see what’s beneath the structure.
The challenge is to combine the available data and visualization methods to present an optimal integrated multimodal scene that shows only the relevant information at any given time to the surgeon. Multimodal visualization and various image fusion techniques can be very beneficial when trying to take advantage of the best features in each modality. It is easier to perceive an integration of two or more volumes in the same scene than to mentally fuse the same volumes when presented in separate display windows. This also offers an opportunity to pick relevant and necessary information from the most appropriate of the available datasets. Ideally, relevant information should include not only anatomical structures for reference and pathological structures to be targeted, but also important structures to be avoided. Finally, augmented reality techniques can be used to mix the virtual representation of the patient provided by 3D medical data and models extracted from these and the real representation provided by a microscope or a laparoscope for example, giving an even more realistic picture of the treatment delivered through small incisions in minimally invasive procedures.
6. Ultrasound-based navigation accuracy
The delicacy, precision and extent of the work the surgeon can perform based on image information rely on his/her confidence in the overall clinical accuracy and the anatomical or pathological representation. The overall clinical accuracy in image-guided surgery is the difference between the location of a surgical tool relative to some structure as indicated in the image information, and the location relative to the same structure in the patient. This accuracy is difficult to assess in a clinical setting due to the lack of fixed and well-defined landmarks inside the patient that can be accurately reached with a pointer. Common practice is therefore to estimate the system’s overall accuracy in a controlled laboratory setting using precisely built phantoms. In order to conclude on the potential clinical accuracy, the differences between the clinical and the laboratory settings must be carefully examined.
6.1. Error sources and key points
A comprehensive analysis of the error sources involved in neuronavigation based on intraoperative ultrasound as well as preoperative MRI can be found in Lindseth et al. . The overall accuracy is often referred to as the Navigation System Accuracy (NSA) and the essential points to remember can be summarized like this:
The accuracy associated with navigation based on pre.op. MR/CT is independent of the accuracy associated with navigation based on intraoperative ultrasound, and vice versa.
The main error sources associated with preoperative MR/CT-based navigation are related to the patient registration process in a clinical setting, and the fact that the image maps are not updated to reflect the changing patient terrain as surgery proceeds.
In contrast, intraoperative ultrasound volumes are acquired in the same coordinate system as navigation is performed. Patient registration is therefore not necessary, and a new ultrasound volume can be acquired to reflect the current patient anatomy whenever needed. However, navigation based on ultrasound is associated with its own error chain. The main error source in this chain is the ultrasound probe calibration process. In addition, small variations in the speed of sound in different tissue types are a potential problem .
These points have major implications for the rational behind testing a navigation system in the lab using a phantom, and make a statement about the interesting parameter to the surgeon: the overall clinical navigation system accuracy. A lab test of a system based on preoperative MR/CT using a rigid phantom will give a very good navigation system accuracy (NSA<0.5mm, se figure 19, red line). Such a test will have limited validity in the general clinical situation, but is important to make sure that the system works as expected. The next phase in the evaluation of such a system would be to conduct a clinical study to investigate the system’s ability to deal with a variety of different patient registration problems. Documenting that the system performs well in the rigid case and can deal in a satisfactory way with difficult patient registration cases is the best a system vendor can do. This does not give any information about the NSA experienced during a clinical case though. The surgeon must verify that the accuracy is acceptable after he has performed the patient registration procedure and anatomical landmarks inside the patient must be used to gain an impression about the amount of tissue shift and deformation. This shift and deformation makes systems based on preoperative MR/CT of limited use during the procedure.
In contrast, probe calibration, the major error source associated with ultrasound-based navigation, is included in the NSA resulting from accuracy evaluations using a rigid phantom in a laboratory setting. Furthermore, the surgeon is in control of the amount of tissue shift and deformation that is acceptable in a particular clinical case. A new scan can be acquired whenever needed in order to navigate using an updated image map (se figure 19, green line). As a consequence, the NSA found in a controlled laboratory setting will also be valid in the clinical case given that navigation is based on a recently acquired ultrasound scan (real-time 3D ultrasound being the extreme case) and that the speed of sound used in the ultrasound scanner corresponds to the average speed of sound in the tissue.
A common mistake is to interpret a mismatch between MR/CT and Ultrasound in corresponding or fused displays as tissue shift. An observed mismatch between MR/CT and Ultrasound can only be interpreted as brain shift if 1a) navigation based on pre.op. data is accurate in the rigid case, 1b) the NSA, after the patient registration process, has been verified to be low, 2a) the NSA of ultrasound-based navigation in a controlled setting is low and 2b) the ultrasound data shown originate from an ultrasound volume that has recently been acquired.
Preoperative MR/CT data can be “corrected” for brain shift using intraoperative ultrasound and advanced image-to-image registration techniques  as can be seen in figure 19. However this is a challenging task introducing additional error sources. Therefore the NSA associated with corrected preoperative MR/CT will not be as good as the NSA for ultrasound (see figure 19, blue lines). In addition, the independence between the NSA based on MR/CT and Ultrasound will be broken (NSA for MR/CT will be dependent on NSA for Ultrasound).
The overall clinical accuracy of a navigation system will be determined by the contribution from all the individual error sources involved . The net effect will not be the sum of all the error sources, but rather a stochastic contribution from all the terms. Stochastically independent contributions are summed using the following equation:
6.2. Clinical navigation system accuracy
As stated previously, the most important parameter for the surgeon is the overall clinical Navigation System Accuracy (NSA). Although this parameter is difficult to assess, we believe that for ultrasound-based navigation an estimate can be made, based on a comprehensive laboratory evaluation and a thorough understanding of the significant additional error sources that occur in the clinical setting. Table 1 summarizes how such a calculation can be carried out assuming that a comprehensive evaluation of the system gives a NSA below 1.4 mm in a controlled laboratory setting. The error sources are assumed to be stochastically independent so that their contributions can be added on a sum-of-squares basis.
|NSA using a phantom in the lab||< 1.4 mm|
|+ Calibration and position tracking of rigid surgical tool||< 0.5 mm|
|+ Interpolation of a 2D slice from a 3D volume / tool cross indication||< 0.1 mm|
|= Overall NSA||< 1.5 mm|
|+ Sound speed uncertainty||0 – 2.0 mm|
|+ Brain shift||0 – 10.0 mm|
|= Overall clinical NSA||1.5 – 10.5 mm|
As can be seen from table 1 it is possible to achieve an overall clinical NSA close to the NSA found in the laboratory under favorable conditions, i.e., when the speed of sound used in the scanner is close to the average speed of sound in the tissue imaged, and the ultrasound volumes are frequently updated. The need for updates can be determined by real-time 2D imaging. If these conditions are not met, the accuracy becomes poorer.
6.3. Method for assessing ultrasound-based navigation accuracy
As we have seen the ultrasound-based NSA found in the lab using a phantom is valid in the OR (Operating Room) as well, under normal conditions. This makes it very interesting to develop a method that can measure the NSA automatically. We have previously suggested a method based on a phantom with 27 wire crosses and correlating an ultrasound sub-image of each cross to a synthetic template of the cross , and the method has been used in a thorough accuracy evaluation of a commercial navigation system . We have since that developed a method that seems to be even more robust, in addition to being more flexible and more convenient to integrate in a navigation system (see figure 20). The method can be used for substantially different ultrasound probes and the phantom is easier to build and to measure accurately. The technique is based on sweeping over the single wire cross with the ultrasound probe, reconstruct all the frames into a volume containing the cross, segment and extract the centerline of the cross and register it to a centerline representation of the accurately measured physical cross, acting as a gold standard, using a modified version of the ICP algorithm .
7. Ultrasound-based guidance of minimally invasive procedures
While the main focus of this chapter will be navigation and image guidance using 3D ultrasound images, conventional 2D ultrasound is used for guidance in a variety of clinical applications. The simplest form of ultrasound guidance is placement of a needle inside a target using freehand 2D ultrasound imaging. First, the operator has to localize the target using ultrasound imaging, and second place the needle inside the target while keeping the needle tip in the image plane in order to verify its position. This technique requires a skilled and experienced operator due to the difficulty in keeping the needle in the image plane and the fact that the ultrasound image is not oriented relative to the patient. Despite the difficulties, this technique has been used for biopsies of the liver [99-101], lung  and prostate , placement of central vein catheters [104, 105] and for brain operations .
A slightly more advanced technique for 2D ultrasound guidance includes a needle guide mounted on the ultrasound probe. The guide will ensure that the needle tip is in the image plane at a given depth depending of the ultrasound image sector and the angle of the needle guide. The angle of the needle guide has to be adapted to the depth of the target. Even though this system provides assistance in keeping the needle in the image plane, the operator has to do imaging and puncturing at the same time. In addition, the orientation issues concerning the ultrasound image relative to the patient is not solved and the anatomical overview is restricted to the current real time 2D image. However, the method is fast, does not require specialized equipment or complicated logistics, and provides sufficient guidance for a number of applications such as biopsy of thyroid nodules , placement of ventricular catheters in the brain [108, 109] and amniocentesis .
7.1. Ultrasound-based navigation in neurosurgery
Neuronavigation is the term used to describe the use of computer-assisted methods to guide or navigate instruments within the confinements of the scull (or spinal column) during surgery. A neuronavigation system should ideally provide high navigation accuracy throughout the surgical procedure. However, the anatomy of the brain is known to shift position after opening of the skull and dura due to drainage of cerebrospinal fluid (CSF), gravity effects and/or removal of tumor masses or hematomas. This shift in the position of the anatomy is often referred to as brain shift and has been shown to occur in the early stage of the surgery with displacement values ranging up to several centimeters [111-113]. The brain shift may therefore significantly impair the accuracy of navigation based on preoperative images as the surgery proceeds. Intraoperative ultrasound imaging provides a solution to the brain shift problem. Compared to using only preoperative images for guidance, the navigation of instruments based on recently acquired intraoperative images can be performed with higher accuracy and precision .
The combined use of ultrasound imaging and navigation technology has been explored since the early 1990ies. The University of Oulu was one of the pioneers and demonstrated the clinical use of a passive mechanical arm-based navigation system, which could display reconstructions of preoperative images (CT/MR) and corresponding real-time intraoperative ultrasound images .
By attaching position sensors (also referred to as 3D localizers) on the ultrasound probe it is possible to establish the relative spatial position of the image pixels, and it is possible to reconstruct 2D images into an image volume, hence the term 3D ultrasound. The localizer attached to the probe is usually ultrasonic, electromagnetic or optic, and the two latter options (optic, electromagnetic) are currently the most established in commercial systems. Hata et al described in a paper from 1997 the initial clinical experience with a frame- and armless navigation system incorporating an ultrasound scanner and an ultrasound probe equipped with an ultrasonic positioning sensor . In 1998 Jödicke et al presented a system for detection of brain shift, by comparing preoperative MR images and intraoperative 3D ultrasound . The integration of ultrasound and navigation technology was also explored in Trondheim, Norway, and a system with the feasibility of 3D ultrasound and navigation guidance was developed. Using this system Unsgaard et al. performed the first brain tumour operation with 3D ultrasound guidance in 1996, and the system development and clinical experience was described in several papers [117-119]. The technology was further developed and commercialized by the company Sonowand AS (Trondheim, Norway), which is a spin-off company from the research activities of the National Centre for 3D Ultrasound in Neurosurgery (1995-present (2013)) at St. Olavs University Hospital. The technology has been explored for use in several neurosurgical procedures, but its predominant use is within resection of brain tumours . The Sonowand system allows navigation of pre-calibrated tools equipped with an optic localizer, and it allows tools like biopsy forceps to be calibrated to the navigation system in situ in the operating room (figure 21) and used for image guided biopsies. The system facilitates simultaneous displays of reformatted image slices of intraoperative ultrasound and any preoperative MR series like T1, T2, FLAIR, etc. that has been registered to the patient. The position of navigated instruments is indicated in the displayed image slices.
7.1.1. 3D Ultrasound in intracranial tumour surgery
Intracranial tumours include primary and secondary tumours in the brain, pituitary gland, and meninges. Primary tumours are neoplasms originating from supportive tissue in the brain, from meninges, or from pituitary tissue. Secondary brain tumours are metastases of malignant cells that originate from a primary tumour situated in another organ of the body that spreads with the blood flow to the brain. Surgery is the primary treatment for most intracranial tumours. The patient's prognosis is in most cases related to the degree of resection of tumour. The surgical goal is usually to perform a total extirpation of the tumour, but without damaging adjacent normal brain tissue. If the tumour is located in so-called eloquent regions, harboring important functional tissue for movement, speech or vision, less extensive resections is often the result. Brain tumour surgery can therefore be a delicate balance between obtaining extensive resections and avoiding functional deficits and loss of quality of life due to the surgical trauma.
3D ultrasound is an established technique for intraoperative imaging in surgery of brain tumours, and is used for localization of the tumour and for resection control. The first acquisition of 3D ultrasound images is usually performed after opening the bone (craniotomy), but before opening the dura. Several ultrasound volumes (typically 3 to 6) are acquired during the operation to compensate for brain shift and to monitor the progress of tumour removal (figure 22).
Preoperative MR data can be displayed along with one or several ultrasound image volumes acquired at different stages of surgery. It may also be possible to import functional MR images to the navigation system. One way of doing this is to import anatomical MR images (e.g. T1/T2/FLAIR) with bold fMRI enhancements and DTI tractography overlaid as contours on the anatomical images [121-123], as shown in figure 23. The navigation system may therefore provide multimodal visualization of medical images, incorporating functional and anatomical information.
Clinically, modern image technology has enabled more targeted surgical approaches, as compared to standardized explorative brain dissections that were more common two decades ago. This reduces the surgical trauma, eases anatomical orientation within the surgical field, and makes it possible for less experienced surgeons to obtain the same results as their more experienced peers. Today, even in eloquent regions where surgery is associated with increased risk, good clinical results can be obtained . We have also observed that survival increased after the introduction of 3D ultrasound imaging in malignant primary brain tumour surgery . Intraoperative imaging with ultrasound has also enabled more aggressive treatment strategies in tumours that microscopically resemble the brain tissue and therefore are difficult to remove with sufficient accuracy. This has improved survival without compromising risks . Tailored probes designed for special surgical procedures such as the transphenoidal approach  through the nose can guide operations in narrow approaches with limited abilities for direct visualization. With further developments in ultrasound technology, clinical results can continue to improve since good ultrasound image quality has direct consequences for the obtained clinical results, both in terms of resection grades  and for patient’s quality of life .
7.1.2. 3D Ultrasound in intracranial vascular surgery
It’s also possible to acquire power Doppler based 3D ultrasound data of the vascular tree in the target area. This can be useful in both tumor and vascular surgery. In tumor operations the objective is to avoid injury to the vessels caused by the surgical instruments. In vascular surgery power Doppler can be useful in surgical treatment of both aneurysms (figure 18G) and arteriovenous malformations (AVMs, figure 15). For surgical treatment of aneurysms this mode is most useful for evaluating the flow in distal vessels after clipping of the aneurysm. In addition, 3D power Doppler can be used to localize peripheral aneurysms and guide direct surgical approaches. For AVM surgery intra-operative 3D power Doppler has been found to be useful in localizing deep-seated AVMs, identifying feeders and draining veins and for resection control . Navigated display of 3D power Doppler based data can be used to identify and clip the larger feeders of AVMs in the initial phase of the operation, thus making it easier to perform the extirpation of the AVM.
Power Doppler based 3D ultrasound data are usually displayed in reddish color superimposed on the B-mode ultrasound slices, but the vessels are usually shown in a more optimal way using 3D rendering techniques. The power Doppler signal is often too intense and smeared out to give a sharp delineation of the small vessels. Robust acquisition of power Doppler based 3D ultrasound data of sufficient quality is essential for vessel-based shift correction and it’s important to increase the spatial resolution of such data in the coming years.
7.2. Ultrasound-based navigation in laparoscopic surgery
Open surgery is the gold standard for abdominal surgeries. But over the last few decades, there has been an increasing demand to shift from open surgery to a minimally invasive approach to make the intervention and the post-operative phase less traumatizing for the patient. Advantages of laparoscopic surgery include decreased morbidity, reduced costs for society (less hospital time and quicker recovery), and also improved long-term outcomes when compared to open surgery. During laparoscopy, the surgeons make use of a video camera for instrument guidance. However, the video laparoscope can only provide two-dimensional (2D) surface visualization of the abdominal cavity. Laparoscopic ultrasound (LUS) provides information beyond the surface of the organs, and was therefore introduced by Yamakawa and coworkers in 1958 . In 1991, Jakimowicz and Reuers introduced LUS scanning for examination of the biliary tree during laparoscopic cholecystectomy . It seemed that LUS gave valuable information and has since expanded in use with the increase in laparoscopic procedures. LUS is today applied in laparoscopy in numerous ways for screening, diagnostics and therapeutic purposes [132, 133]. Some examples of use are screening, like stone detection or identification of lymph nodes, diagnostics, like staging of disease or assessment of operability and resection range, and therapeutic, like resection guidance or guidance of radio frequency and cryoablation. Harms and coworkers were the first to integrate an electromagnetic (EM) tracking sensor into the tip of a conventional laparoscopic ultrasound probe  and this made it possible to combine LUS with navigation technology, solving some of the orientation problems experienced when using laparoscopic ultrasound. The combination of navigation technology and LUS is becoming an active field of research to further improve the safety, accuracy, and outcome of laparoscopic surgery.
Navigation, as explained earlier, is the combined use of tracking and imaging technology to provide a visualization of the position of the tip of a surgical instrument relative to a target and surrounding anatomy. Various display and visualizations methods of both instruments and the medical images can be used in laparoscopic surgery. Preoperative images are useful for planning as well as for guidance during the initial phase of the procedure as long as the target area is in the retroperitoneum . When preoperative images are registered to the patient, the surgeon is able to use navigation to plan the surgical pathway from the tip of the instrument to the target site inside the patient. Thus, navigation provides the intuitive correspondence between the patient (physical space), the images (image space that represent the patient) and the tracked surgical instruments. However, when the surgical procedure starts, tissue will shift and deform and preoperative data will no longer represent the true patient anatomy. LUS then makes it possible to update the map for guidance and acquire image data that display the true patient anatomy during surgery. Preoperative CT images will, however, still be useful for reference and overview as illustrated in figure 24, showing various display possibilities using LUS and navigation in laparoscopy. An example of simple overlay of tracked surgical tools onto a 3D volume rendering of computerized tomography (CT) images is shown in figure 24A. In this figure, we used the preoperative 3D CT images for initial in-the-OR planning of the procedure. The view direction of the volume was set by the view direction of the laparoscope. The LUS image could be displayed in the same scene, with an indication of the probe position in yellow. Furthermore, when 3D preoperative images are displayed together with 3D LUS, anatomic shifts can easily be visualized and measured, thereby providing updated information of the true patient anatomy to the surgical team as illustrated in figure 24C. This may improve the accuracy and precision of the procedure. Additionally, the tracked position of the LUS probe can be used to display the corresponding slice from a preoperative CT volume, providing improved overview of the position of the LUS image as shown in figure 24D. Having 3D LUS available, it is possible to display these data the same way as traditional orthogonal display of MR and CT volumes, as shown in figure 24E-G. Intraoperative augmented reality visualizations in combination with navigation technology could be valuable for the surgeons . A possible future development, useful for spotting the true position of lesions and vessels and hence detect anatomic shifts quickly, would be to introduce LUS data into such a multimodal display.
Intraoperative ultrasound systems are inexpensive, compact, mobile, and have no requirements for special facilities in the operating room (OR) compared to MRI or CT. Ultrasound image quality is continuously improving and for certain cases (e.g. liver) LUS could obtain image quality comparable to what is achieved in neurosurgery, as the probe is placed directly on the surface of the organ. In neurosurgery, the image quality of ultrasound has been demonstrated above. The most common LUS probe is a flexible 2- or 4-way array, linear or curved, with a frequency range of 5-10 MHz. Typical imaging depths are in the range 0-10 cm, but with 5MHz deeper imaging can be performed. The LUS transducers usually have a footprint of less than 10 mm wide to fit through trocars and 20-50 mm long. They can be manipulated at the shaft allowing real time images at user-controlled orientations and positions, depending only on the specific probe configuration. Figure 25 shows various configurations of LUS probes, while Table 2 provides an overview of currently available probes. Most LUS probes  can be sterilized .
|Vendor||Probe||Frequency||Type of probe (see Fig. 2)||Transducer length, scan angle, other|
|Aloka||UST-52109||3-7.5 MHz||A||10 mm, 90°|
|UST-5524-LAP||4-10 MHz||E||38 mm|
|UST-5526L-7.5||5-10 MHz||D||33 mm|
|UST-5536-7.5||5-10 MHz||E||38 mm|
|BK Medical||8666-RF||5-10 MHz||E||30 mm, Puncture and biopsy guide|
|Hitachi||EUP OL531||5-10 MHz||C||120°, Biopsy and therapy|
|Toshiba||PEF 704LA||5, 7.5, 10 MHz||E||34 mm|
|PVM 787LA||5, 7.5, 10 MHz||B||85°|
|Gore||Tetrad VersaPlane||7.5 MHz (center frequency)||E||56 mm|
|Philips / ATL||LAP L9-5||5-9 MHz||E||NA|
Being a relatively new area of research, it is interesting to note that the number of active research groups in the field of navigated laparoscopic ultrasound is approximately ten. Based on literature and almost two decades working with surgeons on developments for advanced laparoscopic surgery, a complete system designed for navigated LUS could be used according to the following clinical scenario:
The preoperative data are imported and reconstructed into 3D volumes; several structures and organs are segmented automatically (e.g. vessels from contrast CT scan) or semi-automatically (e.g. seed point set inside the tumor).
A quick plan is made from the visualization in the navigation system just prior to surgery, perhaps in the OR during other preparations.
Registration is performed without fiducials using a pointer (orientation of patient) and two landmarks for a rough first approximation.
Before mobilizing the target organ (e.g. the liver) a 3D LUS scan of major vessels near or around the tumor is performed.
The LUS images are reconstructed in 3D and an automatic vessel based registration (CT-to-ultrasound) is performed to fine tune the patient registration.
Augmented reality visualization, e.g. on/off overlay of preoperative data and LUS on the video laparoscope view is preformed as needed by the surgeons during the procedure
3D LUS scans are updated a few times during the procedure, while the real time 2D LUS image is available as either:
A full size image with a corresponding indication in a 3D CT rendering of its orientation and position, or
An overlay on the video laparoscope view with or without elements from the CT data (segmented structures for instance).
For rigid organ navigation, a single preoperative scan, highly accurate tracking (optical), and rigid surgical tools are sufficient to guide the procedure. However, for soft tissue navigation, additional tools are needed due to deformation and mobile organs in the abdominal cavity, resulting in more complex systems and additional devices in the OR. LUS can provide real time behind-the-surface information (tissue, blood flow, elasticity). When combined with advanced visualization techniques and preoperative images, LUS can enhance an augmented reality scene to include updated images of details, important for high precision surgery thus enhancing the perception for surgeons during minimal access therapy. LUS integrated with miniaturized tracking technology is likely to play an important role in guiding future laparoscopic surgery.
7.3. Other applications
One of the first, and still one of the most important applications of ultrasound imaging is in diagnostics of various heart conditions. The dynamic real-time imaging makes ultrasound the modality of choice for characterization of a moving organ such as the heart. Some examples of the use of echocardiography are evaluation of cardiovascular anomalies in fetuses and newborns , assessment of aortic stenosis , evaluation of the function of the valves and examination of the flow and function after heart attacks. These examples are purely diagnostic applications without any kind of intervention associated, but ultrasound has also been used for guidance in cardiac surgery. One example was presented by Wang et al. . They evaluated 129 patients who underwent robotic cardiac surgery. Transesophageal echocardiography was used for guidance of the cannula for peripheral cardiopulmonary bypass. Ultrasound imaging can potentially also be used for guidance in minimally invasive mitral valve repair on the beating heart .
Intra-operative guidance during endovascular procedures is usually performed with x-ray fluoroscopy. However, some investigators have reported the use of transabdominal ultrasound for guidance. Lie et al.  studied the use of 2D transabdominal ultrasound during endovascular procedures. They found that ultrasound could be useful for guiding the insertion of guidewires, and control the wire position before connecting the second graft limb to the main limb of bifurcated grafts. Kaspersen et al.  reported a feasibility study registering ultrasound to pre-operatve CT data. This may be useful for updating the CT data used for navigation and correct for breathing motion and deformation of the blood vessels during the procedure. With recent advances in ultrasound technology, we believe that real-time 3D ultrasound have potential for further advancing the accuracy in the insertion of stentgrafts, and in particular the placement of fenestrated stentgrafts. Specifically, it is easier to track the tip of guidewires in three dimensions, while simultaneously visualizing a focused area of the 3D anatomy in real-time. A systematic review by Malkawi et al  concluded that percutaneous endovascular repair was associated with fewer access related complications and reduced operative time. In a study by Arthurs et al , it was shown that use of ultrasound guided access significantly reduced access-related complications compared to percutanous access without ultrasound guidance. Successful ultrasound guidance in secondary interventions, for sealing endoleak after endovascular repair, has also been reported. Boks et al.  described transabdominal embolization using duplex ultrasound guidance, and Kasthuri et al.  used ultrasound for guiding percutaneous thrombin injection. Navigation of stentgrafts during endovascular procedures has also been demonstrated in patients using CT imaging . However, 3D or 4D ultrasound integrated with navigation technology for guidance of endovascular procedures has not yet been demonstrated in patients.
3D ultrasound has also been used to guide surgery of the spine. Kolstad et al  reported in 2006 a study, where spinal cord tumors were visualized using ultrasound imaging, and 3D ultrasound-guided tumor resection were performed using navigation technology. The technical application of integrating ultrasound and navigation seems feasible since it solves the orientation problem with conventional 2D ultrasound and may have the potential of improving functional outcome of spinal cord tumor surgery.
8. Ultrasound and non-invasive therapy
8.1. High Intensity Focused Ultrasound (HIFU)
High-intensity focused ultrasound (HIFU or FUS) has been known and developed for decades and can be applied to produce sharply delineated lesions in biological tissue (figure 26) [150-153]. The development of magnetic resonance (MR) thermometry enabled the thermal ablation progress to be monitored during sonication . MR-guided HIFU (MRgFUS) has been approved by the FDA for the symptomatic treatment of uterine fibroids since 2004 ; clinical trials have been reported for breast [156, 157] and brain [158, 159] therapy, as well as pain palliation in bone [160, 161]. The MRgFUS treatment of abdominal organs, such as the kidney, pancreas or liver, poses additional technological and clinical challenges. First, for most therapeutic applications within the human body, tissue displacement caused by respiration and/or the cardiac cycle must be considered, and can be assumed to be periodic in anaesthetized patients. However, this may not be the case for free-breathing patients. This movement in addition to drift due to gravity and the intestine and bowel movement is important to account for during sonication in order to achieve accurately located FUS with respect to the target (e.g. tumour in the liver). Secondly, the presence of the rib cage affects the HIFU treatment planning and set-up. The rib cage acts as an aberrator that might affect the focusing [162, 163] and on the other hand, due to the high value of the absorption coefficient of the bone , the overheating on the ribs can be quite significant. These two aspects are currently the main challenges in order to achieve MRgFUS in moving abdominal organs.
Ultrasound is an inexpensive, flexible, real-time imaging modality, with high temporal and spatial resolution, i.e. sub-millimeter spatial resolution inplane along the beam direction. However, it provides little contrast between normal tissue and FUS-treated tissue and so far ultrasound-based temperature monitoring has not been validated under a clinical scenario.
Motion of the abdominal organs is an important issue to be accounted for during FUS treatment, but also in other therapies like radiotherapy [165, 166]. The motion estimation is useful in delineating the target and organs at risk and also determining the dosage of treatment during therapeutic irradiation. Several techniques exist and are in development to handle abdominal organ motion during FUS. A straightforward approach is to use respiratory gating. However, respiratory gating generally increases treatment time, which has been demonstrated in controlled apnea on anesthetized pigs [167, 168]. Another approach is to employ repeated breath-holds and breathing feedback to ensure a reproducible liver position . De Senneville and coworkers  proposed a system that generates an atlas of motion fields during an initial learning phase based on magnitude data of temperature-sensitive gradient-recalled sequence acquisition. The motion field of the most similar image in the atlas is then used to correct the target position. Under the hypothesis of periodic motion, the focal point position for the next cycle is estimated. The method can only manage liver deformations caused by the periodic breathing cycle and is not capable of handling the non-rigid liver deformations, i.e. drift, caused by intestinal activity (peristalsis) or muscle relaxation . Although it is established that MR imaging can provide motion estimates with a high spatial resolution, it is difficult in practice to acquire online three-dimensional (3D) isotropic images because of technical limitations, spatial and temporal resolution trade-offs, and low signal-to-noise ratio associated with fast 3D acquisition sequences . In addition, the time duration between the actual target displacement and the availability of the motion information from MR data is not negligible . Hence, MR information-based real-time motion compensation generally compromises spatial resolution, geometric distortion and the precision of the MR thermometry , of which the latter is of crucial importance during MRgFUS.
A first attempt at ultrasound-based motion tracking during MRgFUS was reported in phantoms undergoing periodic and rigid motion of small amplitude . Continuous 1D ultrasound echo detection, along a direction parallel to the main axis of motion was used. This setup is not suitable for clinical applications as the external ultrasound imaging probe cannot send beams parallel to the axis of respiratory motion. Moreover, the local motion in the liver is spatially dependent and a 1D projection would not be sufficient. Truly simultaneous ultrasound and MR imaging has only been reported in literature recently [175-178]. Only one of these studies was targeted towards MRgFUS and moving abdominal organs sonication . They demonstrated in moving phantoms the feasibility of ultrasound-based 2D motion-compensated sonications integrated with reference free MR temperature monitoring, using a clinical ultrasound probe and a phased-array HIFU transducer . An overview of our own efforts for motion correction using 4D ultrasound can be found in section 4.3.
8.2. Ultrasound-induced drug delivery
Although diagnostic ultrasound is considered safe with no adverse effects, ultrasound can with high acoustic outputs induce significant bioeffects (e.g. HIFU) and these bioeffects are divided into thermal and mechanical effects. The thermal effect is related to energy absorption in the tissue where part of the mechanical wave energy is converted to thermal energy and hence results in an increase in tissue temperature. The mechanical effects are related to cavitation and to radiation forces. Radiation forces arise when part of the forward propagating wave is back-scattered or absorbed and result in a pushing force on the tissue along the direction of the forward propagating wave. Within fluids, such radiation forces can give rise to acoustic streaming. Cavitation is related to the oscillation and possible collapse of gas nuclei occurring naturally within the body or artificially introduced as contrast agent in the form of microbubbles. Oscillating gas bubbles will generate streaming currents in surrounding liquids and hence shear forces on nearby cells that potentially result in bioeffects. Collapsing gas bubbles can result in high local temperatures, release of free radicals, emitted shock waves and high velocity micro jets piercing into nearby cell membranes.
The indicated bioeffects can be utilized in ultrasound induced drug delivery. The general goal of encapsulated drug delivery and targeting is to improve the efficacy of drugs within the region of diseased tissue while reducing undesired side effects in the healthy tissues. As an example, with non-encapsulated conventional chemotherapy systemic toxicity limits the drug concentration that can be obtained within the tumor and hence the efficacy of the therapy. With focused ultrasound, it is possible to obtain release of encapsulated drugs and this release can be controlled both temporally and spatially.
Ultrasound energy deposition within a localized tissue region provides a potentially efficient way of releasing drugs encapsulated in thermally sensitive carriers [179-181] by inducing a temperature increase and in sonosensitive carriers [182-184] by inducing cavitation (figure 27). The thermal and especially the mechanical cavitation effects of ultrasound also provide ways of perturbing cell membranes and thus increasing their permeability for improved drug delivery. With the introduction of microbubbles administered intraveneously that will serve as cavitation nuclei, the threshold for cavitation is significantly reduced hence facilitating this effect for endothelial cells that are close to the administered microbubbles. This effect of increased cell membrane permeability has been investigated extensively in the brain where the blood-brain barrier acts as an effective barrier for delivery of more than 95% of the drugs that potentially could be interesting for treatment of diseases in the central nervous system [185, 186]. For blood clot dissolution the combined use of ultrasound, microbubbles and thrombolytic agents have been demonstrated in several clinical trials to result in faster clot dissolution without release of large amounts of potentially hazardous clot fragments [187, 188].
Ultrasound has been used for many years as a diagnostic and interventional imaging modality, and the use is increasing in a number of different clinical areas. It is often conceded that the image quality of ultrasound is inferior to that attainable with MR or CT, but the rapid development of new ultrasound technology (scanners, transducers, specialized probes, etc.) has resulted in significantly improved image quality and make ultrasound the modality of choice for several applications. Some of the obvious advantages being real-time imaging even for blood flow, portability, flexibility, safety and low cost. In addition, ultrasound images can be acquired in the coordinate system of a patient when combined with a tracking system without any need for registration. This makes surgical guidance based on intra-operative ultrasound highly accurate. The combination of several image modalities such as MR, CT and ultrasound registered to each other and to the patient make the interpretation of the individual images easier and enables the surgeon to take advantage of the complimentary information contained in each image. In this context, the ultrasound images provide real-time information in the region of interest, while MR and CT provide anatomical overview facilitating the interpretation of the ultrasound data. The use of contrast agents enhance the visualization of vessels and increase the number and types of lesions that can be detected using ultrasound. New technologies such as high-intensity focused ultrasound and the use of microbubbles for targeted drug delivery are examples of non-invasive therapeutic applications where ultrasound will play an increasingly important role in the future.