Extreme Telesurgery

1. Introduction

The technological development of the last decades resulted in the rise of entirely new paradigms in healthcare. Within interventional medicine, first Minimally Invasive Surgery (MIS) and later robot-assisted surgery redefined the standards of clinical care. The concept of telemedicine dates back to the early 1970s, and in the late ‘80s, the idea of surgical robotics was born on the principle to provide active telepresence to surgeons. With the help of mechatronic devices physicians were first able to affect remote patients with the Green telepresence system in 1991. Soon after, many new research projects were initiated, creating a set of instruments for telesurgery. Visionary surgeons created networks for telesurgical patient care, demonstrated trans-continental surgery and performed procedures in weightlessness. The U.S. Army has always been interested in this technology for the battlefield, and currently the Telemedicine and Advanced Technology Research Center (TATRC) enforces research to test and extend the reach of remote healthcare. However, due to the high business risk, not many surgical robots succeeded to pass clinical trials, and barely some become profitable.

Beyond intercontinental operations, probably the most extreme field of application is medical support of long duration space missions. With a possible foundation of an extra-planetary human outpost either on the Moon or on Mars, space agencies are carefully looking for effective and affordable solutions for life-support and medical care. Teleoperated surgical robots have the potential to shape the future of extreme healthcare.

Besides the apparent advantages, there are some serious challenges of robotic healthcare that must be dealt with. The primary difficulty with teleoperation over large distances or beyond Earth orbit is communication lag time. Even in the case of intercontinental teleoperation—assuming the usage of commercial communication lines—latency can be in the order of several hundred milliseconds. While military satellite networks show better performance, these are not accessible for regular use. Surgery robot control communication protocols must be robust and false-tolerant, while advanced virtualization and augmented reality techniques should help the human operators to better adapt to the special challenges. A novel virtual reality based, extended surgical environment control concept is proposed. To meet safety standards and requirements in space, a three-layered architecture is recommended to provide the highest quality of telepresence with the provisional exploration missions. Today’s extreme telesurgery concept may well find a way to common civil applications for the benefit of many patients.

2. Concept of telemedicine and telesurgery

2.1. Advanced medical technology

Telemedicine allows physicians to treat patients geologically separated from themselves. Pilot networks have been installed and tested in the second half of the 20^th century, and the first intercontinental procedures were conducted in the 1990s (Rosser et al., 2007).

Telemedicine in general can be broken down to three main categories based on the timing and synchrony of the connection. Store-and-forward telemedicine means there is only one way communication at a time, the remote physician evaluate medical information offline, and sends those back to the original site at another time. Next, remote monitoring enables medical professionals to collect information about patients from a distance with different modality sensors. Finally, interactive telemedicine services provide real-time communication between the two sites, which might be extended with different forms of interactions, achieving real telepresence.

According to the functionality, three levels of telepresence can be defined within telemedicine based on the actual capabilities of the physician at the remote location. If instant and unlimited access is provided to the medical site, that is real-time teleoperation (or telesurgery, in the case of surgical procedure). Telementoring means the use of telecommunications technology—including the internet—to support and guide locally operating medics. Consultancy telemedicine (or telehealth consultancy) requires only limited access to the remote site, and as a result, the distant group cannot use real-time services or information updates.

The advantages of telemedicine are various, in the case of short-distance operations, the technology involved can mean great added value, such as an externally controlled tool holder or surgical robot (Herman 2005). In long-distance telementoring, the time/cost effectiveness and the provided higher level of medical care are the most important benefits, while in extreme telemedicine, such as space exploration it may be the only available form of adequate medical aid.

Figure 1.

Integration of different modality feedback information to the concept of telesurgery. Interaction is only possible through a system of sensors and human-machine interfaces. Modified from (Hager et al., 2008).

Beyond the possibility to observe the remote site, the quality of telepresence has always been paramount for the surgeons to be able to perform a procedure. The availability of different modalities combined, such as 2D/3D visual, tactile/haptic, acoustic, etc. has been proved to dramatically increase human performance. Figure 1 shows the integration of different modalities to the control diagram of telesurgery concept. Currently, the dominant form of sensory feedback is visual, as that provides highest density of information. The resolution of video cameras has been increasing in the past years, and currently full HD resolution is available with most systems, accompanied with a high fidelity 3D stereoscopic view. Although haptic feedback was provided with the first robot prototypes, the commercially available systems miss this modality due to the complexity (and additional cost) of the hardware and the challenges to provide life-like tactile feedback to the surgeon.

2.2. History of telehealth

There have been several experiments conducted in the past two decades to verify the usability of remote health care paradigms. Due to the fact that surgeons navigate based on a camera image, telementoring techniques are well applicable in laparoscopy, MIS. It is considered to be one of the most important breakthroughs in medicine in the past decades (Ballantyne, 2007). In 1997, laparoscopic colectomy and laparoscopic Nissen fundoplications were the first procedures performed with the aid of professional telementoring, from over 8 km distance (Rosser et al., 1997). The same group performed the first international telementoring between the John Hopkins Medical Institute (Baltimore, MD) and Innsbruck, Austria and Bangkok, Thailand (Lee et al., 1998). In 1999, they telementored from Maryland five laparoscopic hernia repairs, performed on board of the USS Abraham Lincoln aircraft carrier in California (Cubano et al. 1999). Later, several intercontinental telementorig experiments have been performed, mainly from the USA to Italy, France, Singapore, Nepal and Brazil (Fabrizio et al., 2000).

The U.S. Department of Defense (DoD) got interested in the feasibility of telesurgery even earlier; aimed to develop a system that allows the combat surgeons to perform life saving operations on wounded soldiers from a safe distance (Satava, 1995). The idea of robotic support in space dates back to the early ‘70s, proposed in a study for the National Aeronautics and Space Administration (NASA) to provide surgical care for astronauts with remote controlled robots (Alexander, 1973). This is particularly desirable, as the specific, high level medical education of the flight surgeons might be impossible to achieve. Proficiency in MIS, laparoscopic surgery requires extreme amount of practice, and maintenance of skills is only possible with continuous training.

3. Robotic telesurgery

In most of the cases, mechatronic systems and cameras are the remote hands and eyes of the surgeon, and therefore key elements of the operation. Out of the 370 international surgical robotic projects listed in the Medical Robotic Database (MeRoDa, 2009), there are several dozens with the capability of teleoperation.

In general, robots can be involved in medical procedures with differnet level of autonomy (Nathoo et al. 2005). Many of the developed systems only serve as a robust tool holding equipment, once directed to the desired position. Systems that are able to perform fully automated procedures—such as CT-based biopsy or drilling—are called autonomous, or supervisory controlled. (A human supervisor would always be present to intervene if deviation occurs compared to the surgical plan.) This can be combined with the classic tools of image guided surgery, once the robot is registered to the patient.

When the robot is entirely remote-controlled, and the surgeon is absolutely in charge of the motion of the robot, we call it a teleoperated system. These complex systems typically consist of three parts; one or more slave manipulators, a master controller and a sensory (e.g. vision) system providing feedback to the user. Based on the gathered visual (and haptic, acoustic, etc.) information, the surgeon guides the arm by moving the controller and closely watching its effect.

By modifying the teleoperation control paradigm, we can introduce cooperative (also called compliant) control. It means that the surgeon is directly giving the control signals to the machine through a force sensor, performing hands-on operation.

3.1. First telesurgery systems

Funded by the DoD, the first prototype of telesurgery robot was developed at Stanford Research International (SRI) (Menlo Park, CA) called the Green Telepresence System (Green et al., 1991). It was assembled by 1991, primarily aimed for open surgery. The idea to use it with MIS came with the rapid spread of laparoscopic technique. A series of ex-vivo and in-vivo trials were performed by 1995 (Bowersox et al., 1996).

NASA Jet Propulsion Laboratory (JPL) (Pasadena, CA) also started to develop a system in the early times, and by 1993 they created the RAMS (Robot-Assisted Microsurgery System), targeting high-precision ophthalmic procedures (Schenker et al., 1995).

Based on the experience at SRI and NASA, the Defense Advanced Research Projects Agency (DARPA) of DoD initiated the Trauma Pod project in 1994. The main goal was to “enhance battlefield casualty care by developing autonomous and semi-autonomous mobile platforms through the integration of tele-robotic and robotic medical systems. The initial phase has successfully automated functions typically performed by the scrub nurse and circulating nurse… The next phase of the program will develop methods for autonomous airway control and intravenous access... Finally, these systems will be miniaturized and incorporated into a tactical platform capable of operating in a battlefield or mass casualty environment.” (Trauma Pod, 2009). The robots developed with the help of DARPA have already been tested under extreme circumstances, in weightlessness and at NASA Aquarius underwater habitat.

3.2. Commercialized systems

The most well known commercialized robots are the da Vinci Surgical System from Intuitive Surgical Inc. (Sunnyvale, CA) and the discontinued Zeus from Computer Motion Inc. (Santa Barbara, CA). While these robots inherited the structure and features that make them capable of performing telesurgical operations, most commonly they are used for on-site surgery. Their primary advantage is easing the complexity of laparoscopic procedures, providing better visualization, control and ergonomics to the surgeon, and higher precision to the patient.

Presently, the market leader (and the only available) complete teleoperated robot is the da Vinci, created with roughly 500M USD investment. The patient side consists of two or three tendon-driven, 6+1 degree of freedom (DOF) slave manipulators. These are designed with a Remote Center of Motion (RCM) kinematics, resulting in an inherent safety regarding the stability of the entry port. The camera holder arm allows 3 DOF navigation controlled with the same master interface. The system provides high quality 3D vision with stereo-endoscopes, adjustable tremor filtering (~6 Hz) and motion scaling (1:1 – 1:5).

In 1995, Intuitive licensed technology from NASA, SRI, IBM and several universities, and by 1997, the first prototype—Lenny—was developed for animal trials. Next, Mona was made for the very first human trials involving vascular and gynaecological procedures in the Saint-BlasiusHospital (Dendermonde, Belgium) in March 1997. As the system was originally intended for cardio-vascular (beating-heart) surgery, specific clinical trials were performed in Paris and Leipzig in May 1998 (Ballantyne et al., 2004). Based on the initial experience, the market-ready version of the robot (named da Vinci honouring the great inventor) got advanced control and ergonomic features compared to the Mona. Final clinical tests began in 1999, and the U.S. Food and Drug Administration (FDA) approved the system for general laparoscopic surgery (gallbladder, gastroesophageal reflux and gynecologic surgery) in July 2000, followed by many other approvals. Once the system was on the market, Intuitive continued perfecting it, and the second generation—the da Vinci S—was released in 2006 (Figure 2). The latest version, the da Vinci Si became available in April 2009 with improved full HD camera system, advanced ergonomic features, and most importantly, the possibility to use two consoles for assisted surgery.

Figure 2.

Master controllers and the patient side manipulators of the new da Vinci Si surgical system. (Photo: Intuitive Surgical Inc.)

Currently, there are more than 1300 da Vinci units around the world, ¾ of them in the U.S. The number of procedures performed is well over 300,000, the most successful application of the robot became prostatectomy. Around 70% of all radical prostate removal procedures were performed robotically in the U.S. in 2008.

The concept of the da Vinci theoretically allows remote teleoperation, but that has not been the primary focus of Intuitive. The previous versions of the robot used a proprietary short-distance communication protocol through optic fibre to connect the master and the slave, while the latest Si facilitates further displacement of the two units. In 2005, TATRC presented collaborative telerobotic surgery on animals with modified da Vinci consoles, being able to overtake a master controller with a remote one through public internet connection (Flynn, 2005). During the experiment, the average roundtrip latency was 500 ms from Denver to Sunnyvale, which was disturbing for the physicians.

Another similar robot was the Zeus Telesurgical System developed by Computer Motion Inc. (Santa Barbara, CA). It was based on the AESOP (Automated Endoscopic System for Optimal Positioning) camera holder arm (FDA approved in December 1993). The Zeus received FDA clearance in 2001. The Zeus was controlled in master-slave setup, and used UDP/IP (User Datagram Protocol over Internet Protocol) for communication. This facilitated various experimental telesurgery procedures as described later (Kumar & Marescaux, 2008). After long litigation with Intuitive over mutual intellectually property violations, the whole company was bought by Intuitive, and first the production, then the support of the Zeus system was suspended.

3.3. Light-weight prototypes

Although some systems never got commercialized, they were created with the aim to facilitate extreme telesurgery. NASA JPL and MicroDexterity Systems Inc. (Albuquerque, NM) developed the RAMS (Robot-Assisted Micro-Surgery) system (Das et al., 1998). The RAMS consists of two 6 DOF arms, equipped with 6 DOF tip-force sensors, providing haptic feedback to the operator (Figure 3). It used the concept of telesurgery for control; however, the operator sat right next to the slave arms. The robot was originally aimed for ophthalmic procedures, especially for laser retina surgery. It is capable of 1:100 scaling (achieving 10 micron accuracy), tremor filtering (8-14 Hz) and eye tracking. Currently the prototype rests idle at JPL, as the project was discontinued.

Figure 3.

The RAMS robot developed at NASA JPL in laboratory trials and in-vivo animal tests in 1998. (Photo: NASA)

Doctors and scientists at the BioRobotics Lab., University of Washington (Seattle, WA) have developed a portable surgical robot that can be a compromised solution to install on spacecrafts with its 22 kg overall mass (Rosen & Hannaford, 2006). The DARPA supported robot—called Raven—works along the same principle as the da Vinci. It has two articulated, tendon driven arms, each holding a stainless steel shaft for different surgical tools. It can easily be assembled even by non-engineers, and its communication links have been designed for long distance remote-control. The system has participated in multiple field tests, and now several units are being built for large scale clinical trials (Lum et al., 2009).

Realizing the importance of a light, but stiff structure, SRI started to develop the M7 in 1998 (Figure 3.). The system weights only 15 kg, but able to exert significant forces compared to its size. It is equipped with two 7 DOF arms, motion scaling (1:10), tremor filtering and haptic feedback. The end-effectors can be changed very rapidly, and even laser tissue welding tool can be mounted. The controller has been designed to operate under extremely different atmospheric conditions, e.g. it only contains solid-state memory drives. The software of the M7 has been updated lately to better suit the requirements of teleoperation and communication via Ethernet link. The M7 performed the world’s first automated ultrasound guided tumor biopsy in 2007.

The German Aerospace Center (DLR) Institute of Robotics and Mechatronics (Wessling, Germany) has already built several generations of light-weight robotic arms for ground and space applications. They have also taken part in many telerobotic space experiments in the past decades. The KineMedic and the most recent MIRO 7 DOF surgical robots are considered for real teleoperation—even in extreme locations—as one arm is only 10 kg and capably of handling 30 N payload with high accuracy (Hagn et al., 2008).

Small scale, in-body robots offer great advantages, as they are always remote controlled, opening the possibility of spatial displacement of the physician from the patient. Engineers at the University of Nebraska (Lincoln, NE) together with the physicians of the local Medical Center developed a special mobile in-vivo wheeled robot for biopsy (Rentschler et al., 2006). Equipped with a camera, the coin-sized robot can enter the abdominal cavity through one small incision and move teleoperated around the organs. The robot is able to traverse the abdominal organs without causing any damage, therefore reduces patient trauma. More recently, the group has developed various swallowable robots that can be controlled with external magnets.

The CRIM group at Scuola Superiore Sant'Anna (Pisa, Italy) leads a European Union FP7 founded international research collaboration to develop tethered, partially autonomous robots to perform surgery in the endolumen (Menciassi & Dario, 2009). Another EU project—Vector—aims for the creation of effective capsule robots for local surgical procedures throughout the GI tract (Eirik et al., 2009).

Figure 4.

The Zeus robot during the first intercontinental surgery, the colecystectomy was performed on the patient in Strasbourg from New York. (Photo: IRCAD)

4. Remarkable experiments

4.1. Long distance telesurgery

The Zeus robot proved to be a solid platform to test and experiment different telesurgical scenarios. Between 1994 and 2003 the French Institut de Recherche contre les Cancers de l'Appareil Digestif (IRCAD) (Strasbourg, France) and Computer Motion Inc. worked together in several experiments to learn about the feasibility of long distance telesurgery and effects of latency, signal quality degradation. After six porcine surgeries, the first transatlantic human procedure—the Lindbergh operation—was performed with a Zeus 7. September, 2001 (Marescaux et al., 2002). The surgeons were controlling the robot from New York, while the patient laid 7,000 km away in Strasbourg (Figure 4). Based on previous research (Fabrizio et al., 2000), it was estimated that the time delay between the master consol and the robot should be less than 330 ms to perform the operation safely, while above 700 ms, the operator may have real difficulties controlling the Zeus. A high quality, dedicated 10 Mbps ATM fibre optic link was provided by France Telecom, transmitting not just the control signals and video feedback, but also servicing the video conferencing facilities, and an average of 155 ms communication lag time was experienced. Out of that roughly 85 ms was the communication lag through the transmission, and 70 ms the coding and decoding of the video signals.

Figure 5.

Network Route Director (NRD) designed for CMAS by Bell research in 2002 to support the telesurgical network in Canada.

In Canada, the world’s first regular telerobotic surgical service network was build and managed routinely between the Centre for Minimal Access Surgery (CMAS), a McMaster University Centre (Hamilton, Ontario) and a community hospital in North Bay some 400 km away, using the Zeus robot (Anvari, 2005). The average latency recorded was about 150 ms using commercial high-speed internet link Virtual Private Network (VPN) protocol (Figure 5). CMAS performed 22 telerobotic cases with North Bay General Hospital and over 35 telementoring cases with North Bay General Hospital, Ontario and the Complexe Hospitalier La Sagamie, Quebec. The network was eater extended to include more centers in Canada. While the FDA only permitted the single case of telesurgery of the Lindbergh operation in the USA, Canadian health authorities cleared the methodology for routine procedures.

A remotely-controlled catheter guiding device guided by a robot was used in Milan in 2006 to automatically perform heart ablation, initiated and supervised by a group of professionals from Boston, MA. The robot uses high magnetic fields to direct the catheter to the desired location, taking advantage of the pre-operative CT scans of the patient and real time electromagnetic navigation. Initial trials were performed on 40 patients before the telesurgical experiment took place. The novelty of the system was that it could create the surgical plan on its own based on an anatomical atlas including 10,000 patients (Pappone et al., 2006).

4.1. Underwater trials

NASA has conducted several experiments to examine the effect of latency on human performance in the case of telesurgery and telementoring. The NASA Extreme Environment Mission Operations (NEEMO) take place on the world’s only permanent undersea laboratory, Aquarius. It operates a few kilometers away from Key Largo in the Florida Keys National Marine Sanctuary, 19 meters below the sea surface. A special buoy provides connections for electricity, life support and communication, and a shore-based control center supports the habitat and the crew. Twelve NEEMO projects have been organized since 2001, and three were focusing on teleoperation recently.

The 7^th NEEMO project took place in October 2004. The mission objectives included a series of simulated medical procedures with an AESOP robot, using teleoperation and telementoring (Thirsk et al., 2007). The four crew members (one with surgical experience, one physician without significant experience and two aquanauts without any medical background) had to perform five test conditions: ultrasonic examination of abdominal organs and structures, ultrasonic-guided abscess drainage, repair of vascular injury, cystoscopy, renal stone removal and laparoscopic cholecystectomy. The AESOP was controlled from the CMAS (Ontario, Canada) 2,500 km away. A Multi-protocol Label Switching (MPLS) VPN was established, with a minimum bandwidth of 5 Mbps. The signal delay was tuned between 100 ms and 2 s to observe the effects of latency. High latency resulted in extreme degradation of performance: a single knot tying took 10 minutes to accomplish. The results showed that the non-trained crew members were also able to perform satisfyingly by exactly following the guidance of the skilled telementor. They even outperformed the non-surgeon physician, but fell behind the trained surgeon. Scientists also compared effectiveness of the telementoring and the quality of teleoperated robotic procedures. Even though the teleoperation got slightly higher grades, it took a lot more time to complete (Doarn et al., 2009).

During the 9^th NEEMO in April 2006 the crew had to assemble and install an M7 robot, and perform real-time abdominal surgery on a patient simulator. Throughout the procedure a microwave satellite connection was used, and time delay went up to 3 s to mimic the Moon-Earth communication links. Each of the four astronauts had to train at least 2 hours with the wheeled in-vivo robots designed at the University of Nebraska. In another experiment, pre-established two-way telecom links were used for telementoring. The crew had to prove the effectiveness of telemedicine through the assessment and diagnosis of extremity injuries and surgical management of fractures. The effects of fatigue and different stressors on the human crew’s performance in extreme environments were also measured. Latency was set up to 750 ms in these experiments. The significant performance degradation of the microwave connection was noticed during stormy weather, causing a jitter in latency up to 1 s (Doarn et al., 2009).

Figure 6.

The Raven robot performing FLS tasks on board of NASA Aquarius in Florida, while guided by a surgeon from Seattle. (Photo: University of Washington)

The 12^th NEEMO project ran in May 2007, and one of its primary goals was to measure the feasibility of telesurgery with the Raven and the M7 robots (Figure 6). NASA sent a flight surgeon, two astronauts and a physician into the ocean. Suing operations were performed on a phantom in simulated zero gravity environment to measure the capabilities of surgeons controlling the robots from Seattle. This time the Aquarius was connected to the mainland through a Spectra 5.4 GHz Wireless Bridge, allowing for a minimum of 30 Mbps bandwidth, and average latency of 70 ms. The HaiVision CODEC was used for video compression giving very good quality, but also introduced significant latency, up to 1 sec.

A group of three professionals guided the robot using commercial internet connection, and the communication lag time was increased till up to 1 s. Several simple tasks were performed, as part of the Fundamentals of Laparoscopic Surgery (FLS). The M7 demonstrated the first image-guided autonomous surgery (using a portable ultra sound system). It was live broadcasted at the American Telemedicine Conference 2007 (Nashville, TN). The M7 was able to insert the needle into a tissue phantom by itself.

4.2. Surgery in space

To facilitate exploration missions beyond Earth, space agencies have always been pushing for more advanced telehealth concepts. Surgical experiments (laparotomy and celiotomy on rabbits) were first reported from Russian cosmonauts in 1967. The first survival procedure was performed on STS-90 Neurolab mission on rats in 1998 (Campbell el al., 2001). The world’s first human operation was a cyst removal from a patient’s arm, on board of the European Space Agency’s Airbus A-300 Zero-G aircraft. The plane performed 25 parabola curves, providing 20–25 s of weightlessness every time (New Scientist Space & AFP, 2006). ESA had plans to perform teleoperation in 2008 with a robot controlled through satellite connection, but the mission was postponed. NASA had its first zero gravity robotic surgery experiment in late September 2007 (Kamler, 2007). On a DC-9 hyperbolic aircraft suturing tasks were performed with the M7 (Figure 7). The performance of classical and teleoperated robotic knob tying was measured. Both the master and the slave devices were equipped with acceleration compensators, otherwise it would have been almost impossible to succeed with the tasks. The results showed that humans can still better adapt to extreme environments, however, advanced robotic solutions do not fall far behind.

Figure 7.

The M7 on board of a NASA parabolic flight, and the robot performing autonomous ultrasound-guided tissue biopsy. (Photo: NASA, SRI)

5. Challenges in teleoperation

Effectiveness of surgical care heavily relies on the prompt delivery of treatment, and extreme teleoperation serves this principle. Beyond the obvious challenges of reduced medical equipment, constrained resources and probably limited experience of the on-site staff, several technical difficulties arise with extreme telesurgery.

Significant delay in the sensor feedback can totally distract the surgeon and cause serious safety hazard, as examined by different research groups. Engineering methods have been developed to overcome the difficulties originating from the absence of communication infrastructure, unpredictable propagation condition changes and hardware failures. The U.S. Robotics roadmap points to robotic telesurgery as a major focus of research in order to improve quality of care (Christensen et al., 2009). It calls for engineering solutions to ensure natural interaction between the human operator and the remote robot through specific patient models, from whole-body level to tissue characteristics. This would allow for advanced off-site surgical planning, automated guidance and also for realistic training opportunity.

5.1. Effect of latency

The primary difficulty with teleoperation over large distances or low quality network infrastructure is the communication lag time. The continuous development of the internet backbone infrastructure has resulted in a significant reduction of typical delays. Using commercial services, delay may be around 85 ms, across the United States, and lag time might be anywhere from 20-400 ms world-wide. Due to the Transmission Control Protocol (TCP/IP) and the routing algorithms latency can vary over trials, and this further degrades user performance.

Satellite based internet connections can use a fleet of Low or Medium Earth Orbit (LEO/MEO) satellites such as the commercial constellations of Orbcom, Globalstar or Iridium. Typical roundtrip delays are 40 ms, but the bandwidth is only 64 kbps per channel. Currently developing O3b Networks (scheduled for deployment late 2010) would provide 1 Gbps with approximately 125 ms lag time. Geosynchronous satellites provide higher latency due to their 36,000 km altitude above the equator. Round trip latency is 540-700 ms typically. Understandably, designated military satellites can provide a lot faster communication channel, the minimum latency per satellite hop is expected to be 4.3–7.8 ms one way (Berlocher, 2009). Despite the recent improvement in surface line speed, satellite communication has the potential to overcome ground lines primarily in speed, with a reasonable quality of service and availability.

Beyond Earth orbit radio and microwave frequency signals propagate at almost the speed of light in space, however already in the range of long distance manned space missions, several minutes of latency can be experienced. Planet Mars orbits 56,000,000 km to 399,000,000 km from Earth which means a 6.5 to 44 minutes of delay in transmission. In addition, for about two weeks every synodic period, direct communication can be blocked, as the Sun stays in between Earth and Mars, direct.

5.2. Adaptation to latency

Most humans are capable of adapting to sensory feedback latency up 500 ms (Anvari, 2004) and some experiments suggest that individuals might be able to perform tasks even with a consistent 1000 ms delay (Lum et al., 2008). Researchers showed that varying latency significantly reduces the operators’ performance both with robotic telesurgery and virtual reality (VR) applications (Thomson et al., 1999). It is advisable to use the consistent, maximum latency of system to achieve performance continuity. General approach to handle latency is to slow down the surgeons’ movements, allowing time for the visual feedback to confirm the intended move. However, in extreme cases this prevents the effective work and firm reaction to unexpected events. The on-site medical assistant can also help with imminent moves and minor tasks (Hanly et al., 2009).

While keeping the stereo vision system’s two video channels synchronized is crucial for high quality performance, other modalities might be useful delivered earlier. By reflecting e.g. forces sooner back to the operator (bypassing the time video coding/decoding takes), they can improve their ability to compensate for latency. The compression and decompression of the video stream can be reduced by the use of novel CODECs and state-of–the-art computing hardware, but it still takes significant time, generally 100-700 ms.

Approaching from the control theory point of view, several strategies have been developed to overcome the difficulties on the master side. One solution is to realize communication delays as force reflection for the human operator (Nohmi, 2003). It feels as if the remote manipulator was controlled through a virtually coupled spring, applying adequate forces. This method can be well used to provide information on the remote manipulator’s movement while there is no real feedback due to the latency, however it is not precise enough (and may alter the surgeons decisions by occasionally providing unreal forces) to approve it for surgical applications.

When humans are adapting to asynchronous sensory feedback, they tend to create a virtual representation of the remote site to predict the consequences of their moves and project the tools ahead. A similar concept has been proposed for virtual reality based simulators (Frank, 1986). This has been successfully extended with sensory fusion for VR control (Kadavasal & Oliver, 2007), where the operator controls the model of the remote system in a virtual environment, and the lag time between the simulated and the real setup is handled by the environment model. We propose to apply this concept to telesurgery, where robot commands could be sent to a simulator that predicts the dynamic state of the virtual surgical site, including the robot’s position, and acceleration along with the patient’s body and tissue properties and reactions (Figure 8). The optimal control of the robot is calculated autonomously with high frequency on the patient site, while the Intelligent Surgical Interpreter provides an interface to the master controller. Sensory updates are processed by the simulators at the surgeon site, and the virtual environment is updated according to the new measurements. This framework allows the smooth integration of different modalities, but require a very precise 3D model of the patient, gained from pre-operative MRI, CT and PET scanning. The models might be continuously updated through intra-operative imaging techniques. A variety of operations and possible outcomes could be simulated and analyzed before the actual surgery takes place, reducing the risk of complications.

Figure 8.

The concept of virtual reality extended control of surgical robots in order to locally deal with the disturbing effects of latency.

5.3. Communication protocol

Further difficulties may arise with the data protocol of the robots that links the master consol and the slave arms. Presently, the majority of telepresence systems communicate through Transmission Control Protocol (TCP) that is used along with the Internet Protocol (IP) to send data in the form of individual units–packets. Another common type is the User Datagram Protocol (UDP), a connectionless protocol that—like the TCP—runs on top of IP networks. UDP/IP provides very few error recovery services, offering instead a direct way to send and receive datagrams. Asynchronous Transfer Mode (ATM/AAL1), which encodes data traffic into small fixed sized packets is also frequently used, but lacks advanced security services, just as the UDP/IP. In the case of communication breakdown, the recovery time may be critical, therefore redesigned gateway architecture should be added to allow TCP transfers to survive a long duration blockage.

The Zeus and the da Vinci were designed to discard each packet that has any sort of internal error; they do not correct bit-level errors. If several packets are lost, or there is a breakdown, the robots suspend their operation. To meet the special communication requirements through satellites, the Space Communications Protocol Standards (SCPS) was developed and tested by DoD and NASA (Wang & Horan, 2009). SCPS uses similar architecture to TCP/IP, but it is more effective in handling latency created by long distance transmissions and the noise associated with wireless links. The SCPS exists as a full ISO standard, and also meets the U.S. Military Standards.

To ensure superior quality visual and tactile feedback, high sampling rate must be used on the patient site (app. 1 ms). Along with the high definition video feedback this has a significant bandwidth demand. Under regular circumstances a 10 Mbps connection is already suitable for teleoperation, however in the case of high definition, multimodal equipment, a 40 Mbps two-channel link would be required (Spearing & Regan, 2005). This may not cause much problem on the International Space Station (ISS) that was equipped with a 150 Mbps connection in 2005, but in the case of a Mars mission, NASA only plans to develop a 5 Mbps connection as a part of the new space communication architecture relying on the reconfigurable Space Telecommunications Radio System (STRS), and upgrade it to 20 Mbps by 2020 (Reinhart & Johnson, 2008).

6. Long duration space mission support

While advanced internet based communication networks enables effective telesurgery all over the Earth (with the discomfort of latency), serious technological problems arise in the case of long-haul space exploration missions. Space medicine and health care have always been a critical issue for aerospace agencies and several comprehensive studies reckon with the technological capabilities and challenges (Campbell & Billica, 2008).

Despite the current financial difficulties, the overall goal and future direction in exploration is towards the continuous human presence on the Moon and on Mars. The previously proposed MARS mission by NASA would have required the support of a 6 member crew for 18 months. The team’s designed healthcare module is a subsystem within the prefabricated habitat providing 15.9 m² space, and 2,500 kg payload (Drake, 2009). The presently used communication systems have serious limitations concerning the achievable minimum communication lag time, maximum bandwidth and robustness. Based on recent experiments illustrated above, we have a better understanding of the effects and drawbacks of extreme long distance telesmedicine that also apply for telementoring. As long as a new level of machine automation is not reached, it seems inevitable to have a flight surgeon on board of the spacecraft, to adapt to any unforeseeable event. Flight surgeons should receive special training for a better command over computer integrated surgical technology to be provided on board. The rest of the crew should also undergo comprehensive medical training to attain the skills required to monitor any surgical procedure, and to interact in the case of immediate danger. It is also important to practice the skills with the surgery robot throughout the mission, even if no accident occurs.

Based on the pre-described conditions, difficulties and system requirements, a three-layered mission architecture is proposed to achieve the highest degree of performance possible, by combining robotic and human surgery (Figure 9). Depending on the physical distance between the space ship and the ground control centre, different telepresence technologies may end up with the best result. Basically, with the accession of the latency, real-time control strategies and communication techniques’ effectiveness decreases significantly. By adaptively switching, different levels of surgical service can be provided throughout the mission.

Figure 9.

Concept of telehealth support to provide maximum level of available medical care to astronauts during long distance exploration missions.

Mainly within the range of 380,000 km (app. the Earth-Moon average distance), regular telesurgery techniques can be used in space to provide medical support in the case of emergency. Leaving the Earth orbit, special control strategies have to be applied, to extend the feasibility of telesurgery up to a maximum of 2 s of delay. With robot assisted surgery, a shared control approach should be followed, integrating high-fidelity automated functions into the robot, to extend the capabilities of the human surgeon. For example, to automatically follow the movements of the organs (the beating heart and breathing lung), the robots should be equipped with adequate visual and force sensors, and the precise control algorithms have to be built in on the slave side. Assuming the minimum expertise of the crew to be equivalent of Emergency Medical Technician in these missions, telepresence systems should only rely on assistance and support from the crew. Successful methods have been developed recently to provide automatic movement compensation (Ortmaier et al., 2006). This concept could be most beneficial for long duration on-orbit missions, primarily on board of the ISS. Presently, there is no other option than the immediate evacuation of the affected astronaut, which poses bigger health risk and huge costs.

Flying further from the Earth and having reached the limits of pseudo real-time communication, the procedures should be performed by the flight surgeon, or by any other trained astronaut, under the telementoring guidance of the master surgeons on the ground. Telementoring requires exchange of still images, motion video, digital image editing, voice conferencing, electronic chat and data file transfer. As showed by the NASA undersea experiments (Thirsk et al, 2007), telementoring can be an effective alternative to direct teleoperation, allowing the controller to perform the tasks based on the visual and voice commands of the ground centre. With adequate training and practice, the astronauts with a basic surgical training should be able to successfully accomplish complete procedures. As tested on USS Abraham Lincoln carrier in 1998, 9.6 28.8 kbps connection can already be enough to transfer images at 2-4 fps speed (Cubano et al. 1999). Telementoring may extend the boundaries of telepresence, as it can still be effective with a 50-70 s delay (within the range of app. 10,000,000 km). Upon this phase, the built-in semi-automatic functions of the surgical robot may have a significant role to improve the overall quality of the surgery. Motion scaling, adaptive tremor filtering, the automated following of the organ’s movement, automated suturing could significantly improve the less practiced crew members’ performance on one hand, while special security measures could also be applied. On the other hand, the setting of virtual boundaries for the robot, tool limitations and speed constraints may reduce the risk of accidents. Astronauts should also benefit from advanced imaging technologies (e.g.: accurately matched anatomic atlases for better navigation around the organs.) With the use of augmented reality systems, live and virtual images can be merged in real time to make the operation even smoother. Surgical malpractice can be reduced significantly by applying safe zones (virtual fixtures) that allow the robot to operate only within the predefined area (Lin et al., 2006). The safeguard teleoperation concept developed originally for mobile space robots could be useful in surgery. The robot can autonomously perform the routine tasks with the real time supervision of a human, however in the case of any malfunction or sudden events, the human operator can take over the control.

There is no crisp boundary between the application of telementoring and consultancy telemedicine. Above a certain signal delay, the terrestrial medical support crew will not be able to react in time to unforeseeable events during the procedure; the flight surgeon will be left alone for longer and longer periods. Above around one minute of delay, it is inconvenient and impractical for the crew to wait for the guidance of the ground after every step, and in some cases, it would endanger the success of the operation. For these missions, the physician must be trained to conduct the operation and make decisions on its own.

It was shown during the NEEMO missions that the general performance of telesurgery is higher than of the telemedicine, and a team of experts may do better than flight surgeons. Therefore depending on the feasibility, telesurgery should be preferred over telementoring.

With a complete remote controlled robot on board of a vessel, high quality surgical assistance could be provided for long duration space missions. Besides, the astronauts would be able to conduct several material and life science experiments and research, using the robots for micro-manipulations, and assist post-operative interventions. Advanced system design will help to deal with signal delays and to extend human capabilities.

7. Conclusion

In the case of telepresence surgery, intercontinental procedures, space missions and other highly complex exploration tasks and telehealth may mean the only sustainable solution to provide complete medical support. Telemedicine gains increasing importance as technology provides new and more effective means of telepresence to gain access to distant places. Medical robotic systems are already able to widely extend the human surgeons’ capabilities, to provide high fidelity support for a great variety of procedures. The new generation of robots will lead to a breakthrough in general healthcare.

Extensive effort has been put into the exploration of this new domain of telemedicine, as reviewed in this chapter. To meet the special requirements, completely new and innovative structures are built for long distance telesurgery. The most extreme example of teleoperation is in space applications. Long duration missions and venturing beyond Earth orbit mean new technological challenges, especially considering life-support systems. By integrating cutting-edge mechatronic equipment, semi-autonomous robots could ensure the medical support for a 2-3-year-long mission through telehealth technology. With the combination of real-time teleoperation, telementoring and consultancy telemedicine, the overall mission safety can be greatly increased, assuming a limited level of autonomy to improve the usability. The next few years of research will lead to breakthroughs in the interdisciplinary field of Computer-Integrated Surgery, opening further opportunities for telesurgical application. Robots and smart tools controlled through communication networks are soon to answer the special needs of remote patients.

Acknowledgments

The research was supported by the National Office for Research and Technology (NKTH), Hungarian National Scientific Research Foundation grants OTKA T69055, CK80316.