VR and AR simulator examples with respect to each educational field.
Virtual reality (VR) and augmented reality (AR) are two contemporary simulation models that are currently upgrading medical education. VR provides a 3D and dynamic view of structures and the ability of the user to interact with them. The recent technological advances in haptics, display systems, and motion detection allow the user to have a realistic and interactive experience, enabling VR to be ideal for training in hands-on procedures. Consequently, surgical and other interventional procedures are the main fields of application of VR. AR provides the ability of projecting virtual information and structures over physical objects, thus enhancing or altering the real environment. The integration of AR applications in the understanding of anatomical structures and physiological mechanisms seems to be beneficial. Studies have tried to demonstrate the validity and educational effect of many VR and AR applications, in many different areas, employed via various hardware platforms. Some of them even propose a curriculum that integrates these methods. This chapter provides a brief history of VR and AR in medicine, as well as the principles and standards of their function. Finally, the studies that show the effect of the implementation of these methods in different fields of medical training are summarized and presented.
- virtual reality
- augmented reality
- medical education
- advanced learning
Virtual reality (VR) and augmented reality (AR) are the current trends in medical education. VR is the virtual construction of an artificial world. The key element of VR is the high level of user’s immersion over the virtual environment, namely, the high level of structures’ fidelity, as well as the interaction of the user with them in a realistic manner. This has become recently available, as it requires high standards of certain technologies, including advanced haptic devices with force-feedback capabilities (bidirectional stimuli), high-resolution audio-visual effects, motion detection technology, and high-performance processing power to transmit, and processes all this information with near-zero latency.
The first virtual system in medicine was introduced in 1965 by Robert Mann, in order to facilitate a new training environment for orthopedic. In the late 1980s, the head-mounted display (HMD) was introduced as a wearable device for VR visualizations in medicine . The first pioneering applications in medical education were some on hands-on procedures that appeared over a decade later [2, 3]. In the 1990s, many adverse effects from the use of VR were reported. Nausea, dizziness, temporarily impaired vision, and lack in the sense of presence, even after 20 min of use, were some of them. The adverse effects were attributed to technical defects, such as the lag time and the inability of the human eye to fixate in-depth to “artificially distant” 3D objects [4, 5]. Although these effects are minor and subside quickly nowadays, the risk of learning inappropriate handling moves, when low-fidelity or badly simulated models are used and particularly under unsupervised teaching, still remains .
Although AR and VR share many technical aspects, AR differs from VR as its target is not to construct a fully artificial environment but to overlay computer-generated images onto images of the real world . Therefore, it uses machines that allow physical view of the surrounding environment to be visible but enhanced with virtual images. Tablets, mobile phones, AR glasses, and other optimized devices can be employed as hardware for running AR applications. Historically, the development of AR starts in the 1960s, but the term “augmented reality” was established in 1990. The increase in the use of AR came along with the technological advances that made it available and useful. Nowadays, AR is used widely in the clinical setting, providing extra information for the clinician during interventional procedures (CT/MRI guidance, visualization of paths), but it is also applied in the educational world. Anatomy, with the 3D visualization of hardly comprehensible structures, and physiology, with the representation of mechanisms in 4D (in space and time dimensions), are the main areas of interest .
2. Technical aspects
VR and AR come with certain hardware requirements in order to retain high standards of simulation. The computerized 3D pictures and audio have to be realistic and simulate both real and abstract structures. The motion of the user should be detected with high precision so that the visual field (size, shape, angle of objects) and auditory stimuli (volume, sound balance) can change accordingly. The user must be able to affect the virtual environment but also to be affected via haptic feedback stimuli. Haptic devices, such as joysticks, gloves, and other specialized tools, serve these haptic interaction needs. High-performance computer power is needed to process the huge amount of information produced, with low, unnoticeable latency. All these contribute to the key element of VR and AR: immersion of the user over the environment. Moreover, fast and reliable Internet connection is needed to support the latest trend in VR field, namely, the establishment of VR fora where users meet and interact with each other online, in a common virtual environment [1, 3, 9].
Many VR settings involve HMD. HMD is a wearable device that offers visualization of the constructed virtual reality in wide angle and does not permit the disturbance of the virtual experience by external visual stimuli. Although the first HMDs had a restricted resolution of 800 × 600 pixels and a narrow vision angle of 30°, the current HMDs display fully high-definition images, with angles that can reach 360°. Some of them are wireless and may incorporate position and motion detection systems for the eye and head movements. Although HMDs play a significant role in many VR settings, they are not mandatory for all medical applications. The simulation of many procedures, such as laparoscopic or endoscopic, requires the display of the virtual images on a computer screen . Sound quality is also important for a complete VR experience. Volume and balance adjustments according to the moves of the users are significant elements of the VR environment .
Input devices consist of all the devices that transmit the user’s stimuli to the VR system. Gloves, joysticks, or other specialized tools (laparoscopic or endoscopic instruments) are usually employed. A special subcategory of necessary input devices for VR is sensors and trackers. Sensors and trackers identify the position and the direction of the user in space as an input stimulus, allowing interaction. They employ various technologies such as lasers, infrared radiation, and mechanical detectors and can detect the properties of the whole body, the head, or even the subtle gestures and moves of the hands, ability that is extremely useful in medical procedures’ simulation. The recent technological improvement of these devices, which made the VR setting suitable for medical training, is the tactile-feedback ability. Adjusted force is exerted via the handling instruments, depending on the hand moves. Thus, the operator has a more realistic, bidirectional haptic experience when performing a virtual surgical process or other interventional processes .
Most AR hardware use glasses that project virtual 3D images onto the real environment. While Google Glass and Epson Smart Glasses were the pioneering devices, newer devices, such as Hololens (Microsoft), combine AR glasses with a system of tracking cameras and sensors. Tablets are an alternative platform suitable for AR applications. While the camera captures the physical world, virtual structures are added in the screen .
3. Quality control and educational impact
VR and AR simulators must possess quality standards so as to be suitable for medical education purposes. However, even if these standards are met, the educational benefit is still not guaranteed a priori. There are many published studies that try to test VR and AR simulators in specific educational settings (concerning different medical fields and stages of training) in order to examine their validity, the transferability of the taught skills to the real world, the acceleration of the learning curve, and the retention period of the skills. Following the determination of these parameters, establishing a curriculum that integrates this evidence in a productive and cost-efficient manner is an essential consequence.
High-resolution images, quality in sound, haptic input and feedback devices, and high processing power are the parameters that are required to create realistic VR/AR environment. Moreover, the structures of the organs and tissues must be of high fidelity, and the change in their shape, size, and angle of view must correspond completely to the user’s moves and handlings. Otherwise, there is a risk that the moves and skills learnt will be inappropriate in real situations . ISO criteria have also been established. They include initial assessment of the educational needs and establishment of a technical solution. This requires the in-depth knowledge of the procedures or mechanisms that will be simulated. After the concept generation, the setting must be validated, in order to examine if it meets the educational expectations. Moreover, technical experts, medical experts, representatives from end-user groups, human factor experts, and designers should collaborate and form an interdisciplinary team that will design, evaluate, and upgrade the VR/AR products [11, 12].
Furthermore, VR and AR systems meeting these standards have to undergo post-market validation studies, in order to examine if they provide the educational effect they are expected to. In terms of medical education, the studies examine the following types of validity: (1) face validity—usually the end users make subjective comments as feedback for the quality and realism of the setting. (2) Content validity: once again, it is subjectively defined by the users’ comments but with emphasis to the content of the simulation (procedures, fidelity of organs, etc.). (3) Construct validity: this employs objective statistical methods that aim to correlate the actual skill level of the participants with their performance in the VR setting. Usually, there are two or more groups that differ only to the level of skills. Their performance in the VR/AR setting is evaluated, either by external human evaluators or, if it is feasible, by the VR system itself. If there is a correlation, that means that the VR setting can effectively discriminate between novices and experienced users. This is extremely useful when using VR/AR settings as assessment tools for the progress of trainees, as they provide an objective evaluation that may not require the need of human evaluators. (4) Concurrent validity: another objective validity control that aims to correlate the performance in the VR/AR setting with other existing evaluation tools. This practically checks whether the VR/AR evaluation results point to the same direction with other evaluation tools that are considered as gold standards. (5) Predictive validity: the last but probably most useful objective validity control. Two homogenous cohorts of trainees are formed. After that, only one receives a specific VR/AR training, with predefined parameters (such as the simulator model, the procedure/topic learnt, the time, the frequency and the difficulty level of the procedure, and the presence of a supervisor). On the contrary, the other cohort receives training with differently optimized parameters (perhaps another VR/AR simulator or even a completely different simulation method) or does not receive training at all and is used as a control group. Subsequently, the skills of both cohorts are evaluated in real patients or other simulation settings that are considered as gold standards with proven discriminating ability. If a correlation exists, the VR setting can have an educational effect that can be indeed transferred to real patients. Some studies evaluate the effect of the training after a long time period to examine the retention period of the skills [8, 13].
If there is confirmed predictive validity of a VR/AR simulator for acquiring a specific skill set, then the next step is to wonder: “How much does the VR/AR training contribute to the acceleration of the acquired skills?” This is a more complex question that can be approached with learning curves. A learning curve is a graph that correlates the level of acquired skills with the amount of effort exerted (measured in time, number of trials or procedures performed). In general, it is desired that the learning curve has a high plateau, meaning that the final level of skills acquired with this method will be high, but also is steep, meaning that there is an early high acceleration of skills with little effort. If a trainee that has completed the training in the VR setting (namely, he/she has reached the plateau of the VR/AR learning curve) and then is able to receive training in other settings (e.g., other simulation settings or real patients) from a higher than the basic level, then we assume that the VR/AR training offers transferable skills and has “accelerated” the learning curve. The common characteristics of the learning curves on most VR/AR simulators are the low plateau, the transferability of the skills, and the steep shape. Low plateau means that the total level of skills that can be acquired is lower than the one with other education methods; thus, the VR/AR is usually appropriate for novices, but is not of any significant benefit for more experienced users. Fortunately, it usually has a steep shape and transferability properties, meaning that these skills are acquired quickly and then can be transferred to other education settings. In other words, VR/AR training compensates for some training period in the initial steps of other educational settings.
After these VR/AR setting-specific properties are evaluated, the next step is to define a curriculum that incorporates VR/AR methods in education. This curriculum must also take into account the cost of both VR/AR and alternative ways of training, as well as the patient risk for both settings. For example, if there is a high risk in training with real patients or high cost with animal-simulation models, a VR/AR simulator (with low plateau and steep learning curve) can compensate quickly for the initial steps, offering a safe, relatively inexpensive, and reproducible alternative choice that may provide transferrable skills.
As there are many validation and evaluating studies in literature, an overall profile of VR/AR applications can be formed. In general, it seems that VR offers an appropriate environment for training in hands-on procedures. As Table 1 shows, laparoscopic surgery, neurosurgery, catheterization techniques, and endoscopic procedures seem to be the main fields of VR applications. Novices are the most benefited trainees, while there is not a significant learning effect in experienced users. VR offers great diversity with only one setting. The educational setting can be reproducible with a variety of different optimization details, not restricted by the physical world . Although VR simulators are expensive to purchase, they can be used recurrently with low cost, require less staff than other methods (animal models or real patients), and guarantee no risk for the patient [8, 12]. However, there are certain disadvantages in their use. In the common case of crashing, the virtual experience is destroyed. From a technical point of view, they are still not ready to present a virtual environment with absolute fidelity, as if it was real. Last but not least, wrong handling can be learnt, particularly under unsupervised learning. The VR simulators can reconstruct a whole virtual surgery, specific procedural tasks, or even more abstract procedures .
|Fields of educational training||VR and AR simulator examples|
|Physiology and anatomy||Visible Human Project, Visible Korean Human, The Virtual Body, The Virtual Human Embryo, The Visible Human Server, AR glasses, mobile phone and tablet based AR applications|
|Open surgery||Virtual Reality Educational Surgical Tools|
|Laparoscopic surgery||MIST-VR, LaparoscopyVR™, LapMentor™, LapSim™, SINERGIA, Xitact LS500®, ProMIS®|
|Robotic surgery||RoSS™, DV-Trainer®, SEP Robot, da Vinci Skills Simulator (dVSS) ™|
|Oesophagogastroduodenoscopy, colonoscopy, ERCP||GI Mentor™, EndoVR™, Olympus Endo TS-1|
|Neurosurgery||NeuroVR (NeuroTouch, NeuroTouch Cranio)™, ImmersiveTouch®, RoboSim, Vascular Intervention Simulation Trainer®, EasyGuide Neuro, ANGIO Mentor™, VIVENDI, Dextroscope®, Anatomical Simulator for Pediatric Neurosurgery|
|Interventional cardiology and cardiothoracic surgery||ANGIO Mentor™, Vascular Intervention Simulation Trainer (VIST)®, Vimedix (equipped with Hololens) ™, Nakao Cardiac Model, Minimally Invasive Cardiac Surgery Simulator, dVSS™, EchoCom|
|Urology||URO Mentor™, University of Washington TURP Trainer, UROSim™, PelvicVisionTURP simulator, GreenLight laser simulator, Kansai HoLEP, ProMIS®|
|Orthopedics||ImmersiveTouch®, Phantom haptics interface®, Gaumard HAL S2001® and S3000® Mannequins, Novint Falcon®, Medtronic model, Arthro-VR®, Arthro MENTOR™, ArthroSIM, ArthroS™|
|Endovascular surgery||ANGIO Mentor™, Vascular Intervention Simulation Trainer (VIST)®, Cardio CT, SimSuite, Compass 2™|
|Obstetrics and gynecology||HystSim™, EssureSim™, AccuTouch (and newly version from CAE Healthcare), MIST-VR, LapSim™|
|ENT||OtoSim™, VOXEL-MAN (supports Phantom haptics), Ohio State University surgical simulator, Stanford Surgical Simulator, Mediseus, ImmersiveTouch, Endoscopic Sinus Surgery Simulator (supports Phantom haptics), Dextroscope®, dVSS ™|
|Ophthalmology||EyeSi®, MicrovisTouch™, PhacoVision®|
|Intubation and bronchoscopy||EndoVR Simulator™, BRONCH Mentor™, ORSIM®|
Moreover, many of the simulators have incorporated assessment tools of the user’s performance. They evaluate parameters like path distance, operation time, moves, and errors and then deliver quantified, timely, and objective metrics. If these simulators meet construct validity criteria, they can be used to objectively assess the trainee’s performance. However, if this ability is not supported, human experts can also play the role of the evaluator, and studies have also shown acceptable metrics .
Although AR finds many clinical applications in medicine, its implementation in medical education is not so extensive (see Table 1). However, tablets and mobile phones have AR applications that enrich traditional reading with pop-up videos, links, and interactive material . Moreover, in medical training there are AR applications that project traumas and other lesions onto healthy humans, aiming at training students and residents in the management of these situations. However, interventional procedures are not performed, and the AR applications are restricted in teaching the theoretical context of these subjects in an increased reality setting .
4. Applications in educational fields
4.1. Preclinical teaching
Anatomy and physiology are the classic paradigms that have the majority of VR/AR applications . A checkpoint in the development of VR/AR applications that study human structures and mechanisms is the construction of massive online databases that contain human images and information, mostly obtained with CT/MRI scanning. The first relevant project, called the Visible Human Project, was created by the University of Colorado in 1991. The male and female versions of the project contain over 7000 digital anatomical images and occupy over 50 gigabytes of space. The National Library of Medicine made this platform free and accessible [6, 16]. Other similar models followed, such as the Korean model Visible Korean Human, The Virtual Body, The Virtual Human Embryo, and The Visible Human Server. Anatomy teaching was enhanced with VR applications that obtained data from these databases. Similar effort was put to reconstruct intracellular organelles in physiology [2, 10]. Structures and mechanisms are digitally simulated and presented in four dimensions (space and time). This contemporary visualization can be called “in silico” biology. Examples include the dynamic simulation of neurons, membranes, and cardiovascular system parameters including heart rate, blood pressure, contractility, and vascular resistance . Anatomy also uses tablet-/smartphone-based AR applications that project extensive information, visual 3D structures, and links onto the traditional pages of anatomy textbooks. Recently, other hardware platforms, such as the Hololens glasses by Microsoft, started to support relevant applications . An application allows the interactive, 3D exploration of the human brain, reconstructed with MRI data. Hand gestures allow the deformation, fly-by view, and other interactions with the 3D model to reveal hidden organs .
VR and AR find most of their applications in surgery and particularly in laparoscopic surgery training. Endoscopic procedures and neurosurgery are also popular. Fidelity and realism are extremely important factors in these fields, not only for the comfort of the user but also for guaranteeing that no improper handling will be learnt and that the actual stressful conditions will be recreated. VR/AR simulators demonstrate certain advantages, such the minimal cost per use, the absence of ethical issues, and the safety compared to training on actual patients. Moreover, greater diversity and complexity in the procedures can be achieved. However, the cost of purchasing and maintaining is high, and also despite the rapid advances in VR/AR technologies, the realism still fails in representing the operating room settings with high fidelity .
4.2.1. Laparoscopic surgery
The current VR laparoscopic simulators are characterized as “hybrid” as they have real instruments and a virtual operating field, projected on a screen or other display devices. They provide haptic feedback to users according to their handlings, and most of them have assessment systems that measure parameters such as the time to complete a task, the errors made during surgery, and the surgeon’s economy of movements. They can have both “task trainer” modules, for training in more simple and abstract handlings, and also “complete operation” modules that simulate the whole operating procedure . They seem to benefit the novices, especially when intensive feedback by the instructor is applied . The following simulators are among the most popular:
Minimally Invasive Surgical Trainer-Virtual Reality (MIST-VR, Wolfson Centre and VR Solutions): it provides a 3D environment for practicing abstract tasks and moves with a cylinder and a ball. It can be adjusted to six difficulty levels, and it can teach basic laparoscopic skills like suturing and knot tying. It is a low-fidelity simulator that does not support force-feedback function [18, 20]. It has demonstrated construct validity but controversial transferability of skills [21, 22].
LaparoscopyVR™ (LapVR™, Immersion): this simulator supports haptic hardware for force-feedback function. Studies show that inexperienced surgeons acquire skills that can be transferred in real procedures. LapVR™ system allows both individual and team training and offers training in six basic handlings with adjustable levels of difficulty: camera navigation, using the hook electrode, cutting, clipping, suturing, and knot tying. It also has a module for laparoscopic cholecystectomy with 18 alternative cases and 3 levels of difficulty. It also contains a module for ectopic pregnancy, tubal occlusion, and adnexa’s pathology [2, 20].
LapMentor™ (3D Systems, formerly Simbionix): it targets both novices and experienced surgeons, teaching from basic laparoscopic skills to complete laparoscopic operations. It also has an incorporated assessment system for providing metrics to the user. It is characterized by high fidelity and the ability to provide haptic feedback. The modules include basic laparoscopic skills, suturing, laparoscopic cholecystectomy, ventral hernia, gastric bypass, and gynecology cases. Information on handling tissue, handling tools, time, and movement efficiency are obtained during the simulation and are processed to produce metrics [2, 20]. It has demonstrated construct validity, predictive validity, and transferability of technical skills, especially when combined with supervision by the instructors [21, 23].
LapSim™ (Surgical Science): this system includes basic skill modules, anastomosis and suture scenarios, and laparoscopic cholecystectomy scenarios, and one case is dedicated to gynecology. It is a high-fidelity simulator with the ability of transferring functions between instructors and institutions. It has shown good construct and predictive validity and transferability of some skills [2, 18, 20, 21].
The sustainability of the skills obtained with VR simulators is supported by few studies with small samples and a short follow-up period . Other VR/AR models are SINERGIA, Xitact LS500® (Xitact), and ProMIS® (Haptica) [13, 20, 21].
The main application of AR in laparoscopic surgical training is telementoring: the supervisor can teach the trainee by indicating the proper surgical moves, paths, and handlings on the AR screen. These pieces of information are displayed to the trainee and guide them . However, research has shown that this might also distract the user. A study proposes displaying this information on an additional sub-monitor, rather than on a single head-up display .
4.2.2. Robotic surgery
RoSS™ (Simulated Surgical Systems) and DV-Trainer® (Mimic Technologies) are robotic surgery VR trainers that have shown face and content validity. DV-Trainer®, along with SEP Robot (SimSurgery) and da Vinci Skills Simulator™ (dVSS™, Intuitive Surgical), provides metrics about the trainees’ performance with construct validity. RoSS™ is the only one that incorporates whole procedural tasks .
As the current robotic surgery systems do not rely on tactile feedback, their simulators are also freed from this need. However, for educational purposes an alteration in color in the surgical field exists, depending on the force that the users apply. Training in robotic surgery with VR simulators seems to provide better and transferable surgical skills, as shown by some studies [28, 29]. Novices are found to be particularly benefited from training in robotic simulators .
4.2.3. Open surgery
There is a limited variety in open surgery VR simulators and a lack of validation studies, probably because the complexity and the hands-on nature (without the involvement of machine systems) of open surgery operations do not provide an easily simulated environment. Virtual Reality Educational Surgical Tools (VREST) has developed an open surgery simulator for training and assessment. It consists of two haptic devices with haptic feedback enabled and a system that represents 3D images. The software allows the optimization of the scenarios from the teacher. The trainees decide what instruments to use and are assessed by the machine. The first module built on the VREST platform was an inguinal hernia repair according to Lichtenstein. Another similar simulator was developed by the Imperial College London, which has shown construct validity .
4.3. Endoscopic procedures
4.3.1. Oesophagogastroduodenoscopy (OGD)
GI Mentor™ (3D Systems, formerly Simbionix) is the only validated VR simulator for upper endoscopy training. Although beneficial for novices, it is of limited benefit for the experienced users. Procedure time, time to reach specific landmarks, intubation time, movement techniques, procedural success rates, and other patient-related outcomes are evaluated in metrics. Studies show that trainees have a significantly lower overall procedure time and a significantly improved technical accuracy, compared to controls .
More VR simulators are available for colonoscopy training. The evidence that supports the use of VR trainers in the early stages of training is well established. The repetitions required to reach the plateau of VR training are 7–10, depending on the study. After this checkpoint there must be a switch to traditional endoscopy training. The combination of these two methods has been found more beneficial than either of the two alone. The skills acquired with VR training were maintained for at least 9 months . GI Mentor™, EndoVR™ (CAE Healthcare, the old AccuTouch®, Immersion) and Olympus Endo TS-1 (Olympus) are among the simulators that have been studied and have showed construct and predictive validity [21, 33]. GI Mentor™ has been chosen by SAGES as the platform for the Fundamentals of Endoscopic Surgery™ (FES™) examination .
4.3.3. Endoscopic retrograde cholangiopancreatography (ERCP)
GI Mentor™ (3D Systems, formerly Simbionix) is the only validated simulator for ERCP among six devices in total. It has been found to be beneficial in acquiring transferable skills and to have the potential to be included in training programs, although it lacks in terms of realism, usefulness, and performance compared to bench model trainers [32, 35]. EndoVR™ is another popular simulator with ERCP modules available.
Neurosurgery training is a field where VR and AR simulators are usually found. Such simulators must be comfortable enough to wear for many hours; extremely realistic, so that they represent effectively the detailed neurosurgery structures; and affordable . They demonstrate several advantages when compared to physical models (animal or bench models): they have a lower cost in use, there is no limit in the repetitions, and they offer a great sense of variety and diversity in the simulated cases. However, the resolution and realism of the constructed world are still issues to be resolved .
NeuroVR™ (CAE Healthcare): a neurosurgical training platform that simulates microdissections, tumor aspiration, and hemostasis. It was first introduced as NeuroTouch, a platform designed from over 50 experts from the National Research Council Canada . It provides complex computer-generated metrics in 13 categories; thus it is suitable for use as assessment tool. As there seems to be a plateau in acquired skills, the educational benefit may be limited to early learners. Other specialized variations of the simulator also exist, such as the NeuroTouch Cranio, a simulator for brain tumor resection. The trainer provides haptic feedback and measures the trainee’s performance .
ImmersiveTouch® (ImmersiveTouch) is an AR simulator used for training in many surgical disciplines, including spine surgery, neurosurgery, ENT surgery, and ophthalmology . It consists of two physical instruments, which can simulate various surgical instruments, and a system that projects the virtual surgical field onto the hands and the instruments of the user in an interactive way. It was found a valid training method for thoracic pedicle screw placement (face and predictive validity) and clipping aneurysms (face validity) but also for percutaneous trigeminal rhizotomy (construct validity) [13, 38]. Moreover, it had positive learning effect and can accelerate the learning curve in many neurosurgical procedures, such as ventriculostomy, vertebroplasty, and finding the anatomical landmarks for various interventional techniques [39, 40].
Various other VR/AR simulators exist. RoboSim, Vascular Intervention Simulation Trainer® (VIST®, Mentice), EasyGuide Neuro (Philips Medical Systems), ANGIO Mentor™ (3D Systems, formerly Simbionix), VIVENDI (University of Tübingen, Germany), Dextroscope® (Bracco AMT), and Anatomical Simulator for Pediatric Neurosurgery (ASPN, Pro Delphus) are commercially or academically developed systems that simulate solely neurosurgical procedures or incorporate also other additional modules [38, 41, 42].
4.5. Interventional cardiology and cardiothoracic surgery
ANGIO Mentor™ (3D Systems, formerly Simbionix): it provides a virtual field created from CT/MRI images of real patients and data from cardiac intervention procedures (balloon angioplasty, coronary stenting). Standard guidewires, catheters, balloons, stents, and similar devices are used with the system. Monitoring the patient’s vital signs and electrocardiographic changes is available. ANGIO Mentor™ is a robust simulator used in many endovascular procedures, such as carotid stenting and renal, iliac, and other vascular interventions .
Vascular Intervention Simulation Trainer® (VIST®, Mentice): it is similar to the ANGIO Mentor™ containing a library of virtual patients, tools, and procedures. These support tactile feedback and generate fluoroscopic images. It also supports a variety of other endovascular procedures, along with an electrophysiology module for training in pacemaker lead placement .
Vimedix™ (CAE Healthcare): it is an AR system for training in echocardiography. It consists of a mannequin and a transducer transthoracic or transesophageal echocardiogram. It has displayed face validity. Recently, CAE Healthcare has equipped it with Hololens in order to provide the ability to view the images with glasses, unrestricted from the dimensions of the screen . EchoCom is another echocardiography AR trainer but for neonates. It has shown face, content, and construct validity .
Nakao Cardiac Model is a VR simulator for training in surgical palpitation of the beating heart. It supports haptic interaction. It does not include metrics tools and does not offer variability. The Minimally Invasive Cardiac Surgery Simulator by Berlage and colleagues is another VR simulator that virtually projects beating heart, ribs, and chest surface based on the patient’s unique CT images. It is not interactive but supports variability of the simulated environments. A VR Lobectomy Simulator was developed by Solomon and colleagues. It includes video-assisted thoracoscopic surgery resection, lobectomy; it is controlled with haptic stimuli, and it has incorporated metrics tools that track performance, measure surgical times and errors, and ask questions .
dVSS™: cardiothoracic surgery also employs VR robotic training. A study found that the group previously trained in dVSS™ performed better in robotic internal thoracic artery harvest and mitral valve annuloplasty than the control group .
URO Mentor™ (3D Systems, formerly Simbionix): a VR simulator for training in many urologic fields. It has shown educational benefit in cystoscopy and ureteroscopy . It also supports several other modules, including essential skills, stone extraction, stone lithotripsy, cutting strictures, and taking biopsy modules. Face and content validity have been confirmed, and it is considered as a realistic and useful training model for endourological procedures.
University of Washington TURP Trainer (University of Washington): this VR trainer was designed for training in transurethral resection of the prostate (TURP). It has been extensively validated and shown face and construct validity. This is the only model to have shown concurrent validity with other assessment tools in studies with large numbers of participants [47, 48].
UROSim™ (VirtaMed): a commercial TURP VR simulator that has recurrently shown face, content, and construct validity. It simulates endoscopic movement and allows users to perform resection, coagulation, and also complete TURP operations. It can be adjusted to different levels of difficulty and has advanced metrics tools [47, 48].
Other TURP simulators using VR are the PelvicVisionTURP simulator, the GreenLight laser simulator (GreenLight SIM, American Medical Systems Inc.), and the Kansai HoLEP simulator. All these demonstrate face and construct validity [47, 48].
A module employing Phantom® haptics interface (Geomagic) was developed for digital rectal examination. However further research is required. The University of Grenoble developed in 2013 a VR simulator for transrectal ultrasound-guided prostate biopsy. Since then, several systems have been constructed . Moreover, the ProMIS® laparoscopic VR trainer has displayed construct validity in urologic procedures . Studies have compared the educational effect of VR trainers with that of bench models and found no difference in the early steps. However, VR trainers are far more expensive than bench models but also offer greater diversity . AR has limited applications in urology training, as it is only used to teach theoretically the procedural steps. A relevant application based on Google Glass trains urology residents in how to place an inflatable penile prosthesis .
The ImmersiveTouch® simulator is used in a variety of spine surgery procedures, including thoracic pedicle screw placement and percutaneous spinal needle placement. Phantom® haptics interface is used for spinal needle insertion. Gaumard HAL S2001® and HAL S3000® Mannequins and Novint Falcon® (Novint Technologies) are systems for vertebroplasty simulation training. A model from Medtronic has also been used in pedicle screw placement and placement of lateral mass screws .
Three VR models are validated and commercially available for training in arthroscopy. Both Arthro MENTOR™ (3D Systems, formerly Simbionix) which was the evolution of ARTHRO-VR®(GMV) and ArthroSIM have displayed face, content, and construct validity [53, 54, 55, 56]. ArthroS™ (Virtamed) has demonstrated greater face validity than Arthro MENTOR™ and ArthroSIM in a head-to-head comparison study .
4.8. Endovascular surgery
Usually, endovascular VR trainers have metric tools that evaluate the procedural time, the volume of the contrast used, and the time for fluoroscopy. They provide plenty of modules including angioplasty and stenting of coronary, renal, iliac, and femoral vessels. They also seem to benefit the inexperienced users, and the skills acquired seem to be transferable .
ANGIO Mentor™ (3D Systems, formerly Simbionix): it contains 23 different endovascular procedures and over 150 patient scenarios. It simulates the environment of endovascular procedures performed under fluoroscopy in the cath lab, interventional suite, or operating room. It supports haptic feedback, and it can benefit trainees of different disciplines, including interventional cardiology, interventional radiology, vascular surgery, electrophysiology, neuro surgery, and interventional neuroradiology and trauma. It has shown better procedure time, fluoroscopy time, and contrast volume in many procedures, such as carotid artery stenting, renal artery stenting, and cerebral artery angioplasty [42, 58].
Vascular Intervention Simulation Trainer® (VIST®, Mentice): as already mentioned it contains many training modules. It also supports haptic feedback and has many difficulty levels. Studies have shown improvement of users’ performance in many endovascular procedures, including carotid artery, cerebral artery, renal artery, iliofemoral artery, aorta, and superficial femoral artery procedures [42, 58].
Cardio CT (Shina System), SimSuite (Medical Simulation Corporation), and the newer Compass™ 2 (Medical Simulation Corporation) are other VR simulators that have not been so extensively studied .
4.9. Obstetrics and gynecology
HystSim™ and the newer EssureSim™ (both by VirtaMed) are VR simulators for training in hysteroscopic procedures. Studies have attributed face, construct, and predictive validity to them and have also shown that the skills taught are retained for at least 2 weeks [59, 60]. A model from CAE Healthcare (the old AccuTouch, Immersion) has been used in hysteroscopy training and has shown construct validity and the ability to improve performance equally with box trainers .
MIST-VR might be suitable for training in gynecology training, as it has shown positive results in other fields . LapSim™ has also been used in salpingectomy training and shown construct validity . AR has nonsignificant applications in gynecology training.
OtoSim™ (OtoSim Inc.): it is used in otoscopy training, offering a wide range of realistic images of the tympanic membrane and of various middle ear lesions. It was found to have good face validity .
VOXEL-MAN surgical simulator (Hamburg, Germany): it is a computer-based interface that can be used for temporal bone drilling and mastoidectomy. It supports Phantom® haptic devices for tactile feedback and 3D visual interfaces. It has demonstrated face, content, and construct validity [62, 63, 64]. Other similar simulators that display validity are the Ohio State University surgical simulator, the Stanford Surgical Simulator (Stanford BioRobotics lab), and the Mediseus surgical simulator (CSIRO/University of Melbourne Temporal Bone Simulator) [62, 63, 64].
Endoscopic Sinus Surgery Simulator (ES3, Lockheed Martin): it is used in sinus surgery, enabling haptic feedback with Phantom®. It collects and analyzes performance data providing performance metrics. It consists of three components: a workstation simulation platform (Silicon Graphics Inc.), a haptic controller, and an electromechanical human interaction platform with replica of an endoscope, surgical tools, and a mannequin. It is among the most widely validated sinus simulators, demonstrating content, concurrent, construct, and predictive validity and a significant retention of the acquired skills [62, 64]. Dextroscope® is also used in sinus surgery and has shown predictive validity .
dVSS™ is also used in robotic ENT surgery training. Tonsillectomy, fine-needle biopsy of the thyroid, myringotomy, nasal endoscopy, and cricothyroidotomy VR simulators have also been developed [62, 65].
Cataract and vitreoretinal surgery is the main field of VR training in ophthalmology. EyeSi® simulator (VRmagic) is the most commonly used system. It includes two pedals (one for the microscope and one for phacoemulsification/vitrectomy/infusion and aspiration modes), a physical head and eye, and a microscope, all connected to a computer. It does not support tactile feedback. However, it demonstrates construct and predictive validity. The educational effect is restricted to novices [66, 67, 68]. MicrovisTouch™ (ImmersiveTouch) and PhacoVision® (Melerit Medical) are additional models with only the MicrovisTouch™ supporting haptic feedback . AR is still of limited use, although a study finds educational benefit when AR is applied in ophthalmoscopy teaching .
4.12. Other fields
Pediatrics and emergency medicine are promising fields for integration of VR training. Rigorous work is being done to create realistic virtual humans and virtual pediatric models. Moreover, highly realistic virtual environments that recreate massive destruction situations are being developed for use in emergency medicine [70, 71].
AR has been used in intubation training. More modern systems, such as BRONCH Mentor™ (3D Systems, formerly Simbionix) and EndoVR Simulator™, include intubation modules and have shown positive educational effect .
Ultrasonography simulators have been used for assessing trainees. ScanTrainer® (MedaPhor). It consists of a haptic force-feedback-enabled device and a computer with a touchscreen that visualizes ultrasonography images according to the user’s movements .
EndoVR Simulator™, BRONCH Mentor™, and ORSIM® (Airway Simulation Limited) are some of the systems used for training in bronchoscopy . Most of them have shown positive learning effects. BRONCH Mentor™ offers a bigger variety of pathologies and clinical situations and a more realistic environment. ORSIM® is a small, portable simulator with a physical bronchoscope and an interface based on regular laptops .
Some AR applications that indicate the landmarks for central venous catheterization have been developed. An ultrasound probe locates the vein and a microprojector projects it onto the skin. A study has found positive learning effect, when used in landmark finding training .
5. Conclusion and future perspectives
Most of the educational applications of VR and AR seem to have construct and predictive validity, with the acquired skills to be transferable to real situations. However, the credibility of several of these studied might be questionable. First of all, many of them are not randomized studies, with cohorts of different characteristics and inadequate number of participants (less than 30 in most studies). Moreover, there are few studies for each simulator, and there are no similar standards in their design, so that they could be summarized and directly compared. In addition, many of the studies referring to specific simulators become quickly outdated as they do not take into account the simulators’ continuous upgrades. More randomized control trials, comparing the effect of VR training against no training, other simulation-based training, or different VR training systems, are needed. The samples must be larger to strengthen the results and the designs of different studies similar [5, 7, 8, 21]. Moreover, the extent of the decay of the skills over time must be elucidated. When these properties, along with the cost factors, are clarified, then we can examine the way that VR and AR can be officially incorporated in medical education curricula.
However, the published literature suggests a positive educational impact. VR/AR training displays certain advantages toward other simulation techniques. Although expensive to buy, VR/AR simulators provide a relatively costless opportunity for reproducible training under various environments and difficulty levels. Moreover, they do not raise ethical issues, compared with other animal and living tissue simulation models. They provide immersion for the user and the ability to perform complete procedures, in contrast with partial task trainers. Multiple studies have shown a favorable impact of VR/AR trainers on inexperienced trainees, and we can intuitively assume that they are technically evolving in high pace as the technology progresses. Future improvements could include the integration of olfactory stimuli. Odors can be used as diagnostic tools or even to recreate stressful conditions (e.g., in a combat or in the operating room) with greater realism [1, 8]. Medical informatics is also an evolving field. Medical data will be visualized more clearly and impressively with VR/AR technology. Users will be able to dive into statistical plots and reports, watch them in 4D (in 3D space and time), manipulate them, and “wander” around them. Although significant progress has been made, there is still a need for more processing power, higher resolution, better design of the scenarios, and more advanced haptic devices in order to achieve highly realistic environments [31, 76].