Innovative Technologies for Medical Education

Abstract

This chapter aims to assess the current practices of anatomy education technology and provides future directions for medical education. It begins by presenting a historical synopsis of the current paradigms for anatomy learning followed by listing their limitations. Then, it focuses on several innovative educational technologies, which have been introduced over the past years to enhance the learning. These include E-learning, mobile apps, and mixed reality. The chapter concludes by highlighting future directions and addressing the barriers to fully integrating the technologies in the medical curriculum. As new technologies continue to arise, this process-oriented understanding and outcome-based expectations of educational technology should be embraced. With this view, educational technology should be valued in terms of how well the technological process informs and facilitates learning, and the acquisition and maintenance of clinical expertise.

1. Introduction

Medicine is the science and practice of the diagnosis, treatment, and prevention of disease, and most medical information is about our human body. Medical information is employed in different scenarios, such as education, training, diagnosis, surgery, etc. As the computing technologies develop, more information is digitized from the physical world, and then researchers work on how to process and show them back to the user, enabling the user to perceive and interact with the information naturally and effectively. Perception and interaction with different media and objects are the fundamental human activities, and they are user specific [1].

Anatomical education is an important content of every curriculum and starts already very early in school, to form a good understanding of the body and improve the general population's health awareness [2]. With the advent of the plethora of exciting technological advancements there should be no reason not to include these for the creation of new education paradigms for medical learning. As such, this chapter presents an overview of the fascinating novel research, which is being undertaken in this area.

2. State-of-art

A clinician's knowledge is collected through many years of medical practice. In various settings, the knowledge will be transferred to medical students through medical education and to the general population through public education. For example, in the surgical setting, medical imaging data are collected for diagnosis and navigation. Good communication between the patient and the surgeon is very important for making the patient comply with the surgical procedure. The same communication plays a vital role for patients who perform rehabilitation exercises after surgery. In all of these scenarios, key players act different roles to perceive and interact with medical information. In this section, we discuss the state-of-art and challenges involved in medical education, public education, and the communication to prepare for surgery and rehabilitation.

2.1. Medical education

Though medical education has been existent for centuries, we begin from Abraham Flexner's report in 1910 [3]. Flexner visited 155 medical schools that existed in North America in 1909, and he identified four major problems during medical education: (1) lack of standardization, (2) lack of integration, (3) lack of inquiry, and (4) identity formation. The report had a huge impact and shaped modern medical education, where patient care, teaching, and research are combined. However, academic hospitals do not spend enough time on teaching due to enormous pressure to publish, as well as, for economic reasons [4]. Another issue is that research is focusing on very small subtopics, which do not relate to medical education [5]. On the other hand, only little research is done for teaching. An example is gross anatomy where most topics are already known and only little novel research exists. In today's medical schools students are required to understand both functional and spatial context of human anatomy.

Traditional methodologies: Traditional medical education learning is classified into three categories: cadaver, model, and textbook. Although technology has advanced significantly in the last decades, school education still mostly uses the same methods to convey anatomical knowledge [2]. Typically, the information is collected in printed books like anatomy atlases, displayed in the form of charts and diagrams. Those diagrams provide a simple and well-known method to illustrate form and appearance of organs, having the advantage that the user is accustomed to such methods of display. However, there exist several downsides to this method. First of all, the view is limited to a selected few different cross-sections the author chose to present. This may not be sufficient in some cases to fully convey how an organ is located relative to its surroundings since occlusions limit the possibilities to visualize these spatial relations [6]. Another problem is that often the organs are only depicted schematically by leaving out details or distorting tissue colors; thus, giving only a coarse impression on how the organ actually looks like in reality. For example, it is difficult to interpret the spatial and physical characteristics of anatomy by observing two-dimensional images, diagrams, or photographs. Many physical models also lack detail levels to fully understand the specific anatomy (see Figure 1).

Figure 1.

Traditional Atlas and physical model methodologies for medical learning.

Anatomy education is also performed by the dissection of cadavers. The value of dissection classes as a teaching format lies in the fact that it provides a 3D view on human anatomy including tactile learning experiences. It enables elaboration of knowledge already acquired in lectures and study books and it provides an overall perspective of anatomical structures and their mutual relations in a whole organism [7]. The cadaver-based learning has seen decline due to practical and cost issues (see Figure 2). And so far, no objective empirical evidence exists concerning the effectiveness of dissection classes for learning anatomy [8].

Figure 2.

Anatomy learning with cadavers.

Computer-based learning: It is developed by experts and students can use these materials if there is no available expert in the hospital. Computer-based education can be very powerful for anatomy teaching, where 3D visualization is of great benefit (see Figure 3). Many virtual model databases exist and E-learning is commonly used today. These resources are valuable and are more interactive and interesting than textbooks. In addition, the personalization in E-learning systems has been the subject of much recent research and allows teachers to select parameters and combine them flexibly to define different personalized strategies according to the specifics of the courses. In recent times, digital approaches are evolving, which are trying to improve on these methods. Many of them offer interactive anatomy models usable either as an on-line service or as a standalone application. There also exist organizations specifically offering teaching bundles for use in classrooms, making it possible to use alternative teaching methods at different school levels assisted by videos and interactive tools.

Figure 3.

Anatomy 3D models.

Those interactive applications offer large improvements over the traditional method. It is possible to view organs and structures from any desired angle, control magnification and often even select specific organs and systems to be displayed or hidden. Many commercially available systems use virtual reality for medical education and psychomotor skills training [9, 10]. These systems have indeed proven to be valid and useful. By combining computer models of anatomical structures with custom software, we can demonstrate to students the new ways of interacting with anatomy that could not be achieved during cadaveric dissections or in static diagrams and models, thereby increasing their learning satisfaction [11]. Visualization of medical data can also be used for education. Medical datasets are very large and the visualization must correctly represent the reality. Today, images with a high resolution can be generated by performance graphics for computer gaming (see Figure 4). Through all those materials and interactive elements, the users can control the region of interest and hopefully better understand the information they are looking for. These resources are very valued, however, with the disadvantage of having users still mentally map/link anatomy onto their bodies. Humans usually have to build the knowledge network in the brain after perception of the knowledge. The information is very general but the learning procedure is quite personal. Personalized information is much easier for the user to accept. Novice medical students accept a very long and difficult study during medical school. At the same time, different technologies are developed to simplify the learning process.

Figure 4.

Recent CT volume visualization.

2.2. Public and patient education

Methodologies for education in medical schools are not suitable for public education, as it is not necessary to teach a high level of anatomy understanding. Real-life demonstrations are usually limited to the skin surface, which is why most people learn about anatomy in the form of charts or plastic models of the inner organs. Still, there is a certain fascination about looking inside one's own body, something that can usually only be achieved to a limited degree by the use of X-ray imaging or similar imaging modalities. These methods are not applicable for public education, mostly because the devices are too expensive to use without medical indication. Patient education is the process by which health professionals impart information to patients and their caregivers that will alter their health behaviors or improve their health status. It can improve the patient's understanding of medical condition, diagnosis, disease, or disability. Patient education is an important and often underestimated responsibility of a health professional [12]. It is the responsibility of a doctor to inform and motivate patients to ensure that they understand the diseases, the treatment options available and the consequences of no treatment or noncompliance [13]. This makes it possible for the patients to understand it, eventually allowing them to take responsibility for their own health [14]. Good information and communication increase the patient's possibility to contribute in the decision-making process, leading to higher levels of patient satisfaction, loyalty to the physician and favoring treatment outcomes [14–16]. Traditionally, patient-health professional communication in a clinical setting comprises of a face-to-face narrative interaction often combined with static images or real-time sketching (see Figure 5-left). It is inevitable that patients' comprehension of explanations is often lacking due to the complexity of the medical information [16]. Evidence suggests that patients often do not understand what is being said to them when information is given during a medical encounter due to cultural and educational gaps between clinicians and patients [17]. This discourages the patient and leaves them overly reliant on their doctor's advice. Improving and facilitating the information process and understanding by the patient has been a focus of research in the medical field. Ong et al. [18] presented a literature survey of doctor-patient communication, summarizing the key topics of information exchange, interpersonal relationship building, and shared medical decision making. Wilcox et al. [19] proposed a design for patient-centric information displays to deliver useful information to a patient during an emergency room visit, since the patients are frequently under informed. Previous work has also demonstrated that technology can positively impact communication. Hsu et al. [20] conducted a longitudinal study the impact of in-room access to electronic health records on doctor-patient interactions during outpatient visits. Recent reviews show growing evidence with regard to the benefit of multimedia tools in enhancement of patients' satisfaction and improvement of knowledge retention [17]. A patient education computer program using 3D multimedia videos is shown in Figure 5-right.

Figure 5.

Patient education scenarios using Atlas or multimedia.

In addition, 3D animations have been suggested, for educating audiences that are preliterate or have limited literacy skills, such as children with mental handicaps [21] and to be useful for patients with a lower learning pace as it leads to a better understanding of the disease [22]. Ni et al. [23] presented the design, development, and evaluation of AnatOnMe, a projection-based handheld device designed to facilitate medical information exchange (see Figure 6). Adopting a user-centered design approach, they interviewed medical professionals to understand their practices, attitudes, and difficulties in exchanging information with patients, as well as to identify their workflow, tasks, and design requirements for a technology intervention. Gonzales and Riek [24] presented an application that personalized the content presented on a device of the patient's diagnosis in an easy-to-understand language, rather than hard to understand medical terminology. It also encourages doctor-patient interaction on the main relevant areas of the patient's diagnosis. Ihrig et al. [25] evaluated the view of physicians performing multimedia supported preoperative education within a randomized controlled trial, enrolling 8 physicians and 203 patients. Both patients and physicians profited from multimedia support for education and counseling without prolonging patient education.

Figure 6.

AnatOnMe in use in a simulated physical therapy consultation.

2.3. Rehabilitation exercises

Rehabilitation is the process to regain full function following injury. This involves restoring strength, flexibility, endurance and power and is achieved through various exercises and drills (see Figure 7). Rehabilitation is as important as treatment following an injury but unfortunately is often overlooked. Usually physiatrists are involved in the exercise procedure and optimal physical activities are carried out in an effort to reach specific health objectives. Its purpose is to return to normal musculoskeletal function or to decrease pain caused by injuries. The communication between a medical assistant and a patient is very important during the exercises, e.g., make sure that the patient knows the current state of his/her disability and follows guidelines during the rehabilitation process. The rehabilitation exercises are commonly performed in a rehabilitation center, and the physiotherapist identifies and evaluates the movements and motor functions that are being affected. In order to achieve kinetic gains as quickly as possible, patients are encouraged to perform the same exercise movements several times [26]. Patients' learning by movement repetition is crucial for their successful therapy.

Figure 7.

Physiotherapists leading the exercises during rehabilitation therapy.

The time that the patient expends in the rehabilitation center is relatively short compared with the time he/she spends at home with no supervision [27]. However, there are some issues with home exercises: (i) the patient cannot be motivated by the therapist and (ii) wrong movements might be performed without timely correction. Today, interactive solutions for rehabilitation are being developed using different sensors and motion tracking systems including gloves, magnetic, fiducial or infrared markers, video and depth cameras. Mixed-reality systems are also introduced to motivate the patient and facilitate a more accurate exercise [28, 29]. The activity of patients' muscles can be visualized and the movement is checked based on the contracted muscles [30]. Cho et al. [29] developed a virtual-reality proprioception rehabilitation system for stroke patients to use proprioception feedback in upper limb rehabilitation by blocking visual feedback. The markerless tracking feature of Kinect enables a natural user interaction for rehabilitation applications, which alleviates existing issues for patients having difficulty to hold any sensor or marker. Using Kinect, various researches are being explored for the development of assistive systems that help interact with patients during their rehabilitation exercises [31–34].

3. The magic mirror

Providing adequate learning experience to different learners is a challenging issue as the traditional techniques generally cannot adapt content to suit the individual learner's needs. Personalization for promoting a multi-modal learning environment is a growing area of interest, such as the development of user modeling and personalized processes, which place the student at the center of the learning development. Augmented reality systems can present a virtual representation of the subject material and create a direct connection between the information the user wants to learn and their own body, and activities at the same time. Hence, it could help understand and memorize complex information, and either supplement conventional learning or even supersede it altogether. Previous mixed-reality systems on visualization of anatomy used expensive technologies involving head mounted displays (HMDs) or markers. Recently, a Magic Mirror technology has been developed for anatomy education, by employing a camera for tracking and a TV display for visualization to of an augmented reality view [35]. The system presented is both inexpensive and easy to use. It presents medical data augmented onto the user's body and shows additional 2D and 3D information according to the user's needs. The Magic Mirror provides the user ‘superman ability’ to look into their body. It enables the medical information to be perceived naturally linking it to a real human body. Natural gesture is chosen as the interaction methodology, and interaction with the augmented reality view of the user's body provides a personalized perception in the Magic Mirror framework.

The Magic Mirror [36] is a user interface technique that mimics a normal mirror and presents nonphysical visual feedback in addition to the normally optical effect. Here the user stands in front of a screen and via a camera, the image of the user is shown on the screen such that it acts like a mirror. While previous systems have augmented objects onto the user, this system extends the concept for medical education and rehabilitation. A depiction of this can be seen in Figure 8.

Figure 8.

By using the Magic Mirror system, the user is led to believe that he or she is able to look inside their own body.

To achieve this visualization, the Magic Mirror augments the volume visualization of a CT dataset onto the user. To allow a correct augmentation of the medical data, the pose of the user is tracked. The Magic Mirror concept came out as a framework and it generates a personalized perception of the medical information for every user. In addition, the framework takes the user's natural gesture as input to create an interactive mixed-reality environment. Knowledge about human anatomy is an important issue for everyone working in the field of medicine. It is also an important part of the general education and relevant for many other professions related, e.g., to health-care or sports. Human anatomy is very complex and it does not only involve knowledge about a single organ, but also about issues such as chemical processes, human motion and spatial relations inside the body. Consequently, teaching human anatomy is very difficult and often a large effort is spent on teaching it; e.g., by letting students perform dissection courses, creating illustrations, using plastic models of anatomy or by utilizing 3D computer graphics. The Magic Mirror framework is firstly employed to display anatomical structures overlaid onto the body of the user to intuitively teach human anatomy.

3.1. Hardware setup

The Magic Mirror technology focuses on a few important organs of the abdomen, namely the liver, lungs, pancreas, stomach, and bones. The system prototype has a mirror-like effect to the user by projecting a ‘looking glass’ on the body and displays the skeleton of the user, rendered from CT data and anatomy 3D models. The framework tracks users' movements using a depth camera and an algorithm to detect the pose of the user from the depth image. This is realized using the Microsoft Kinect, which was originally developed to allow controlling computer games by motion. By using the Magic Mirror metaphor, the user is led to believe that he or she is able to look inside their own body. At the same time, medical information (CT, MRI data, and a fully segmented dataset of cross-sectional photographs of the human body) is augmented in real-time. The current system also allows visualization of static anatomy on the user and offers a simple user interface to select CT, MRI, or photographic slices [35, 37]. An illustration of the hardware setup can be seen in Figure 9. The first component of the system is a display device. In different setups of the technology, large TV screens or video projection onto a planar surface has been used. The second component is a color camera, which is mounted next to the display surface and which is looking at the user and enables visual perception of the information.

Figure 9.

Magic Mirror hardware setup showing the general design and the current implementation using Kinect.

The third component is a depth sensor, which is placed next to the color camera and which has a similar field of view and viewing direction as the color camera to collect the user's skeleton information. The current system uses the Microsoft Kinect V1, which consists of a color and a depth camera that are assembled into a mechanical housing. The depth sensor is an infrared camera that uses structured light, which is emitted by an additional infrared projector to estimate depth values for each pixel. The user's skeleton and personal information can be generated from the Kinect sensor based on machine learning. The system employs the color camera to create a mirror-like effect to the user, and all the nonphysical visual feedbacks are generated based on the user's skeleton and personal information via volume rendering the corresponding medical information onto the human body.

3.2. Software setup

The system framework of the prototype is illustrated in Figure 10. To access the Kinect the system uses Microsoft Kinect SDK or OpenNI1, which is an open source software framework that allows retrieving color and depth images from the Kinect. The depth sensor is used for two purposes. First, the depth values are projected to the color image providing depth information for each pixel in the color image. Second, a skeleton tracking algorithm uses the depth image to track the pose of multiple joints of a user who is standing in front of the camera. For skeleton tracking, the Magic Mirror uses NITE, software by PrimeSense, that performs gesture recognition and skeleton tracking based on depth images. NITE can be used with the Kinect through the OpenNI framework. The Magic Mirror augmented reality view is based on the information perceived via Kinect and corresponding medical information.

Figure 10.

Magic Mirror system framework. The lowest two layers represent the open source library to access the Kinect raw image data and skeleton information. The middle layer is the modules used to create Magic Mirror AR view of the user. The top level is the application with basic system features.

As shown in Figure 10, there are four modules: Skeleton & personal information processing is important to achieve personalized perception. Color & depth image processing is the basic module to generate the mirror-like effect and merge the nonphysical visual outcome. 3D medical information rendering processes the corresponding 3D medical images or models and generates virtual elements for the Magic Mirror augmented reality view and it can employ OpenGL and OpenGl Shading Language, and any other 3D rendering libraries, e.g., Coin3D. 2D information includes window management and basic user interface elements, such as 2D text and image information, and it can be implemented using Qt. The applications based on the Magic Mirror AR View have two important features, AR in situ view and interactive mixed reality. The blue rectangles represent the basic visualization functions of the system.

3.3. Medical information

Aside from text and 2D atlases, 3D volumes and models are also important methodologies to represent medical information. The common way in medicine is to use 3D volumes, where we have one pixel or voxel for every point in 3D. This is the kind of data we get from CT or MRI. The other ones are polygonal or surface models, where only the surface (e.g., skin or the surface of the organ) is stored. For models that have been created for education often surface models are used because they look better. While the Magic Mirror technology can use a CT or MRI scan, it does not make sense to acquire a CT or MRI scan of the user if it is not required for medical reasons. Therefore, we augment the medical volume from the visible Korean human (VKH) dataset onto the user [38]. This dataset consists of a CT scan, an MRI volume and a photographic volume, which has been acquired by stacking up cryosections. Most CT and MRI medical images are saved in the DICOM standard, which is the format that is used in all hospitals. Unfortunately, most software for research does not support DICOM. The Magic Mirror system takes an .MHD file as the medical data. One drawback of the volume data is that the 3D dataset cannot be deformed in real time. So if the user bends, this is not reflected in the visualization of the medical data leading to movements of the limbs not visualized correctly. Possible solutions to address this issue have been published in Section 3.4, but for the current technology, which focuses on the abdominal area, this is a minor issue. Visualizing structures other than bones from the CT is more challenging. In a first attempt, the segmentation that is available for the CT volume was used to visualize different organs in the abdominal area. The quality of the visualization was low, as the segmentation does not have subpixel accuracy and transfer functions on CT intensities cannot provide visualization with realistic colors and textures of organs. Therefore instead of using the volumetric data, additional polygonal models were integrated. The Anatomium2 dataset provides polygonal models of many organs of the human body. A scene graph including multiple organs was extracted from the dataset. Using Coin3D this scene graph is augmented onto the user. The simultaneous visualization of bones from CT and a polygonal model of the small intestine is shown in Figure 11.

Figure 11.

The AR in situ visualization of human anatomy in the Magic Mirror technology.

3.4. Related publications

It is my firm belief that providing a more personalized solution allowing in situ contextual visualization onto one's body would provide a greater learning experience when compared to existing solutions. Not only does ‘seeing inside the body’ spark curiosity and excitement, it also permits a sense of ownership and fidelity during the learning process. The Magic Mirror technology allows visualization of medical and radiological CT data directly on the user body and a gesture-based user interface (UI) allows direct interaction with this data [39]. A large user study was carried out earlier in 2016 to verify the learning potential and acceptability of the technology. The Magic Mirror was tested and evaluated by 748 first and second semester medical students at the Anatomy Department at the Klinikum der Universität München, Germany, and the learning outcomes were extremely positive, particularly with respect to three dimensional understanding of organs and better comprehension of anatomical structures. Preliminary work in rehabilitation research [40, 41] resulted in two publications in 2016. The first introduced a MirrARbilitation system that improves patient exercise engagement and performance quality. Compared to state-of-art, the system consists of a gesture recognition tool based on the International Biomechanical Standards terminology for the reporting of human joint motion. An improvement from 69.02 to 93.73% of correct exercises on a cohort of 33 end users was demonstrated by the system. The second introduced a mixed-reality system facilitating the learning of the muscles of the upper extremities. The system consists of two main components: an augmented reality view superimposing the virtual model of the arm on top of the video stream and a virtual reality view, providing a more detailed image of the muscles. A user study including 20 students was performed indicating that the system is useful for learning the structure and function of the muscle and can be a valuable supplementary to established muscle learning paradigms.

4. Conclusion

This chapter presented solutions to educate the medical and general population. It concluded by highlighting a personalized Magic Mirror system for anatomy and rehabilitation education. The system combines both augmented and virtual reality environments, which are certified useful for anatomy learning. Both academic and medical center methodologies must be changed according to the advances of new technologies, but there is still a long way for this. Considering the benefits of the personalized and interactive systems for motivation and perception of anatomy learning, new technologies can additionally be helpful to facilitate autonomous learning and secondarily to reduce laboratory material and instructor costs. Together with the anatomy & medical community, we hope to initiate such discussions in integrating exciting user-specific and gaming concepts via new anatomy learning systems and ultimately for improved patient education.

Acknowledgments

I would like to thank the Chair for Computer Aided Medical Procedures & Augmented Reality, Technische Universität Munchen, for allowing me the great opportunity of growing my research in this area.