This chapter aims to assess the current practices of anatomy education technology and provides future directions for medical education. It begins by presenting a historical synopsis of the current paradigms for anatomy learning followed by listing their limitations. Then, it focuses on several innovative educational technologies, which have been introduced over the past years to enhance the learning. These include E-learning, mobile apps, and mixed reality. The chapter concludes by highlighting future directions and addressing the barriers to fully integrating the technologies in the medical curriculum. As new technologies continue to arise, this process-oriented understanding and outcome-based expectations of educational technology should be embraced. With this view, educational technology should be valued in terms of how well the technological process informs and facilitates learning, and the acquisition and maintenance of clinical expertise.
- anatomy learning
- gross anatomy
- mixed reality
- medical education
Medicine is the science and practice of the diagnosis, treatment, and prevention of disease, and most medical information is about our human body. Medical information is employed in different scenarios, such as education, training, diagnosis, surgery, etc. As the computing technologies develop, more information is digitized from the physical world, and then researchers work on how to process and show them back to the user, enabling the user to perceive and interact with the information naturally and effectively. Perception and interaction with different media and objects are the fundamental human activities, and they are user specific .
Anatomical education is an important content of every curriculum and starts already very early in school, to form a good understanding of the body and improve the general population's health awareness . With the advent of the plethora of exciting technological advancements there should be no reason not to include these for the creation of new education paradigms for medical learning. As such, this chapter presents an overview of the fascinating novel research, which is being undertaken in this area.
A clinician's knowledge is collected through many years of medical practice. In various settings, the knowledge will be transferred to medical students through medical education and to the general population through public education. For example, in the surgical setting, medical imaging data are collected for diagnosis and navigation. Good communication between the patient and the surgeon is very important for making the patient comply with the surgical procedure. The same communication plays a vital role for patients who perform rehabilitation exercises after surgery. In all of these scenarios, key players act different roles to perceive and interact with medical information. In this section, we discuss the state-of-art and challenges involved in medical education, public education, and the communication to prepare for surgery and rehabilitation.
2.1. Medical education
Though medical education has been existent for centuries, we begin from Abraham Flexner's report in 1910 . Flexner visited 155 medical schools that existed in North America in 1909, and he identified four major problems during medical education: (1) lack of standardization, (2) lack of integration, (3) lack of inquiry, and (4) identity formation. The report had a huge impact and shaped modern medical education, where patient care, teaching, and research are combined. However, academic hospitals do not spend enough time on teaching due to enormous pressure to publish, as well as, for economic reasons . Another issue is that research is focusing on very small subtopics, which do not relate to medical education . On the other hand, only little research is done for teaching. An example is gross anatomy where most topics are already known and only little novel research exists. In today's medical schools students are required to understand both functional and spatial context of human anatomy.
Anatomy education is also performed by the dissection of cadavers. The value of dissection classes as a teaching format lies in the fact that it provides a 3D view on human anatomy including tactile learning experiences. It enables elaboration of knowledge already acquired in lectures and study books and it provides an overall perspective of anatomical structures and their mutual relations in a whole organism . The cadaver-based learning has seen decline due to practical and cost issues (see Figure 2). And so far, no objective empirical evidence exists concerning the effectiveness of dissection classes for learning anatomy .
Those interactive applications offer large improvements over the traditional method. It is possible to view organs and structures from any desired angle, control magnification and often even select specific organs and systems to be displayed or hidden. Many commercially available systems use virtual reality for medical education and psychomotor skills training [9, 10]. These systems have indeed proven to be valid and useful. By combining computer models of anatomical structures with custom software, we can demonstrate to students the new ways of interacting with anatomy that could not be achieved during cadaveric dissections or in static diagrams and models, thereby increasing their learning satisfaction . Visualization of medical data can also be used for education. Medical datasets are very large and the visualization must correctly represent the reality. Today, images with a high resolution can be generated by performance graphics for computer gaming (see Figure 4). Through all those materials and interactive elements, the users can control the region of interest and hopefully better understand the information they are looking for. These resources are very valued, however, with the disadvantage of having users still mentally map/link anatomy onto their bodies. Humans usually have to build the knowledge network in the brain after perception of the knowledge. The information is very general but the learning procedure is quite personal. Personalized information is much easier for the user to accept. Novice medical students accept a very long and difficult study during medical school. At the same time, different technologies are developed to simplify the learning process.
2.2. Public and patient education
Methodologies for education in medical schools are not suitable for public education, as it is not necessary to teach a high level of anatomy understanding. Real-life demonstrations are usually limited to the skin surface, which is why most people learn about anatomy in the form of charts or plastic models of the inner organs. Still, there is a certain fascination about looking inside one's own body, something that can usually only be achieved to a limited degree by the use of X-ray imaging or similar imaging modalities. These methods are not applicable for public education, mostly because the devices are too expensive to use without medical indication. Patient education is the process by which health professionals impart information to patients and their caregivers that will alter their health behaviors or improve their health status. It can improve the patient's understanding of medical condition, diagnosis, disease, or disability. Patient education is an important and often underestimated responsibility of a health professional . It is the responsibility of a doctor to inform and motivate patients to ensure that they understand the diseases, the treatment options available and the consequences of no treatment or noncompliance . This makes it possible for the patients to understand it, eventually allowing them to take responsibility for their own health . Good information and communication increase the patient's possibility to contribute in the decision-making process, leading to higher levels of patient satisfaction, loyalty to the physician and favoring treatment outcomes [14–16]. Traditionally, patient-health professional communication in a clinical setting comprises of a face-to-face narrative interaction often combined with static images or real-time sketching (see Figure 5-left). It is inevitable that patients' comprehension of explanations is often lacking due to the complexity of the medical information . Evidence suggests that patients often do not understand what is being said to them when information is given during a medical encounter due to cultural and educational gaps between clinicians and patients . This discourages the patient and leaves them overly reliant on their doctor's advice. Improving and facilitating the information process and understanding by the patient has been a focus of research in the medical field. Ong et al.  presented a literature survey of doctor-patient communication, summarizing the key topics of information exchange, interpersonal relationship building, and shared medical decision making. Wilcox et al.  proposed a design for patient-centric information displays to deliver useful information to a patient during an emergency room visit, since the patients are frequently under informed. Previous work has also demonstrated that technology can positively impact communication. Hsu et al.  conducted a longitudinal study the impact of in-room access to electronic health records on doctor-patient interactions during outpatient visits. Recent reviews show growing evidence with regard to the benefit of multimedia tools in enhancement of patients' satisfaction and improvement of knowledge retention . A patient education computer program using 3D multimedia videos is shown in Figure 5-right.
In addition, 3D animations have been suggested, for educating audiences that are preliterate or have limited literacy skills, such as children with mental handicaps  and to be useful for patients with a lower learning pace as it leads to a better understanding of the disease . Ni et al.  presented the design, development, and evaluation of AnatOnMe, a projection-based handheld device designed to facilitate medical information exchange (see Figure 6). Adopting a user-centered design approach, they interviewed medical professionals to understand their practices, attitudes, and difficulties in exchanging information with patients, as well as to identify their workflow, tasks, and design requirements for a technology intervention. Gonzales and Riek  presented an application that personalized the content presented on a device of the patient's diagnosis in an easy-to-understand language, rather than hard to understand medical terminology. It also encourages doctor-patient interaction on the main relevant areas of the patient's diagnosis. Ihrig et al.  evaluated the view of physicians performing multimedia supported preoperative education within a randomized controlled trial, enrolling 8 physicians and 203 patients. Both patients and physicians profited from multimedia support for education and counseling without prolonging patient education.
2.3. Rehabilitation exercises
Rehabilitation is the process to regain full function following injury. This involves restoring strength, flexibility, endurance and power and is achieved through various exercises and drills (see Figure 7). Rehabilitation is as important as treatment following an injury but unfortunately is often overlooked. Usually physiatrists are involved in the exercise procedure and optimal physical activities are carried out in an effort to reach specific health objectives. Its purpose is to return to normal musculoskeletal function or to decrease pain caused by injuries. The communication between a medical assistant and a patient is very important during the exercises, e.g., make sure that the patient knows the current state of his/her disability and follows guidelines during the rehabilitation process. The rehabilitation exercises are commonly performed in a rehabilitation center, and the physiotherapist identifies and evaluates the movements and motor functions that are being affected. In order to achieve kinetic gains as quickly as possible, patients are encouraged to perform the same exercise movements several times . Patients' learning by movement repetition is crucial for their successful therapy.
The time that the patient expends in the rehabilitation center is relatively short compared with the time he/she spends at home with no supervision . However, there are some issues with home exercises: (i) the patient cannot be motivated by the therapist and (ii) wrong movements might be performed without timely correction. Today, interactive solutions for rehabilitation are being developed using different sensors and motion tracking systems including gloves, magnetic, fiducial or infrared markers, video and depth cameras. Mixed-reality systems are also introduced to motivate the patient and facilitate a more accurate exercise [28, 29]. The activity of patients' muscles can be visualized and the movement is checked based on the contracted muscles . Cho et al.  developed a virtual-reality proprioception rehabilitation system for stroke patients to use proprioception feedback in upper limb rehabilitation by blocking visual feedback. The markerless tracking feature of Kinect enables a natural user interaction for rehabilitation applications, which alleviates existing issues for patients having difficulty to hold any sensor or marker. Using Kinect, various researches are being explored for the development of assistive systems that help interact with patients during their rehabilitation exercises [31–34].
3. The magic mirror
Providing adequate learning experience to different learners is a challenging issue as the traditional techniques generally cannot adapt content to suit the individual learner's needs. Personalization for promoting a multi-modal learning environment is a growing area of interest, such as the development of user modeling and personalized processes, which place the student at the center of the learning development. Augmented reality systems can present a virtual representation of the subject material and create a direct connection between the information the user wants to learn and their own body, and activities at the same time. Hence, it could help understand and memorize complex information, and either supplement conventional learning or even supersede it altogether. Previous mixed-reality systems on visualization of anatomy used expensive technologies involving head mounted displays (HMDs) or markers. Recently, a Magic Mirror technology has been developed for anatomy education, by employing a camera for tracking and a TV display for visualization to of an augmented reality view . The system presented is both inexpensive and easy to use. It presents medical data augmented onto the user's body and shows additional 2D and 3D information according to the user's needs. The Magic Mirror provides the user ‘superman ability’ to look into their body. It enables the medical information to be perceived naturally linking it to a real human body. Natural gesture is chosen as the interaction methodology, and interaction with the augmented reality view of the user's body provides a personalized perception in the Magic Mirror framework.
The Magic Mirror  is a user interface technique that mimics a normal mirror and presents nonphysical visual feedback in addition to the normally optical effect. Here the user stands in front of a screen and via a camera, the image of the user is shown on the screen such that it acts like a mirror. While previous systems have augmented objects onto the user, this system extends the concept for medical education and rehabilitation. A depiction of this can be seen in Figure 8.
To achieve this visualization, the Magic Mirror augments the volume visualization of a CT dataset onto the user. To allow a correct augmentation of the medical data, the pose of the user is tracked. The Magic Mirror concept came out as a framework and it generates a personalized perception of the medical information for every user. In addition, the framework takes the user's natural gesture as input to create an interactive mixed-reality environment. Knowledge about human anatomy is an important issue for everyone working in the field of medicine. It is also an important part of the general education and relevant for many other professions related, e.g., to health-care or sports. Human anatomy is very complex and it does not only involve knowledge about a single organ, but also about issues such as chemical processes, human motion and spatial relations inside the body. Consequently, teaching human anatomy is very difficult and often a large effort is spent on teaching it; e.g., by letting students perform dissection courses, creating illustrations, using plastic models of anatomy or by utilizing 3D computer graphics. The Magic Mirror framework is firstly employed to display anatomical structures overlaid onto the body of the user to intuitively teach human anatomy.
3.1. Hardware setup
The Magic Mirror technology focuses on a few important organs of the abdomen, namely the liver, lungs, pancreas, stomach, and bones. The system prototype has a mirror-like effect to the user by projecting a ‘looking glass’ on the body and displays the skeleton of the user, rendered from CT data and anatomy 3D models. The framework tracks users' movements using a depth camera and an algorithm to detect the pose of the user from the depth image. This is realized using the Microsoft Kinect, which was originally developed to allow controlling computer games by motion. By using the Magic Mirror metaphor, the user is led to believe that he or she is able to look inside their own body. At the same time, medical information (CT, MRI data, and a fully segmented dataset of cross-sectional photographs of the human body) is augmented in real-time. The current system also allows visualization of static anatomy on the user and offers a simple user interface to select CT, MRI, or photographic slices [35, 37]. An illustration of the hardware setup can be seen in Figure 9. The first component of the system is a display device. In different setups of the technology, large TV screens or video projection onto a planar surface has been used. The second component is a color camera, which is mounted next to the display surface and which is looking at the user and enables visual perception of the information.
The third component is a depth sensor, which is placed next to the color camera and which has a similar field of view and viewing direction as the color camera to collect the user's skeleton information. The current system uses the Microsoft Kinect V1, which consists of a color and a depth camera that are assembled into a mechanical housing. The depth sensor is an infrared camera that uses structured light, which is emitted by an additional infrared projector to estimate depth values for each pixel. The user's skeleton and personal information can be generated from the Kinect sensor based on machine learning. The system employs the color camera to create a mirror-like effect to the user, and all the nonphysical visual feedbacks are generated based on the user's skeleton and personal information via volume rendering the corresponding medical information onto the human body.
3.2. Software setup
The system framework of the prototype is illustrated in Figure 10. To access the Kinect the system uses Microsoft Kinect SDK or OpenNI1, which is an open source software framework that allows retrieving color and depth images from the Kinect. The depth sensor is used for two purposes. First, the depth values are projected to the color image providing depth information for each pixel in the color image. Second, a skeleton tracking algorithm uses the depth image to track the pose of multiple joints of a user who is standing in front of the camera. For skeleton tracking, the Magic Mirror uses NITE, software by PrimeSense, that performs gesture recognition and skeleton tracking based on depth images. NITE can be used with the Kinect through the OpenNI framework. The Magic Mirror augmented reality view is based on the information perceived via Kinect and corresponding medical information.
As shown in Figure 10, there are four modules: Skeleton & personal information processing is important to achieve personalized perception. Color & depth image processing is the basic module to generate the mirror-like effect and merge the nonphysical visual outcome. 3D medical information rendering processes the corresponding 3D medical images or models and generates virtual elements for the Magic Mirror augmented reality view and it can employ OpenGL and OpenGl Shading Language, and any other 3D rendering libraries, e.g., Coin3D. 2D information includes window management and basic user interface elements, such as 2D text and image information, and it can be implemented using Qt. The applications based on the Magic Mirror AR View have two important features, AR
3.3. Medical information
Aside from text and 2D atlases, 3D volumes and models are also important methodologies to represent medical information. The common way in medicine is to use 3D volumes, where we have one pixel or voxel for every point in 3D. This is the kind of data we get from CT or MRI. The other ones are polygonal or surface models, where only the surface (e.g., skin or the surface of the organ) is stored. For models that have been created for education often surface models are used because they look better. While the Magic Mirror technology can use a CT or MRI scan, it does not make sense to acquire a CT or MRI scan of the user if it is not required for medical reasons. Therefore, we augment the medical volume from the visible Korean human (VKH) dataset onto the user . This dataset consists of a CT scan, an MRI volume and a photographic volume, which has been acquired by stacking up cryosections. Most CT and MRI medical images are saved in the DICOM standard, which is the format that is used in all hospitals. Unfortunately, most software for research does not support DICOM. The Magic Mirror system takes an
3.4. Related publications
It is my firm belief that providing a more personalized solution allowing
This chapter presented solutions to educate the medical and general population. It concluded by highlighting a personalized Magic Mirror system for anatomy and rehabilitation education. The system combines both augmented and virtual reality environments, which are certified useful for anatomy learning. Both academic and medical center methodologies must be changed according to the advances of new technologies, but there is still a long way for this. Considering the benefits of the personalized and interactive systems for motivation and perception of anatomy learning, new technologies can additionally be helpful to facilitate autonomous learning and secondarily to reduce laboratory material and instructor costs. Together with the anatomy & medical community, we hope to initiate such discussions in integrating exciting user-specific and gaming concepts via new anatomy learning systems and ultimately for improved patient education.
I would like to thank the Chair for Computer Aided Medical Procedures & Augmented Reality, Technische Universität Munchen, for allowing me the great opportunity of growing my research in this area.