Augmented Reality as a New and Innovative Learning Platform for the Medical Area

Abstract

This research paper shows an Augmented Reality (AR) project applied to medicine. The project is crystallized through a system, based on this new technology that serves as an innovative and innovative learning platform, which, in turn, helps in both teaching and learning abstract concepts in medicine, which requires of visual and manipulable objects difficult to obtain, due to the large space they occupy in magnetic media or because of how difficult it is to get their models in physical form. The proposed system strengthens the anatomical identification process in the area of medicine, specifically in the physiological activity of the human heart. In addition, this system allows interaction with the students, through which certain body parts of the human heart are identified, and, consequently, facilitates their learning with an iterative operation. Finally, the system is focused, so that the student uses his/her sense of sight, hearing, and kinesthetic, which, together, will allow a better assimilation of knowledge.

1. Introduction

The need to have better prepared human resources and innovative and/or entrepreneurial ideas, whether generated during their studies or not, motivates the research community to respond, even in a timely manner, to each of the problems generated, in turn, for these needs [1, 2, 3]. In this context, it makes sense to create and provide a new way of learning for a specific group of students, particularly for students pursuing a medical degree. In turn, it is clear that the transfer of knowledge has evolved gradually over time, and it is also logical that teaching (which is nothing more than the process of educating) has also shown changes [4]. Therefore, new generations of students should be prepared and adapt as soon as possible to the challenges that new technologies face in the near future. In this way, new technologies favor education because they generate human resources with a better educational quality. Moreover, [5] it is shown that those countries that have allocated considerable economic investments for the development and education of their population achieved more efficient production and, as a result, maintained higher growth rates. Regarding the transfer of technology and, simultaneously, specialized knowledge have been subjects of study in multiple research papers [6, 7, 8]. However, since the end of the last century in [9] it was emphasized that aid to developing countries consisting only in the transfer of economic capital would not be sufficient if the developing country does not have an adequate level of human capital to take advantage of all possible benefit of that help. Therefore, and through new technologies, it is essential to create a new horizon in science and technology in terms of favoring and/or strengthening the current learning system and, in particular, that of medicine.

On the other hand, it is clear that technological advances have directly impacted all activities of daily life, and the educational level has not been the exception. In the latter, new strategies have been created and implemented that serve as support in the teaching-learning process, which systematically strengthen the way of teaching and learning the most current educational content. Scientific and technological changes are increasingly visible and it is clear that the interaction between technology and the human being has also changed over time. That is, the speed with which current technologies are developed, but especially the enormous amount of them, does not allow a person to assimilate them in an acceptable way. The latter is probably due to the fact that the information that is handled today is so much that contemporary human beings must quickly assimilate a growing amount, precisely, of the so-called new technologies instead of adapting them to their needs. In this way, the human being has gradually become a technodependent entity. As a consequence of all this, it can be mentioned that the current dynamics of almost all daily activities, including education and/or creation of highly specialized human resources, has led us, among other things, to be immersed in a society with technologies of vanguard. Undoubtedly, this is because the assimilation of new technologies is increasingly accessible, interactive and economical.

In this globalized world, universities are taking a fundamental role in generating human resources with cutting-edge knowledge. So much so, that [10] indicates that European universities are currently incorporating digital technologies as a support to facilitate educational processes. In this context, technological elements such as the computer, mobile devices, internet, applications for the cyber cloud, projectors, slides and other hardware or software resources have been the starting point for the creation of new learning environments. In recent years, concern for the study of learning design has increased markedly [11]. Under this approach, Augmented Reality (AR) plays a transcendental and very important role, since the combination of physical reality with virtual resources can be displayed in the form of multimedia content. In [12] the AR is defined as a sequence of digital shots or computer generated information, while in [13] it is identified as a technology of high relevance for teaching, learning and creative research.

Among the strategies that have shown great support for the new educational systems, generated through the AR, stand out the Intelligent Tutorial Systems (ITS). In addition, ITSs have proven to be a fairly efficient tool to support learning processes and if ICTs are added to this dynamic, then completely new results can be obtained [14]. For example, these intelligent systems make use of new technologies to improve the performance and motivation of workers when assembling an appliance, relying on a manual containing AR. In this sense, the “Intelligent augmented reality training for mother board assembly” has shown great contributions in the training for manual assembly tasks [15]. With this perspective, a more effective and faster learning experience has been presented, where less experienced users learn to mount a “motherboard” on the computer. Therefore, AR can be a technological tool of great support for learning, which can be verified in the system “Augmented reality in informal learning environments: A field experiment in a mathematics exhibition” [16]. Through this educational environment, students manage to associate and understand, in a more didactic way, mathematical figures with entities that exist in the physical world. Also, the “An interactive augmented reality system for learning anatomy structure” system, which is set out in [17], shows the parts of the anatomical structure of the human body. This system allows students to identify each anatomical part, but not only shows those anatomical parts but also that the interaction is more dynamic and in greater detail. For its part, the “Interactive augmented reality using Scratch 2.0 to improve physical activities for children with developmental disabilities” system, shown in [18], uses an interactive game for body movement, whose purpose is to improve children’s driving force with a disability To conclude with the applications of the AR, in Magic Book [19], which is a normal book containing bookmarks, the system when it detects a bookmark then shows a three-dimensional image or starts a video story. In addition, in this system users can feel the sensation of flying and appreciate themselves as avatars of a story, that is, the user can enter the virtual world of the book.

In this way, the multiple scenarios offered by the AR are extremely competent, which can be well adopted in multiple fields of knowledge. There are applications that target the mass market for advertising, entertainment and education [20]. However, medicine has been little explored in this context. Learning medicine also requires optimal means, both technological and other, that allow students to learn by doing. This can be achieved with interactive systems, since this dynamic makes knowledge reach the brain in a sensitive, visual, auditory and kinesthetic way. This with the purpose of acquiring the necessary, sufficient and current academic competences for their adequate professional development as future doctors; they also require economic systems that do not need accessories that are not easily accessible and difficult to obtain. Therefore, and to acquire the skills of know-how, medical students must learn to perform medical procedures with living or dead people. It is here where, precisely, new technologies make sense and relevance because multiple medical processes can well be simulated by AR. Developing a platform that manages AR would be a means of training for each student, whose studies are related to medicine, to evaluate their own learning before performing any procedure with living people. Thus, the present work aims to create an interactive system or platform that serves as a support for learning medical procedures and, in particular, to know in a novel way the functioning of the human heart.

2. Application description

The objective of this work is to apply the AR through a knowledge management system, which will function as a novel and efficient learning strategy to be applied in the area of medicine. Based on the software engineering mentioned in [21], where communication guidelines, requirements analysis, design, program construction, testing and support are used for the construction of software. The work program that was established was as follows:

1. Analysis of the system requirements with AR. In this phase, we investigated the knowledge that the student should acquire with this material, that is, the functioning of the human heart.

2. Formulate the abstract and physical design, code and create the software. In this stage the AR was carried out, that is, a human heart was designed to be shown three-dimensionally and the events (the characteristics of the Main Menu presented by the AR) they were encoded with the help of JavaScript, HTML5 and with the implementation of A-frame, free type software.

3. Prepare the files, identify and create test data, test and integrate the software. The elements of the AR were encapsulated, obtaining the file that can be executed in a Windows or Mac environment, with the help of a web server, a webcam, a loud voice or a mobile device.

3. Basic concepts

AR arose in the year of 1996, when ARQuake is presented, the first outdoor game with mobile devices of AR, developed in [22]. Then, in 2008, the Wikitude travel and tour guide application was released for sale [23], which was done through AR by means of a digital compass, orientation sensors and accelerometer, maps, video and informative content from Wikipedia. In 2009, ARToolKit [24] arises, which is a totally oriented platform to generate AR. From these applications, the AR has been used as the basis of numerous projects in different areas, ranging from entertainment, industry, maintenance, music, medicine and education, among others. Moreover, very specific applications have been made as a virtual harp that was designed for people with disabilities, which works through vibrations [25].

In the educational field, with the AR, molecular structures, mathematics, architecture, astronomy and physical activities can be taught to children with disabilities. In medicine and education, specialized projects have been developed that show AR as an efficient learning tool, such as the “An interactive augmented reality system for learning anatomy structure” system [26] which is integrated into three activities; The first is to show the parts of the anatomical structure of the human body, the second is that it allows students to identify each anatomical part of the human body, and the third allows a deep glimpse of the internal parts of the aforementioned anatomical structure.

In this way, our learning platform describes the design of a system such as the establishment of data structures, the general architecture of the software and the representations of interfaces and algorithms. That is, the system process translates the requirements into software specifications. The objective of the design phase is to publicize the behavior of the proposed solution; This is conceived taking into account that the design is a preface that begins the construction of programs and/or activity processes that are normally carried out by users, which seek to be improved by adding speed, efficiency, efficiency, savings and visual design. The first action that begins with the design is the determination of the system architecture, which refers to the hierarchical structure of the program modules and, in addition, is focused on the way to interact between its components and the structure of the data used by them. There are currently several software architectural styles. However, and due to the nature of the system presented in this work, the methodology applied (which is also called active or dynamic practice) has as its main feature to perform a search for the immediate application or use the knowledge acquired to then confront them with practical problems or real concrete circumstances [27]; All of these features adapt perfectly to the objective of our system.

The system oriented to the functioning of the human heart with AR aims to show and obtain an application (software) that helps medical students to identify the parts that make up this operation in a virtual model, through various sensory senses. With the approach shown in [28], the software presented in this work is aimed at making available to students an application that is easy to obtain and at the lowest possible cost, since the system can be run on a desktop computer and whose main requirement is that it contains a webcam for the detection of the images that will be processed with the AR. This technology makes it possible first, for medical students to learn in a three-dimensional entity and then, that they can simulate the physical constitution of a model and the functioning of the parts involved in the psychological process related to the functioning of the human heart.

4. System architecture

After analyzing the system and understanding and determining how it works, the requirements to create it are obtained and it is then time to proceed to elaborate the design and construction, represented by its general software architecture and interfaces and algorithms, that is, the process that translates the requirements into software specifications. The objective of the system design phase, show the behavior of the solution proposal.

The logical view of the system architecture, describes its structure and functionality, for this, a sequence diagram was used, where you can observe the actions that the entities of the system do, since the user of the RA focuses the camera of the device on the marker that triggers the RA, through the learning actions and finally the evaluation of the acquired knowledge.

The system can be seen in Figure 1, where the diagram shows the actions, starting with the one that gives access to the system, which is stored on a website, and contains link buttons showing the functions with the options: video, audio and learning, from the last one an evaluation is derived.

Figure 1.

System sequence diagram (source: prepared by the authors).

Process view: to represent this view, it is done with an activity diagram shown in Figure 2, for this view only the core activities of the systems are shown, which is to demonstrate the AR.

Figure 2.

System activities diagram (source: prepared by the authors).

For the development of the system, it was taken into account that the users already have an academic trajectory and that, their required disciplinary knowledge, are more specific, so, the resources made include knowledge directed towards students of professional level, so that in prototype of the system, which can be seen in Figure 3.

Figure 3.

Prototype of the system. (source: prepared by the authors).

5. User interface

AR is shown with virtual objects that can be shown by the association between the two-dimensional (2D), called marker, initial image and a three-dimensional (3D) image, where the first image is focused on the camera and takes the form of a physiological model of the human heart (the image was made in SketchUp and the AR was encoded with JavaScript and HTML5 on an A-frame platform), which can be manipulated to visualize it from a different position and angle, giving the appearance of a real physical model (Figure 4).

Figure 4.

Image of a two-dimensional (2D) human heart placed in front of a camera (source: prepared by the authors).

The system displays a Menu of options (Functions, Audio, Video and Learning), which “trigger” an event when the user clicks on any of its options (these actions were programmed using JavaScript on an A-frame platform). That way, when the user clicks on the Functions option, a web page will be displayed, whose purpose will be to show each element that is involved in the functioning of the human heart, showing images and text that describe it (Figure 5). When the user selects the Audio option in the Menu then an audio file will be displayed where the main functions of the human heart are related (Figure 6).

Figure 5.

Result when selecting the menu functions option (source: prepared by the authors).

Figure 6.

Result when selecting the audio option from the menu (source: prepared by the authors).

On the other hand, when the user selects the Video option from the Menu, a video file will be displayed, showing the operation, in extenso, of the heart (see Figure 7). Finally, when the user selects the Learning Menu option then the system will show a didactic test where the student will have the experience of experiencing and practicing, through the AR, the functioning of all parts of the human heart. In this didactic test, you can accumulate the evolution of your learning and different levels of learning will be shown, such as a video game (Figure 8).

Figure 7.

Result when selecting the video option from the menu (source: prepared by the authors).

Figure 8.

Result when selecting the menu learning option (source: prepared by the authors).

In this didactic test, a series of questions are asked for student self-evaluation, where you can appreciate the evolution of their acquired knowledge. This evaluation contains questions with dichotomous answers (the questions were implemented with JavaScript events carried out on the SDK platform), which serve as input for the system to show the score obtained by the user (Figure 9).

Figure 9.

Didactic test carried out by AR (source: prepared by the authors).

6. Results and tests

To evaluate the system presented in this work, computers were installed in a school where medical learning units are taught: 2 text sheets were provided to 28 fifth-semester students to learn about the functioning of the human heart and, subsequently, they were shown the system developed in this work. Since the students had the opportunity and experience to know the system, a survey was applied. The purpose of this survey was to know three assessment parameters (according to the objective of this research and whose concept implies that students learn by doing) about the AR-based system: learning, motivation and ease of use. Likewise, the level of knowledge retention was evaluated with respect to the aforementioned variables, which was carried out by comparing the two ways in which students studied the functioning of the human heart, that is, with and without AR. Table 1 summarizes the responses of the students surveyed regarding; (1) the self-assessment score in the system; (2) how they felt with the use of the system and; (3) how accessible or complicated the system management seemed to them.

	With AR	Without AR
Significant learning	89.3%	35%
Motivation	94.6	35.7
Easy to use	93.8	96.6
Performance	82%	60%

Table 1.

Results for the AR-based learning system (source: prepared by the authors).

7. Conclusions

The creation of a new learning platform, through the AR, will allow medical students, particularly anatomy and physiology, to obtain quality knowledge, as well as a correct approach to new technologies for timely execution. of your activities either in an operating room or in your classroom. To acquire the skills of know-how, medical students must learn to perform medical procedures with living or dead people, these processes can be simulated with AR. A platform with these qualities is a means of training for each student to evaluate their own learning, before doing it with living people.

There are multiple complements to make RA a more interesting and attractive experience. Among them we can highlight the lenses for AR (which can even handle and display certain data when used), helmets adapted with visors for AR and surround sound, or smartphones of the latest generation with AR. In virtual or mixed environments, you can use gloves, screens, or rooms equipped with specific objects to feel and/or appreciate the reality extended virtually. Undoubtedly, there are countless options to generate new environments, whether educational or not, where the AR is the main tool for its development. These complements and new environments will depend on the budget available for their implementation, development or creation, but, in general, their creation is not too expensive. What is clear is that these environments favor and facilitate the learning process of students at any educational level.

Through this work, the following is verified: (1) AR is a useful and easy-to-use tool that works to build suitable learning environments, which allow students to feel motivated, encouraged and eager to continue learning; (2) With the support of the AR systems can be generated that help the learning of abstract or difficult to perceive knowledge; (3) Medicine handles models whose appearance and form help and strengthen learning, which can be represented with three-dimensional entities (3D); (4) The interaction with the AR and the multimedia materials that are added to the physical reality allow the sensory senses of the human being, and in particular of the students, to be progressively stimulated, thereby achieving that the student learns in an auditory, visual and kinesthetic; (5) For its part, the design cost will depend on how much is available to invest in accessories and/or complements to show the AR, but the programming and design of virtual reality (three-dimensional (3D) design, simulation and page web) are generally not very expensive. However, an advantage of the proposed system is that it could be reused, adapting new knowledge and free three-dimensional (3D) models on the web. Finally, mention that the challenges of the AR for educational environments are vast and transcendental. However, a contribution of this nature allows cementing the foundations to broaden the current horizon of learning and create a new mosaic of knowledge.

Acknowledgments

The authors also thank to CONACYT for the postdoctoral grant of Dr. Gerardo Reyes Ruiz.

Conflict of interest

The authors declare no conflict of interest.