The Immersive Power of Augmented Reality

Abstract

Augmented reality is one of the technologies that have received great attention and interest in recent years. In a world where the boundaries between the real and the virtual are blurring, this immersive technology enriches and complements the reality with digital content and allows people to gain a complete and real sense of the objects around them. Currently, the applications of augmented reality go beyond the domains of games and entertainment and are aimed at taking the full advantage of the technology in areas such as medicine, architecture, business, tourism, education and more. The current paper presents the essence of technology and types of augmented reality systems. The basic approaches for creating augmented reality applications are discussed. Specific examples of the application of the technology in the field of education are given-an augmented book and augmented reality educational projects, whose purpose is to make learning an interesting, immersive, engaging and motivating process.

1. Introduction

Technological innovations in the information society have a profoundly transformative impact and significantly change the way people and companies carry out their activities.

According to [1], among the leading technological trends for 2020 is extended reality (XR). This new term encompasses several popular technologies-virtual, augmented and mixed reality-that guarantee immersive experiences. They blur the boundaries between the digital and physical world. The digital world is not just a reflection of the physical environment where people live and work, but it complements the real world with new, more personalized and sensitive sensations. The connection between real and virtual is becoming increasingly narrow and inextricable.

Virtual reality completely immerses users in the digital world, detaching them from the surrounding reality. Augmented reality mixes the real-world physical environments and computer-generated virtual objects and enhances users’ perceptions of reality. Mixed reality offers a new environment where physical and virtual objects exist and interact in real time [2].

Augmented reality directly engages and provokes user interaction with both the real-world objects around them and the generated and provided virtual content. Users can use different devices such as smartphones, tablets and wearable devices to view the content that augments physical world.

The world is keenly interested in augmented reality and the technology is expanding into different areas. More and more people, working in the field of education, are setting the question whether it is possible to use augmented reality as an effective tool to realize the ideas of new pedagogical paradigms and engage the learners’ attention. This question is provoked by the effect and success of the game Pokemon Go, which is the most striking example of the application of augmented reality. It was the game that made the technology popular and put it in the public spotlight.

Education is constantly changing to keep up with the trends in society. The main reason for changes is not emerging or improved technologies, but the new learners with their specific needs and requirements. Technologies are just tools to create the necessary learning environment where the training can be carried out in the most efficient way.

Augmented and virtual realities create conditions for the realization of the ideas of the immersive learning. Immersive learning supposes that learners acquire new knowledge and skills in an environment rich in sensations, perceptions and emotions. The major benefits of augmented reality are connection and interaction between all participants in the learning process, physical objects and virtual content.

Increasingly, augmented reality-based educational applications are being offered on the educational application market. Their goal is to engage learners’ attention, enhance their participation in training and help the perception of abstract scientific concepts by specifying and connecting them to the real world.

2. Augmented reality technology

Augmented reality plays the role of the bridge that connects the digital and real world. It does not replace the physical world, as virtual reality does, but complements, expands and enriches the real physical environment with layers of computer-generated content. The technology provides interactive, accessible and digitally manipulative information about real objects. It transforms the way people interact with virtual content and creates the preconditions for engaging all human senses in perceiving the physical world.

Ref. [3] defines augmented reality as a system that:

• combines real and virtual objects. In augmented reality systems, virtual and real objects coexist at the same time, in the same place.

• offers real-time interactivity. Users can interact with virtual content, which responds to their actions.

• allows registration of virtual objects in the physical 3D world. There is a geometric alignment of virtual objects with real ones in the physical world.

Augmented reality enhances users’ perception of and interaction with the real world. With technology, any real object can be enriched with additional information, which sometimes cannot be detected and perceived directly through the users’ senses.

The term augmented reality was introduced in 1990 by Tom Caudell. The technology has already had a long history. In 1968, Ivan Sutherland invented the first head-mounted display model, which, despite its weight, is a step toward the use of augmented reality in the real world. In 1992, Luis Rosenberg created the first functioning augmented reality system Virtual Fixtures, designed to train pilots in the US Air Force. The commercial application of the technology dates back to 1998 with the introduction of yellow first down line. In 1999, Hirokazu Kato created ARToolKit-a library for developing augmented reality applications. In 2014, the launch of Google Glass set the era of the wearable devices, which gave a strong impetus to the development and spread of augmented reality technology.

2.1 Augmented and virtual reality: common goal, different approaches

The reality-virtuality continuum, proposed by Paul Milgram, is a continuous scale, varying between reality (a physical space with no virtual elements) and virtuality (a virtual space with no physical elements). The area between the two endpoints is mixed reality-a space where real and virtual objects are combined in different ratio. It is clear from the scale (Figure 1) that augmented reality is closer to the reality than to virtuality, since it is a physical space with integrated virtual elements.

Figure 1.

Reality-virtuality continuum (adapted from [4]).

When discussing augmented reality technology, analogies are invariably made with virtual reality. The two technologies have a common purpose-to immerse users and give them a complete and real sense of the world around them, but they achieve the goal in different ways.

Virtual reality replaces physical world with a digitally created one. Users are completely immersed in the artificial environment and cannot see, feel or interact with the physical world around them. Virtual reality is one of the technologies ensuring that people are completely focused on situations in the digital world. But the technology detaches them from the physical world and is not possible to make a connection between what is happening in the digital world and the surrounding reality.

Unlike virtual reality, augmented reality creates virtual objects that overlay the physical world and provide additional details for real objects. It allows users to be in touch with the real world and interact with real objects and people as well as with computer-generated content. The basic idea behind augmented reality is to immerse users in the physical world and give them more information about what they see, but are not able to perceive through their own senses.

2.2 Types of augmented reality

There are different classifications of augmented reality according to various criteria.

Depending on the hardware devices that users use, there are the following (based on [5]):

• Stationary augmented reality systems. They are equipped with powerful cameras and can provide accurate recognition of objects and scenes from the reality. Examples of stationary systems are the virtual fitting rooms created in physical stores.

• Spatial augmented reality systems. They are also known as video mapping or projection mapping. Spatial systems project virtual content in actual size and proportions directly onto real-world objects. Many companies from automotive industry use such systems to present their new models. On the other side, video mapping show programs are receiving great attention from consumers.

• Desktop augmented reality systems. These systems use the device camera to recognize objects or scenes from the reality and embed virtual objects, both displayed on the desktop screen. Many companies use desktop augmented reality to create virtual fitting rooms in their online stores.

• Mobile augmented reality. This type of augmented reality is extremely popular recently due to the widespread of mobile devices (phones, tablets and hand-held devices). Many mobile augmented reality applications use the global positioning system (GPS) capabilities of the devices to determine users’ position and provide and show information that is directly related to their location. The information comes in different formats (text, images, audio and video) and is integrated into the real environment.

• Head-mounted displays. This group of hardware devices delivers a combined image of the physical world and virtual objects. Depending on the used technology, head-mounted displays can be divided into optical and video see-through devices. The most popular representative of this group of hardware devices is smart glasses.

• Contact lenses. They are the future of augmented reality. Although they are still under the development process, Bionic Lens is already being tested.

Depending on how the objects or scenes from the real world are identified, the augmented reality is divided into:

• Vision-based augmented reality. The augmented reality applications recognize and interpret real-world images, scenes or objects. Vision-based augmented reality can be marker-based or markerless. Marker-based applications use the camera of the device to recognize and interpret markers, usually black and white barcodes. The software analyzes the marker and creates virtual objects that are displayed on the device screen integrated to the recognized real objects. Markerless technology uses real-world objects (photos, objects, scenes, etc.) as targets and the software recognizes them by specific features that distinguish them from the surrounding environment.

• Location-based augmented reality. It is a kind of markerless augmented reality. Applications use the GPS capabilities of the devices to determine their position and provide content that is relevant to the current location of the users-for example, information about hotels, restaurants, museums and others that are close to the users.

2.3 Augmented reality systems work process

There are four main stages in augmented reality systems’ functioning [5]: scene capture, identification, processing and preview.

The reality that has to be augmented with digital content is captured by video or optical see-through devices. The captured reality is scanned to determine the position of virtual content that augments it by markers or tracking technologies. When the scene becomes recognized and identified, relevant virtual content is required from different sources. The final step is to create a mixed image of the real scene and embedded virtual content.

Users need the appropriate software to view and interact with digital content that augments the physical world. They can use augmented reality browsers or specific applications.

3. Augmented reality: an innovative learning tool

Recently, augmented reality is a popular tool used in various fields of business, advertising and tourism. There are many examples of successful augmented reality-based campaigns. Some companies choose this technological approach to increase their sales, attract more customers and guarantee their social presence. Others use augmented reality to create a personal and emotional connection between the products and services they offered and potential and actual customers. Some companies rely on unforgettable authentic experiences that can be offered to consumers through augmented reality applications and subsequent positive results from such campaigns.

Augmented reality can improve the perception of the surrounding real world with new sensations and perceptions, which is a prerequisite for a better understanding of the physical world [6]. This is of utmost importance for the education sector. Augmented reality makes possible learning in the real world and changes the way learners interact with the objects around them.

Present-day digital learners, on the other hand, are keen to use technologies from their daily lives in the learning process. Like any technology that is able to engage learners’ attention and provoke their interest and activity, augmented reality is a subject of interest for teachers.

The main questions that should be asked in an emerging idea of implementing new technologies in education are as follows:

• Why to use them and what positive effects can be expected?

• How they can be used to support the realization of dominant pedagogical paradigms?

3.1 Why to use augmented reality in education?

The concept of immersive learning has become more and more relevant in recent years. Immersive learning is a theory based on the use of technologies and technical tools to create an educational environment, where learners are immersed in new worlds, learn through different perceptions and senses and have first-person experiences [7]. The idea is to place learners in new situations or environments that offer different sensations and experiences and create conditions for more curious and motivating learning. Technologies that help realize the ideas of immersive learning are virtual reality and augmented reality.

With augmented reality, the training is carried out in a different type of learning environment, with opportunities to enrich the learning content and acquire new skills and competences.

The technology creates conditions for building a new type of learning environment that combines reality and the virtual world. The boundaries between them are blurred and a united learning space is created that is rich in possibilities, perceptions and sensations and offers new ways of acquiring knowledge and skills [7]. Learners are placed in an authentic learning environment and provided with a real-world experience [8, 9]. Every real object is enriched with dynamically changing information. Learners may receive additional information that depends on their specific actions with objects and each interaction can cause a different reaction, which can be a source of new knowledge for them [10]. The interaction with real and virtual objects is a way to realize active learning.

Learners can interact, explore and experiment with virtual objects, discovering their properties and behavior-something that is not always possible in the real world or is not safe. There are many examples of using augmented reality in training-from studying the structure of molecules and atoms, exploring the universe, the planets and the Earth, to acquiring knowledge for the structure of the human body and the functioning of human organs and systems. Learners are active participants who take control of the interaction with digital content in real time. They have the ability to take decisions and actions in order to achieve the learning objectives [7]. Such kind of learning approach can facilitate the understanding of abstract or complex concepts through their concretization [11, 12]. At the same time, it provokes imagination and creativity and increases the learners’ interest and motivation.

In a hybrid learning space, learners can interact not only with the real objects and the digital content that augments them but also with other participants [7]. The learning environment allows the implementation of some of the approaches of modern smart pedagogy, where learners are active, motivated and engaged participants-learning by doing, applying a research approach to training and working in a constructivist environment, context-based learning.

Today’s learners are digital learners. They were born and live with technologies and prefer to study in the digital environments. Most of them spend more and more time using various electronic devices mainly for chat or games. A serious problem with the reluctance of young people to read books and textbooks is arising. Parents and teachers face difficult questions about how to encourage teens to read more. Integrating augmented reality to traditional books can help solve the problem and generate engagement and fun in reading.

The integration of augmented reality technology and traditional books creates the so-called augmented books. They look like traditional books, but when readers point their smartphones or tablets to pages, the content comes to life. 3D models, video, audio, animations and interactive elements appear. Through technology, print materials are enriched with digital multimedia information, which can make learning content easier to read and more engaging.

Augmented reality supports the acquisition of practical skills and experience through training in a mixed environment and simulations. To be prepared for the real world, learners must have not only knowledge but also practical skills and experience. Augmented reality applications allow them to actively develop their skills by working, manipulating and experimenting with both virtual objects and 3D models of real objects. Augmented reality helps different situations and problems of practice to be recreated, which resolved by learners leads to the acquisition of practical skills and experience.

The ability to manage their own learning stimulates and motivates learners. They can discover new properties and behavior of objects and processes, following their individual research approach. Augmented reality is a technology that is able to inspire learners, spark their interest, stimulate their creativity and curiosity, create positive emotions and attitude to the learning, and enhance their motivation. These are the necessary prerequisites for a positive change in the educational process and its orientation toward digital learners’ needs.

Augmented reality is a promising technology that has its place in education. The listed benefits make it a preferred tool for both teachers and learners. The technology enables innovative forms of training and learning, and supports the realization of new pedagogical paradigms.

Augmented reality also has some disadvantages that must be taken into account in order to use technology in the best possible way in education. The technology requires the availability of smart devices, which can put learners in an unequal position. Technical problems of different nature are possible, caused by device camera malfunctions, lack of high-speed Internet connection as well as software issues. On the other hand, it is possible that augmented reality applications may distract and divert the learners’ attention from the learning materials and cause undesirable reactions and results. Developing appropriate educational applications is a difficult process. It takes time and resources and requires an innovative approach to present virtual content as well as to choose tools and approaches to access and interact with it.

3.2 How to use augmented reality in education?

Augmented reality can be integrated in different ways in training. It can be used to create augmented textbooks or to develop augmented reality educational applications.

3.2.1 Augmented textbooks

Nowadays, interactive books are very popular because they make reading an active process. Augmented books are a subtype of interactive books.

Augmented book is a traditional (paper) book with additional virtual content, which is accessible through appropriate hardware and software tools [13]. The augmented books enhance the traditional ones with interactive visualizations, animations, 3D models, and audio and video materials. All these multimedia components improve reading and engage readers’ attention [14]. A very important feature of augmented textbooks is that they allow learners to interact with virtual content, explore and make experiments, which guarantee active learning, better and easier perception and understanding of complex and abstract concepts. They can have a positive impact on learners’ achievements and attitude toward their own learning [15].

Augmented textbooks have a significant advantage over other e-textbooks. People can read them as traditional books, but at the same time, they have an access to additional interactive digital content. There is no need to buy special equipment to use them since a smartphone or tablet is required to visualize integrated digital content-devices that learners use every day.

Augmented reality gives new dimensions to traditional textbooks, which many digital learners consider being boring. The technology can turn textbooks into an interactive tool that engages learners, motivates them to read and learn, as well as entertains them. Much more learning content can be included in augmented textbooks that exceeds the requirements of the curriculum. The additional content is accessible to anyone who has the need or desire to deepen his knowledge.

Figure 2 shows an augmented book Bugs: Interact with Augmented Reality Creepy Crawlies (by Hannah Wilson, Carlton Kids Publisher). After scanning the pages, readers can see 3D models of different species of bugs and interact with them. Magic Books, Animal Kingdom Education Book, Animals Encyclopedia, Secrets of the Ocean, iStorm: Wild Weather and Other Forces of Nature, Dinosaur 4D +, Live Solar System and many others are good examples of books and textbooks that are created with augmented reality technology. Many of them target younger learners to provoke and stimulate their desire to read. Among the augmented textbooks that are used in secondary schools, colleges and universities are the statistical software for processing experimental data, cutting tools, Ethnobotany Workbook, Imagina Books: Human Body and others.

Figure 2.

Augmented book bugs: Interact with augmented reality creepy crawlies.

3.2.2 Augmented reality applications

Another approach to use augmented reality in education is to develop augmented reality applications.

More and more products created with augmented reality technology are available in the educational applications market. Many teachers choose such applications as an effective and powerful tool to help students better understand abstract concepts.

Augmented reality can represent objects and processes that are difficult to imagine and turn them into 3D models allowing learners to interact with them. The abstract concepts come to life and become part of the real world, which make them easier to learn.

Augmented reality can be used as an effective tool that “translates” theoretical scientific concepts into things from the real world and daily life, making them understandable to learners.

Physics, astronomy, chemistry, biology and mathematics are among the subjects where augmented reality technology can be used effectively. The various physical and chemical properties of the objects and the existing regularities can be easily understood with the help of augmented reality. The anatomy and structure of the human body and the functioning of human systems can be visualized by augmented reality in 3D space in biology courses. The properties of curves, surfaces and 2D and 3D geometric figures can be studied in mathematics and geometry classes. Trips to the farthest corners of the earth and the universe are possible, as well as going back to the past and tracking historical events and places.

Working with virtual objects can help learners to acquire and develop practical skills and experience in safe environment.

Figure 3 shows an educational application with augmented reality-Anatomy 4D that can be used to study human organs and systems. There are markers that need to be printed in advance and 3D models of human organs and systems are displayed after scanning them. The application allows learners to interact with 3D objects and digital content.

Figure 3.

Educational application anatomy 4D.

The combination of virtual and augmented reality can help to develop educational applications that immerse learners into new worlds with many different sensations. A good example of mixture of both technologies is Google Expeditions.

Google Expeditions is a learning application that allows learners to get to know the world, its history and culture through virtual reality or augmented reality tours.

In virtual reality tours, participants in the learning process use their mobile devices and Google Cardboard to immerse themselves in the digital world and explore the world, the universe, oceans, mountains, caves, galleries and museums without leaving the classroom (Figure 4). In augmented reality tours, virtual objects are integrated into the physical environment and they become part of the learning space. Training occurs in a shared environment, where interactions and experiments are possible under the guidance of teachers.

Figure 4.

Virtual reality tour with Google expeditions.

Augmented reality applications generate a great interest and desire to work, because they allow learners to be active, to be researchers and to control their own learning.

Regardless of the chosen approach of integration of augmented reality in learning, the technology creates prerequisites and conditions for acquiring new knowledge and skills in an environment that is rich in sensations, perceptions and emotions.

4. Tools for creating augmented reality educational applications

There are many augmented reality applications that are specifically designed for education and are free for use for both teachers and learners. Often, teachers prefer to develop their own learning content and materials that are designed and tailored to their learners’ goals.

Users can create augmented reality applications with various software platforms (many of them are free to use). There are two main approaches: using augmented reality browsers or augmented reality software development kit.

4.1 Augmented reality browsers

Augmented reality browsers are applications that display virtual geolocation multimedia content upon real-world objects. Browsers access remote resources through web protocols and services, index content through media streams and support various MIME formats [16]. Augmented reality browsers offer not only opportunities for publishing content that augments the reality but also tools for developing augmented reality projects.

With browsers, the process of creating augmented reality projects is quick and easy, and no special programming skills are required since templates are used and most of the procedures are automated. There are some limitations of created projects-recognition of only 2D images is possible and inability for interaction between users and virtual content. A major drawback of this approach is that there are no standards that allow augmented content to be interchangeable. Content created with the tools of one browser cannot be accessed through another browser [17].

Popular augmented reality browsers and their project creation tools include HP Reveal (HP Reveal Studio), BlippAR (Blippar Studio and Blipp-Builder), Zappar (ZapWorks Studio and ZapWorks Designer), Wikitude (Wikitude Studio) and more.

Different terms are used to refer to the created products (layers, channels and worlds), but the principle of action is identical-consecutive steps: (1) selection of the images-targets, which will be scanned; (2) add different types of digital content that will enrich the images; (3) review and test and (4) publish the project. After creating and publishing augmented reality projects, users (learners) must scan the images with augmented reality browser in order to visualize the digital content.

Figure 5 shows an augmented reality project created with Wikitude Studio. When scanning the image (a table with logical functions), learners can see a video material, which illustrates the results of the logical functions and their dependence on the values of the input arguments. Video material is created by Google Doodle and is accessible in YouTube.

Figure 5.

Wikitude studio project.

4.2 Augmented reality software development kit

Augmented reality software development kits (AR SDK) have fundamental features such as image, object and location recognition, position estimation and others. Among the popular AR SDKs are ARKit, ARCore, Vuforia Engine, Wikitude AR SDK, Kudan, AR SDK, EasyAR SDK and ARToolKit. They all allow 2D image recognition and are compatible with different platforms (iOS, Android, Windows and others). Some of them provide 3D object recognition features, while others help create geolocation applications. AR SDKs can be used to create augmented reality applications for smartphones, tablets and a variety of wearable devices. This approach results in developing stand-alone applications that users use to view virtual content that augments and enriches real objects.

Augmented reality applications, created with AR SDKs, are interactive, are feature-rich and embody the best of augmented reality technology. Their development is a complex and time-consuming process, requiring programming skills.

Figure 6 shows an augmented reality application BookStatistica, developed with Unity and Vuforia Engine. After scanning different markers, virtual buttons appear on the device screen. Learners can make a choice between different options-for example, video materials that show how statistical methods are done in different statistical packages.

Figure 6.

Virtual buttons in BookStatistica application.

5. Conclusion

The digital transformation is inherent in today’s society and is possible due to the diversity of information and communication technologies and comprehensive Internet access. The transition covers all spheres of people’s lives and is rapidly advancing into education. It is compulsory for the educational goals to incorporate ideas for preparing learners to work with digital technologies in the digital world.

Technologies in education can greatly facilitate all activities in the learning process. There are different ways to integrate them into training. They can be used to deliver traditional training and guarantee automating and speeding up of most of learning and administrative activities. But the most important idea behind the use of innovative technologies is to change the role of learners-to put them in the center of learning and turn them into active participants, who construct their own knowledge, follow their personalized learning path, and create and share content with other learners in an interactive mode.

Today’s learners are born in digital world. They use different devices and technologies in their daily lives but cannot successfully implement them in an educational environment [7]. Therefore, the teachers’ part is very important. They have to integrate technologies in the most appropriate way in the training, taking into account the needs and characteristics of learners in order to maximize their educational benefits.

Augmented and virtual realities have a great potential in the field of education because they can significantly change the way the learning is carried out and fill it with new sensations and perceptions. Augmented reality creates an immersive educational space that is a mix of real world and digital content and offers rich experience. Learners can work and interact with virtual objects that are integrated into the real world. They receive context-based digital information for each observed physical object or process. The use of the technology makes it possible for learners to understand complex and abstract theories and concepts much faster and more easily that inspires and motivates them. Augmented reality is one of the innovative technologies that combine attractiveness and effectiveness of the training. It brings to life print textbooks and turns them into attractive, fun and interactive tools that make reading and studying active processes. A significant advantage of augmented reality to virtual reality is that it overcomes the risk of learners’ social exclusion and insufficient social and communication skills since they are not isolated in virtual worlds and can interact with other learners.

Augmented reality is among the innovative technologies that are able to transform education into smart education to correspond to the developing smart digital society.

Conflict of interest

The author declares no conflict of interest.