Learning Mathematics in an Immersive Way

Abstract

In this work we introduce a learning system based on Virtual Reality and Augmented Reality for studying analytical-geometric structures that are part of the curriculum in mathematics and physics in high school classes. We believe that an immersive study environment has several advantages with respect to traditional two-dimensional environments (such as a book or the simple screen of a PC or even a tablet), such as the spatial understanding of the concepts exposed, more peripheral awareness and moreover an evident decreasing in the information dispersion phenomenon. This does not mean that our teaching proposal has to substitute the traditional approaches, but it can be seen as a robust tool to support learning. In the first phase of our research we have sought to understand which mathematical objects and which tools could have been used to enhance the teaching of mathematics, in order to demonstrate that the use of Virtual Reality and Augmented Reality techniques significantly improves the level of understanding of the mathematical subject being studied by the students. The system which provides for the integration of two machine levels, hardware and software, was subsequently tested by a representative sample of students who then provided feedback through a questionnaire.

1. Introduction

Virtual Reality (VR) and Augmented Reality (AR) are technologies that since their inception have sometimes suffered fluctuating fates, sometimes due to the lack of suitable low-cost hardware, sometimes due to the inherent complexity of the technologies adopted. With the advent of mobile technologies, their fates have radically changed and today we have both low-cost hardware and software approaches that make them widely usable in many areas of modern life. Nowadays, we find VR and AR everywhere, in manifold applications: from entertainment [1, 2], teaching [3, 4], tourism [5, 6], manufacturing [7, 8], networking and communications [9, 10], microelectronic and high performances hardware industries [11, 12], e-commerce [13], medicine [14, 15, 16, 17, 18, 19].

In this work we are going to focus on the adoption of VR and AR technologies teaching mathematics, analyzing the feasibility of the process and analyzing the usability of the implemented software platform.

During our work, we were guided by a specific goal: the possibility of giving a real and visual form to the abstract objects of mathematics. This represents a further development in the visual representation of mathematical concepts, which in the course of history has been evolving from the primitive use representations of simple counting objects, such as the tally sticks, through the elegant structures of the symbolic algebra of the seventeenth century, to the imposing constructions of mathematical analysis and modern geometry, to get to the current and amazing views of numerical analysis through computer graphics. In this context, we have set out to investigate the possibility of enhancing students’ understanding of the concept of link between an algebraic-set structure and its geometric representation on an orthogonal Cartesian space (concept best known with the name of function).

In the first phase of the work, our attention has been directed to a certain number of functions, that are used in senior high-school classes, such as the representation of trajectories in the Cartesian plane and simply surfaces in the three-dimensional space.

The very first proposed functions are as follows:

$y=cosx$ The cosine function is well known to students who currently use it to solve trigonometric, analytical and geometric problems. Moreover, it is also present in all fields of physics in order to model the behavior of several phenomena in mechanics, thermodynamics, optics and electrodynamics, just to limit our analysis to some main cases [20]. In Figure 1 is shown the VR representation of such function.

Figure 1.

Cosine wave plot.

$z=e^{-x}$ The exponential function is one of the most important functions in mathematical analysis, whose applications span to different fields of knowledge and is well studied in high school. Exponential functions can be used in many contexts, such as the compound interest in finance, to study the evolution of the population growth and the radioactive decay [21]. In Figure 2 is shown the VR representation of the named function.

Figure 2.

Exponential function plot.

$z=ln\left(x^2+y^2\right)$ This function is an example of bi-dimensional logarithm, useful to describe astrophysical objects. In Figure 3 is shown the VR representation of the named function.

Figure 3.

Bi-dimensional natural logarithm plot.

$z=\frac{sin\left(x^2+y^2\right)}{x^2+y^2}$ This function is an example of bi-dimensional dumped sine, useful to describe objects in fluid dynamics, electronics and telecommunications. In Figure 4 is shown the VR representation of the function.

Figure 4.

Dumped sine plot.

2. Related works

Stimulating multiple sensory dimensions during the learning process of a concept or idea makes the learning itself more effective, because our mind needs to experience the reality all around by multiple experiential levels. Indeed, it is essential for us to look, listen and touch in order to build a complex conceptual structure that bears and strengthens our knowledge about reality itself [22, 23]. This multi-sensorial learning is also important in special-need situations, such as autism [24, 25] or dyslexia [26, 27].

Compared to traditional two-dimensional learning systems, such as a book or the simple screen of a PC or tablet, an immersive study environment has several advantages, as it gives the spatial understanding of the concepts exposed, more peripheral awareness or more useful information bandwidth and decreasing the information dispersion phenomenon [28].

The natural sciences have pioneered the use of Virtual Reality techniques to facilitate the learning of their subjects: chemistry [29, 30, 31, 32, 33], biology [34], physiology [35], physics [36].

An effective approach to enhance math skills, knowledge and competences inevitably involves the creation of dynamic digital environments rich in captivating content, which would even reduce anxiety and improve results [37].

Mathematics is known to be one of the most difficult subjects for many students. A recent study asks questions about traditional way of learning and recommends more active and attractive learning approaches [38].

Furthermore, many several studies have shown that immersion in a digital environment can significantly improve education in several ways, because, as mentioned earlier, multiple perspectives are activated at the same time [39]; this fact has an important impact in all areas of mathematics [40].

In some works it has been pointed out that many students find very difficult to relate a function’s analytical form to its relative graph, despite the mathematical simplicity of the concept of the function itself [41]; it almost seems that the intermediate layers existing between the set concept and the analytic-geometric one prevent a clear understanding of the link [42, 43].

A recent study has shown that a deeper understanding of the analytical-geometric links existing in functions, if supported by specific software, can help students to develop a positive mindset towards mathematics itself, in terms of attitude, motivation, interest and competence [44]. But we assume for what is the current state of the art that more research is needed to grasp the profound implications that exist.

3. The virtual world made on Unity3D, blender and Vuforia

In this section, we explain how to represent a three-dimensional function within a virtual world created with Unity3D.

The creation of the figures is carried out through two scripts working on different levels:

1. at the beginning it is important to define the resolution, that is the level of detail the figure must have. If for instance we set a resolution equal to 500, then we will have a matrix of 500x500 function values obtained varying X and Y along the grid. The implemented code produces in output the list of vertices that make up the figure.

2. then, the list of triangles must be calculated, since they are the basic elements in computer graphics. Indeed, they specify how the points are interconnected and how they should be represented on the screen.

3. the three-dimensional mesh is generated from the triangle vertices. To do this, a generic script has to be written, which can be called from all the script programs of the Unity3D project.

Once the first phase has been completed, the surface lighting needs to be adjusted, to optimize the appearance of the represented shape, with respect to the user’s camera position. This calculation, which seems to be complex, is carried out very quickly in Unity3D. At the end of this operation the mesh is ready to be shown on screen.

4. The architecture of the system

The software used to create the scenarios is Unity3D that allows the composition of virtual environments starting from basic elements which the scene is composed with. It also takes care of rendering, real-time lighting calculation and user interaction management. The fundamental tools that have been used are the following:

• the game objects

• the scripts

• the colliders

The game objects are the basic elements that make up the scene that must be shown to the user.

The scripts are code files written in C# language that permit some predefined tasks to be executed, like managing the appearance of objects on the displayed area, or the camera movement as soon as a key on the keyboard gets pressed.

The colliders are objects that prevent intersection or collision between the character controlled by the user and other different objects in the scene.

Two consecutive C# scripts are tasked with generating all the necessary shapes and modeling any mathematical function in two or three dimensions:

• the first script generates the vertices

• the second script receives in input a list of vertices and draws a three-dimensional figure

Generally, we are interested in drawing mathematical functions with a continuous domain and relative range in $\mathbb{R}$ (Real Number Set); however, this is only an ideal case, since plotting a function through a calculating machine requires simplifying the set of calculations by passing through a discrete space of points. In other words, we need to trace a grid of points that are going to end up defining the level of maximum detail we want to achieve. Also, we must keep in mind that the higher the level of detail, the more calculations the user’s device ought to perform to display the object on the screen. A balance must therefore be found between the graphic quality to be obtained and the computational complexity necessary to achieve similar results. In fact, defining a grid of points is equivalent to defining a sampling frequency: if the number of samples is too low, we can in fact obtain” aliasing”, getting this way an imprecise representation of the mathematical function we want to show.

In order to improve performance, it was decided to proceed according to this path:

• the shapes generated with Unity3D are saved inside the filesystem so that they can be reloaded directly when the program starts, without the need to recalculate all the objects from the beginning every time

• the shapes have been elaborated with Blender, a software designed for the elaboration of three-dimensional models and objects, in order to reduce the polygonal complexity without modifying the information content

• the elaborated shapes are included again in the Unity3D project

In other words, the complexity of the figures in terms of vertices has been reduced but this is absolutely not perceived by the user who observes them without noticing any difference.

The Augmented Reality (AR) environment uses the Vuforia framework. The program created is an Android Package Kit (APK), which can be installed on Android smartphones with 7.0+ operating system. Vuforia is a Software Development Kit (SDK) that enable the users to analyze the video stream recorded in real time by the phone camera. Vuforia enables the creation of a database of markers (called Vumark), manually associated to the game objects of the scene.

5. Usability of the system

In order to increase the usability and effectiveness of the platform [45], two different modes have been created for user experience: one uses VR only, the other uses AR. In both cases the graphic engine used is the same: Unity3D.

In the Virtual Reality case, a room has been created, and inside the room the three-dimensional figures have been positioned, as shown in Figure 5. This software allows the composition of virtual environments starting from basic elements called Assets which make up the scene. The VR environment is generated and compiled by WebGL technology: this means that the application is compatible with all devices (computers or smartphones) on the market since the virtual world can be viewed with a web browser, regardless of the operating system used (Windows, Linux, Android, iOS, etc). In particular, in the experience we made for collecting the student’s usability evaluations, we focused on smartphones with Android operating system. The graphic quality of the scene adapts according to the computational power of the device, while remaining undemanding in terms of hardware requirements. The scene can be observed through a virtual reality viewer, such as HTC Vive, or through a normal computer monitor. The user has the possibility to move around the virtual environment using the touchscreen on the smartphone, or the mouse and the keyboard on a computer. Inside the environment are visible three-dimensional geometric shapes that support the learning of mathematical functions that are otherwise difficult to be drawn (see Figures 6 and 7).

Figure 5.

Students during training session to learn how to use the platform.

Figure 6.

Evaluation of the experience by the students.

Figure 7.

Environment for the VR experience.

As far as the use with Augmented Reality is concerned, Vuforia software has been used. It is a framework integrated in Unity3D and allows to create projects that use Augmented Reality by providing all functions essential for operating on mobile phones. The system is very dynamic, since every time one of the markers in the database is framed by the user’s device camera, Vuforia warns Unity to show the object on the scene (and therefore on the user’s screen) associated with the framed Vumark. Furthermore, this SDK manages the spatial orientation of the object based on the user’s position relative to the Vumark. If we frame a Vumark and move around it, the object associated with it will also rotate, allowing us to appreciate it in a realistic way (see Figure 8).

Figure 8.

Example of Vumark to draw the function $\mathit{ln}\left(x^2+y^2\right)$ (on the left) three-dimensional image generated by Vuforia for the object described by the previous Vumarks (on the right).

6. Discussion of results

We collected students’ feedback to evaluate the usability of the proposed system and to enhance the user’s experience, making the AR and VR environments more interactive and attractive. This is the reason why we asked high school students to express their opinion about the quality of the experience, by filling out a questionnaire. Our sample set was composed by 90 high school students, homogeneous by age, gender, social and cultural levels. In Figure 9 is shown the distribution of the sample per age, while in Figure 10 is shown the distribution per gender.

Figure 9.

Distribution by age of the sample of students.

Figure 10.

Distribution by gender of the sample of students.

The aim of the experiment conducted during a morning class was to obtain a set of coherent and indicative answers on the following main topic: how much virtual reality and augmented reality can impact the perception of mathematics. To do that, each class of the school composing the sample, was taken to the computer room, equipped with Vmarks, and left it playing with the application on both computers and smartphones for 50 minutes. Finally, the students filled in the questionnaire.

One of the most remarkable aspects of this experience was to observe so clearly and explicitly how the introduction of a play environment stimulated girls and boys in an incredible way. It seemed that the game scenario by itself made them feel comfortable and willing to experience approaching mathematics in a new way.

The results have been being very promising, as most students found the experience of VR and AR applied to some mathematical functions very useful and instructive. Some even asked for an enhancement of the experience, with the addition of new entities and animations.

In Figure 11 the degree of appreciation of Virtual Reality, i.e. the exploration of the virtual world on a PC, is shown. It appears that the usability has been really appreciated, a bit worse were the results in terms of easy of use, user experience and graphic quality.

Figure 11.

Degree of appreciation of virtual reality.

In Figure 12 the degree of appreciation of Augmented Reality, i.e. the exploration of the virtual world on a smartphone, is shown. Also in this case the usability has been really appreciated, while a bit worse were the results in terms of easy of use, user experience and graphic quality.

Figure 12.

Degree of appreciation of augmented reality.

The result about comparison between Virtual Reality and Augmented Reality evaluations are very similar, as it can be seen in the Figures 11 and 12.

We think that, if this approach would be largely adopted by teachers, the interest of students on the discipline may significantly raise up and the score of the students may significantly increase, especially in the case of students with problems approaching mathematics.

7. Conclusions and future works

When we first approached our research we had to understand which mathematical objects and which tools to use to enhance the teaching of mathematics, starting from the assumption that the use of AR and VR techniques greatly increases the level of understanding of mathematics.

Our goal has ever been just from the beginning to give students an environment of objects to study and focus on, by selecting among them those of greatest interest and didactic utility for themselves and the entire class. Moreover, we have led our research towards the field of immersive learning, in particular those applications that allow the user to be immersed in virtual worlds in order to increase brain stimulation during the learning phase.

At the same time it has been essential to understand the degree of absorption and emphatic response of students to the system: sensations, disturbances, emotions. At the current state of work, the system can only display objects that are statically compiled: we are working to make the system more dynamic and responsive, which allows the math teacher to draw graphs, two-dimensional or three-dimensional, without the need to print each time a new Vumark, on precompiled functions.

The goal is to get a dynamic platform that lets us understand how the choice of a function and a complete immersive experience in the mathematical object itself (including its specific characteristics and properties) impacts students’ learning.

Acknowledgments

Our most heartfelt thanks go to the teachers of the Institute of Higher Education (ITAS) Giordano Bruno of Perugia, Italy for the efficient and exquisite collaboration provided.

A particular thanks goes to the students who have been involved in a very constructive experience and have responded diligently to the questionnaire.