In today society, one of the most demanded challenges faced by the current educational system is the educational response to diversity in the various educational contexts. University lecturers are opening new lines of research focused on issues as social demand and current reality of produced new learning environments. The general aim of this study was to design learning environments using immersive virtual reality and evaluate improvements produced by this tool in relation to the difficulties show by the participants. From that point, an action plan was created to recreate school situations with a high degree of realism and interaction using IVRSystem. In this way, we want to obtain answers according to the dysfunctions of educational system to work with these students. This was done by a mixed design. On the one hand, a quasi-experimental methodology was used with a control group and an experimental group. On the other hand, direct and observation and applicative methodology made possible the development of educational intervention in immersive learning environments. The results obtained throughout these years have given a response to the initial problem-question raised: Can immersive virtual learning environments serve as a support tool for working disabilities of students, which have a visual learning style, such as students with autism spectrum disorders?
- learning environments
- immersive virtual reality
- autism spectrum disorders
- technological systems
This chapter aims to meet one of the most demanding and complex challenges of the current educational system: the educational response to diversity in the various educational contexts. Based on their research work, increasingly connected with the social demand and reality, university lecturers are opening lines of research, which deal with these issues. In that sense, our proposal consists in the implementation of new learning environments through immersive virtual reality (VR), from an interdisciplinary (technological and pedagogical) work oriented to face diversity, associated with disability, of students with autism spectrum disorders (ASDs) in this case, and seeking to achieve the following overall aim:
To design learning environments for the purpose of recreating school situations by means of immersive virtual reality and assessing the improvements brought by immersive virtual learning environments with regard to the difficulties posed by the participating students.
After an initial section dedicated to reviewing the literature and all the research works on virtual technologies, the second section provides a description of the most important tools used to design immersive virtual environments as well as the structure of the software developed. An analysis is subsequently performed about the contributions, which can be used for the intervention with students who have deficits in communication skills from immersive virtual reality—as opposed to the desktop one. The third section lists the innovations and works carried out by authors based on the utilisation of the aforementioned technological tool. In addition, the present work will finish with a number of proposals for improvement meant to ensure further progress in this area from the results obtained and their educational implications, along with the bibliographic references used.
2. Review of the literature on virtual reality
Virtual reality (VR) has been used to plan, practise and implement behaviours and to observe the responses within a computer-generated virtual context. These systems offer a three-dimensional representation of real, controlled and safe environments that can be executed in a repetitive manner
A number of early works [1–8] already highlighted the advantages of virtual reality as a tool to create predictable learning environments. Other subsequent and more recent works [9–24] have verified the advantages of virtual reality as a support and help tool for students with deficits in communication, social and emotional skills, and more precisely for students diagnosed with autism spectrum disorders (ASDs).
An aim is consequently set at this stage to undertake a review of all the research carried out on virtual technologies and the advantages that they bring to students with communication difficulties. Szatmari et al.  already started in their first research studies that the utilisation of virtual reality with ASD students made it possible to obtain great academic achievements in a relatively independent way. Along the same lines, aspects characterising VR such as the design of a strictly controlled environment and a highly individualised intervention are the advantages considered by  for its use with students who have communication problems.
This review helps to confirm the increased possibilities of this virtual tool, which can be used to teach ASD children to improve their understanding of other people’s mental condition, authors [3, 5, 27–29] even claim that children with ASDs tend to have normal cognitive skills and do not show delays in their language. Authors [8, 30] advocate an intervention through computer-assisted learning based on presenting the information in such a way that the potential confusion and the anxiety induced by the manifold sources which characterise the real world for such students can be reduced. Research following the same approach was undertaken by Brown et al. , who designed a city-like virtual environment with different buildings, which the user had the chance to explore, simultaneously putting into practice the various skills of everyday life.
All the research works mentioned so far, which were based on desktop virtual environments, together with others [3, 27–29] pursue the aim of teaching children with ASDs to achieve a better understanding of other people’s mental condition. This perspective serves to justify that desktop virtual environments provide the best possible method to train social skills and, as argued by Parsons et al. , the ability to understand other people’s behaviours, as well as the interpretation of language in the terms utilised by the speaker, can be trained by means of computer-based tasks in which it is possible to monitor the level of input stimuli received by the user, and sharing a series of characteristics with the real world through the use of sophisticated graphics and design. These types of learning can even allow the transfer of learned skills.
The research carried out in the 2000s reinforced a positive response to virtual environments on the part of users, insofar as the use of avatars may provide them with the ability to predict emotions , as well as show that they remembered the knowledge acquired during the sessions . These authors see VR as an additional tool for the utilisation and training of social skills in the classroom and perhaps for an increased use of the already existing methods and investigations.
Another of the aspects characteristically associated with VR utilisation has to do with the fact that it offers a safe working environment for students with communication problems and that a need existed to achieve a generalisation of the learned skills . Along these lines, Mohamed et al.  focus their research on the use of platforms for the design of environments. The model designed revolves around tasks to be worked on within a scenario where children had to follow with their eyes a path that the environment indicated to them and any error resulted in the system making a noise. It was possible to check that, thanks to virtual environments, children managed to improve their attention, even though the virtual model designed could not be validated.
The utilisation begins with a kind of social stories in virtual environments combined with video modelling for the purpose of enhancing and developing ASD students’ conversation skills . A bet is also made on a VR understood as attractive and easily administered sophisticated training packages aimed at promoting learning through different contexts , continuing with a proposal to make virtual environments more realistic, reproducing the great complexity represented by the human face and achieving the most complete possible reading of the user’s features . These authors used the TEACCH method, which includes large quantities of visual materials, and each area was structured in such a way that it had some visual information as well as the instructions related to the beginning and the end of the task.
The progress made in the research initiatives implemented shows an absence of adverse psychological responses to virtual environments by students with ASDs, exactly as it happens with typical development children . Furthermore, when the environment has greater realism, children pay much more attention to the contents presented. These authors also insist on the need for animations to be adapted to children’s preferences and for virtual scenarios to be more realistic so as to interest children to a far greater extent in their contents. Their findings ultimately lead them to point out that generalisation problems appear with such environments and that, in certain situations, the user is unable to interpret the avatar’s intentions.
Advances are made in the incorporation of another tool with VR, robotics in virtual environments, as an element that can improve ASD students’ social skills, thus seeking to monitor the child’s look and the social interaction distance. It became obvious that ASD students exponentially increased visual contact at the moment when the robot came closer to them, unlike what happened with the control group , but without forgetting that any variation whatsoever in the virtual environment generated changes in the child’s psychological and social ratios. Another line of research advocates the identification of VR as a platform for social discussion and interaction with other children, which offers the possibility of achieving improvements in the social skills of ASD students .
It was subsequently checked in some results that what had been worked on in virtual reality could be transferred to the real world if highly structured procedural tasks with rules were used, while at the same not forgetting that these programmes had to be suited to the user’s specific characteristics and individual capabilities . Other research studies  applied virtual reality for ASD students as a tool which can facilitate the real-time recognition of emotions (recognising the emotions of others as well as their tones of voice); the theory of the mind (recognising and responding to other people’s thoughts and wishes); and the capacity for conversation (initiation, maintenance and closing)—obtaining high scores in recognition and the theory of the mind thanks to VR utilisation.
Likewise, the paper by Lorenzo et al.  suggests a set of protocols with immersive tasks to train social skills. The results confirm that certain problems arise in their execution at first but also that a considerable improvement as well as a certain transfer of the learned skills take place as those tasks are gradually performed in the immersive environment. Recent research works, such as the one undertaken by Wallace et al. , have stressed the need to provide environments with greater realism, since that would allow children with ASDs to perceive the negative responses, which are sent to them by the environment, thus helping them to enhance their social skills. The current study performed by Lorenzo et al.  constitutes an advance with respect to what has hitherto been reviewed; it stresses the fact that the immersive virtual environments designed to work on emotional competences with ASD students have permitted a high degree of interactivity, along with a chance for the training of social roles which are represented in emotional scripts prepared like social stories. Their findings have similarly revealed medium rates for the transfer of learning from the virtual environment to real situations. Didehbani et al.  equally used three primary domains to work with virtual reality amongst students diagnosed with ASDs: recognitions of emotions, social attributions and attention and executive functions. The results obtained show an improvement in recognition of emotions, social attribution and the analogical reasoning of the executive functions which are largely due to the social-interaction-related advantages brought by virtual reality.
3. Virtual reality systems developed in learning environments
This section is going to offer a review of possible software and hardware architectures used to work with students, on this specific occasion, with students who have problems regarding communication and social skills as well as executive functions, as is the case of students with ASDs.
A set of tasks are presented in a learning environment where children with ASDs must respond to instructions given by a computer or by a/some person/s . The results obtained by these authors show that the increased level of motivation and the reduction of inappropriate behaviours, which had been observed during the instruction stage, is unfortunately not accompanied by a significant improvement in learning. Along the same lines, an interactive computer program is utilised to teach ASD students reading and communicative skills . Such virtual types of learning reflect improvements in the child’s vocabulary during the training period and a high degree of motivation and interest towards the planned tasks as well.
One of the first bets on immersive virtual reality can be found in the work of Szatmari et al. , who implemented a virtual immersive reality where children wore a helmet equipped with two video cameras in front of their eyes—and the point of view changed when the children moved. The learning tasks focused on the children’s learning to cross the street in such an immersive environment. That required a previous process of adaptation to the environment. The results show how children learned to cross the street, but a number of problems arose when the environment parameters were modified. To those advances must be added the contributions made by Strickland , in whose work children could use a pair of 3D vision glasses and a low-cost tracking system with their PC in the classroom. This made it possible for children to learn to know what to do in each situation and to be able to take that learning to the real world right away.
Other systems used were: a dialog program by means of bubbles where the user can think and talk to the main characters of various social situations ; virtual desktop environments aimed at allowing ASD children to have a working environment where they can learn to organise the tasks that these children must do at home after returning from school, with two modes: the ‘training’ mode, very passive and in which the child had to familiarise with the environment; and the environment and mode referred to as ‘late’, where the child carried out the activities and navigated and interacted with the 2D and 3D objects present in the environment .
Continuing with desktop virtual reality, computer games are proposed with eight problems related to social situations that the user had to solve, additionally suggesting alternative solutions . Ten training sessions alternated with six test sessions were developed in the previous works. In the training sessions, the tutor explained which specific solutions corresponded to the different situations raised. The findings showed improvements in the utilisation of problem-solving techniques. Following this same approach, the paper by Cheng et al.  suggests using a software called KidTalk, which allowed the child to interact in very rudimentarily represented real situations where the child participated through a text chat. The results obtained are quite satisfactory, even though a recommendation was made both to broaden the sample and to extend the intervention time.
Desktop virtual reality thus clearly seems the most often used for interventions with these students, and another example can be found in its utilisation by Leonard et al.  to work on decision making: knowing what to do when the time comes to make a choice (a place which was a café or a bus). The scheme programme is complemented with some previous sessions during which the children watched a video of the real situation on which they had to work in the virtual environment. The design of a three-dimensional animated head called ‘Baldi’ additionally served as a kind of avatar, which provides a realistic and visible feeling of realistic speech through which children with ASDs could learn vocabulary . It is checked that vocabulary learning takes place, its generalisation to highly structured situations being largely due to the avatar and not only to speech.
The work of Pioggia et al.  enriches virtual environments with the insertion of a new element: the use of a robot to practise social and emotional skills. In the first part of the session, the same as in the studies performed by Leonard et al. , the children viewed several real situations, after which they interacted with an android and their behaviours were recorded in another room. Along the same lines, Moore et al.  use desktop virtual reality as a collaborative environment for the recognition of emotions. Tasks such as the recognition of what has been expressed by the avatar, the identification of the emotion suited to the context and the identification of the context according to the emotion expressed were used to that end.
It also becomes obvious that the school environment reality increasingly merges into desktop virtual reality. This is illustrated by Vera et al. , who designed a whole school to be used with ASD students, to perform all the tasks carried out therein, taking into account their realism, interactivity and adaptability to the difficulties that these children pose. These authors express a certain degree of imaginary play when it comes to the responses given. Herrera et al.  subsequently added two touch screens similar to what immersive virtual reality might look like, with the aim of working on imagination and symbolism in the social situation of going to a supermarket. The programme is complemented with a number of tests meant to assess the improvements achieved after carrying out the intervention. The results show satisfactory improvements amongst students with ASDs. Taking up the work , interventions are performed in decision making, as exemplified by learning to cross the street, but increasing the number of users; a positive transfer of learned skills is verified .
Another of the software programs used is the iSET desktop application, which permits to record emotions in a variety of social situations so that they can be subsequently tagged, Madsen et al.  along with the creation of a laboratory virtual environment where the child wears a cap with a receiver that sends the emotional information to a website; likewise, the child uses a glove which has as its aim to generate different pressures on the hand . Other authors  resort to desktop virtual reality, reproducing a virtual class and a scene outside the classroom with the aim of training social competences: recognition and expression of feelings, non-verbal behaviours, visual contact and a suitable way to listen to others. Significant improvements become evident in the results obtained with this program.
The utilisation of the FEW program based on the film Alice in Wonderland for the purpose of detecting the possible changes experienced by the character as the story progresses , as well as the Mind Reading system through a library of emotions , together with the FaceIT software, by means of which users identified the changes operated in specific facial expressions , complete the contributions of the desktop virtual reality oriented to the development of emotional competences in students with ASDs.
The paper by Wallace et al.  revolved around the non-utilisation of any type of device, such as glasses or the like. Two working groups were formed: one with ASD children and the other with a control group. Users worked in three scenarios: in the first scenario, the user drove along a real road and all the other cars drove in the opposite direction, the children observed without interacting; in the second scenario, the child will act as the observer of a social situation and will have to react to it; and in the third scenario, the child will have to walk along the corridor of a school, thus being able to witness the different activities in progress. The innovative works of these authors, based on their results, and despite expressing a good connection between virtual and real images, reveal problems linked to interaction and context generalisation.
Nevertheless, as an element of support for desktop virtual environments, Bevilacqua et al.  proposed in their paper the utilisation of a Webcam automatic algorithm to measure levels of disability depending on the expression of emotions. Other new aspects introduced to improve such programs focus on using avatars, the Computer Expression Recognition Toolbox (CERT) facial recognition library and the Emotion Mirror system , even though a number of problems appear with regard to the contextualisation of emotions, insofar as the life experiences presented in desktop virtual reality do not have the same degree of interaction and realism as immersive virtual.
Amongst the first bets on immersive virtual reality stands out the work of Lorenzo et al. , who advocate the use of immersive virtual reality incorporating a series of improvements such as the collection of data not only to evaluate students’ behaviour but also for its training and potential improvements, as well as the function of system sensors when it comes to determining if the task performed was properly developed in the virtual environment, which permits to know the extent to which social skills and executive functions have improved. This line of work initiated by the aforementioned authors was extended with the study undertaken by Matsentidou and Poullis , who designed a cave in immersive virtual reality by means of four HD screens, four projectors and cameras, unlike the previous authors, who used an immersive L. These authors ultimately wanted the virtual reality cave to be effectively used with ASD children, checking whether or not children were able to benefit from the application of the acquired knowledge to their real life. Unlike what happened in the previous case, the system proposed was not tested and should be implemented in non-school-related situations.
Amongst the most current papers dedicated to the new advances in virtual reality stands out the one by Zeng et al. , where immersive virtual reality serves to develop the attention of ASD students, something which had not been proposed in this type of system yet, but had actually been carried out in desktop virtual reality. As for new research initiatives, the work undertaken by Lorenzo et al. —which continues along the lines of investigations already initiated with immersive reality—transfers their contributions to the training and development of tasks such as the recognition of emotions amongst students diagnosed with ASDs. These authors prepared social scripts as well as situations in the style of social stories that these students had to identify and solve in immersive learning environments.
Another of the contributions to immersive virtual reality has recently been made by Newbutt et al.  with the aim of simplifying the installations utilised in the classical immersive virtual reality systems, choosing the OCULUS glasses—an HDMI device which, unlike those previously used by Strickland et al.  and Strickland , needs no cables. Environments are designed in a personal computer that provides a stronger feeling of immersive thanks to its being equipped with a set of loudspeakers—which the 3D glasses used by Lorenzo et al.  did not have. The authors mentioned above point out that the ASD children readily accepted the OCULUS glasses to work and were additionally able to solve the tasks proposed successfully.
4. Virtual reality systems developed in learning environments
The bet on immersive virtual environments for students who have a deficit in communication and social skills, more precisely, who suffer from ASDs, is supported on two basic premises: (a) the characteristics of their cognitive style, which requires an explicit kind of teaching by means of visual aids as well as a highly structured environment and (b) the possibility to exploit the advantages that such environments bring us in terms not only of interacting, instructing and practising quasi-real school situations but also of transferring the practised learning to the school environment . The design of highly structured virtual environments can constitute an educational innovation as well as a learning strategy that can be raised as a challenge and an innovation to be applied with disabled students, and more specifically with ASD students, characterised for being essentially visual learners who find it easier to retain and assimilate visual information.
The basic research question of this study—Can immersive virtual environments serve as a learning tool for disabled students with a visual cognitive style?—was taken as a starting point to formulate the following specific aims:
To design new immersive learning environments to train skills associated with the executive function and the improvement of social and emotional competences with ASD students.
Utilising Immersive Virtual Reality (IVR) as a personalised support tool within a structured visual environment.
Evaluating the level of generalisation of the acquired learning to the school environment.
4.1. Design and procedure
The specific aims set provided the basis for an action plan through an immersive virtual reality system used to recreate school situations with a high degree of realism and interaction with the user seeking to obtain the responses desired according to the dysfunctions faced. This was done by means of a mixed design: on the one hand, a quasi-experimental one with a control group and an experimental group; and, on the other hand, an applicative direct and systematic observational methodology that made it possible to develop the educational intervention in immersive learning environments. The methodological design used implies a change of paradigm, insofar as the aim sought is not to evaluate dysfunctions but to carry out an intervention with them, additionally assessing the extent to which they improve through the use of immersive learning environments. These are the stages implemented during the last few years:
Initial stage. Problem-questions are proposed as a starting point.
Planning and design stage. The attention is focused on the design of immersive environments as well as of the information collection instrument. These immersive environments must permit not only the identification and recognition of disabilities but also their training and improvement.
Implementation stage. Users are made to practise and experiment by means of immersive virtual environments.
Analysis stage. It consists in the study and examination of the results obtained and their educational implications, assessing the improvements provided by immersive virtual environments regarding disabilities in comparison with desktop virtual environments.
4.2. Our immersive environment
The immersive environment used  was created through the arrangement of two screens in an L shape where the different scenarios were projected with a wide-angle lens in order to reduce the projection distance—and therefore the space requirements too. The user’s immersive is achieved using a pair of active glasses of the brand Stereográfics (CristalEyes model), which allow users to have a 3D feeling as something real in front of their eyes. The user additionally wears a cap with light-emitting diode (LED) that will be detected by means of infrared cameras. A Mikrotron MC1324 GigE camera was also used for the detection of the child’s expressions in the immersive environment. The addition of loudspeakers and a high-fidelity amplifier permits to hear any kind of voice, noise or song according to the needs, which may eventually arise in the virtual environment. Thanks to the HP Z800 Work Station—which includes all the necessary software components—it will be possible to generate the scenarios, distributed to both projectors through a video signal, showing their content on the screens. The system is completed with the monitor, the immersive virtual reality generation module, the Vizard program and the data capture module, along with the visual control performed with the manipulating robot. Furthermore, a voice recognition system served to check when the user was carrying out the task, and whether the volume and intonation of the voice were suited or not to the situation presented. Figures 1 and 2 provide a sample of the elements shaping the system described above.
The study undertaken through several stages [42, 44] had students diagnosed with ASDs from the primary and secondary educational stages as its participants. The control group included students who were working in school centres that applied the traditional support methodology used to deal with their difficulties in social skills and executive functions such as emotional competences; in turn, students with similar characteristics formed the experimental group, but their intervention took place in the immersive learning environments designed. The sample was shaped in the first stage with 10 primary education students from public schools located in the city of Alicante (Spain) and a second group of 10 students from secondary schools also located in Alicante; and in a second stage, with 20 randomly chosen children who had to carry out the tasks in the IVRS and a second group, the control group, 20 children, chosen randomly, will carry out the tasks in the VR.
4.2.2. Instruments: immersive learning tasks
Taking the aims established as a reference, priority was given to making a proposal of activities in the immersive learning environment. These were the instruments designed:
-First stage. The focus on work with executive functions and social skills led to design: the TEVISA support task protocol; THE PIAV avatar instructional protocol; and the monthly interview with the teachers of the schools while the aforementioned protocols were implemented. This protocol was used to suggest the students a set of tasks referred to executive functions as well as social competences, and associated with situations at a classroom level in which disabilities were present. The implementation of immersive tasks followed the two-step process described below:
Previous task. Identifying the situation (space, avatars and materials); description of the task to be performed; task instructions and self-evaluation.
Support task. Identifying the situation (space, avatars and materials); following the instructions and carrying out the task: answering the questions about the task performed.
An evaluation using the PIAV protocol took place while the tasks described above were carried out: body motor coordination control; voice control; look control; attention control; and empathy control. The discussion groups created by the participating teachers on a monthly basis assessed the evolution of students’ behaviour in the school tasks, which resembled those undertaken in immersive environments.
-Second stage. Focused on the identification, training and development of emotional behaviours amongst students diagnosed with ASDs. A script of emotional tasks in the form of 10 social stories was designed for the purpose of identifying the emotions implicit in those different social stories and the training of appropriate emotional responses when facing the social situations posed. The students had to identify what the avatars did, where they were and how they felt in the situation which arose, after which attention was paid to the management and training of the situation. The control group performed the tasks based on the social stories in the desktop virtual environment, whereas the experimental group did so in the immersive virtual environment. Figure 5 show one girls carrying out the immersive tasks with social scripts.
4.2.3. Results and conclusions
The results obtained throughout these years  have given a response to the initial problem-question raised: immersive virtual learning environments can serve as a support tool to work on the disabilities of students who have a visual cognitive style—as is the case of students with ASDs.
During the first stage of our study, and even though students showed some confusion at the beginning of the sessions when it came to following task instructions, it was checked that they improved the understanding both of the actual tasks that they had to perform and of the instructions to do so. Despite the absence of high percentages (scores) for Response Category No. 4—which corresponds to tasks being carried out at a highly acceptable level—it became clear that the tasks were acceptably performed even if some confusions might appear. Amongst the data confirming the internal consistency rates for each one of the virtual blocks and environments created in the application of the TEVISA task protocol stands out the fact that reliability rates range between 0.68 and 0.91. As for the results obtained by both primary and secondary school students, overall they show a gradual but significant increase in students’ behaviours in the different PIAV blocks, and even though the maximum values were not reached at the end, an improvement in the behaviour of the students involved was empirically verified . The findings additionally reveal a gradual but relevant reduction of inappropriate behaviours as the intervention sessions are undertaken in immersive environments. The results obtained in the school context by primary school students show an average 2.23 out of 4 in their initial process of learning generalisation from virtual environments to the classroom, whereas secondary school students’ average in this respect is situated at 2.5 out of 4. Averages above three are eventually obtained as work develops in the immersive environment. The participating teachers express the average progress achieved and applied in the classroom context.
As verified by the authors , the findings in the second stage of the present study show emotional behaviours increasingly suited to the situations proposed in the emotional protocol tasks, along with a significant improvement obtained by the students who worked in the immersive virtual environment compared to those who did so in the desktop environment. Hence, the confirmation of the higher degrees in the resolution of emotional responses in the immersive environment than in the desktop one, even though no significant differences appeared in the desktop control group with respect to the immersive one at the beginning of the sessions.
From the very beginning, the present paper advocates the use of educational innovations coming from virtual technologies applied to disabilities. Our suggestion is to keep moving ahead and to extend the implementation of immersive virtual methodologies to other types of disabilities with the aim of achieving a standardised use of such technologies within the dynamics of the teaching-learning process.