Learning by Augmented Reality: Cluster Analysis Approach

Because the use of augmented reality (AR) is increasing, it is important to study its possibilities within both formal and informal learning contexts. We clustered 146 sixth graders using AR at a science center based on their reasoning, motivation, and science learning results using the self-organizing maps method (SOM) to identify AR-using subgroups. The aim was to consider reasons why the AR method could be of more beneficial for some students than others. The clustering results complemented earlier findings on AR gains in learning, as an unexpected response to intervention was discovered using this nonlinear analysis. The previous results had indicated that after the AR experience, science test results generally improved and particularly among students with the lowest achievement. The SOM-clustering results showed a majority group of boys, especially those interested in science learning both at school and at the science center using AR. Despite low school achievement, their high motivation led to good science learning results. The prior results, according to which girls closed the science knowledge gap between boys after using AR, became more relative, as two girl-dominated subgroups were identified. The reasons for the results were considered on the basis of motivation, multimedia learning theory, and concept formation theories.


Introduction
Augmented reality (AR) technology offers possibilities to demonstrate complex phenomena in a novel way. At its best, the novelty of AR makes it an effective servant [1], but on the other hand, it sometimes has been shown to increase cognitive load due to bad practical usability and also because the tasks used are too complicated [2]. The AR advantages can be theoretically understood through multimedia learning theory, which explains how blending virtual contents into the real world can support brain functioning in cognition and learning [3]. The theory stresses the use of pictures in learning instead of just words [4]. Afandi et al. [5] elaborate on the theoretical points further as applied to AR by replacing pictures with real objects and words with symbols and virtual text. From the sociological perspective, the AR method can likely enhance the fulfillment of the essential idea of big principles and ideas of science education [6] and advance understanding about science even for people who otherwise would remain outsiders. AR can be viewed as a great example of tools which, for their part, pave the way for attaining the twenty-first-century competences [7,8].
In the Finnish national core curriculum [9], the twenty-first-century competences are called transversal competences, which include seven areas. The area with 1. What kind of subgroups and results complementing the previous study can be identified by clustering the AR-using students based on cognitive reasoning, motivation, science interest, and knowledge learning test results?
2. How are boys and girls and students achieving differently in the school environment represented in the subgroups?

Participants
The participating 146 pupils were 11-13 years old, and 51% (n = 75) of them were girls. They were from seven schools from the Helsinki Metropolitan Area in Finland.

Context
The pupils visited a typical science center exhibition, which included five AR technology-supported exhibits. They were (1) the Doppler phenomenon, (2) Boltzmann's molecule movement, (3) the Young experiment, (4) the airplane mini wing exhibit, and (5) rolling double cone.
The context of the study was formed as an open learning environment consisting of AR equipment (Figure 1), hands-on exhibit (Figure 2), experimenting with small-scale real objects (Figure 3), and testing AR demonstrations (Figure 4).
The photos above are showing just one case related to flying. Also all the four other topics were taught based on the same pedagogical principle: the mixed reality as an open learning environment was formed by bridging the gap between virtual AR technology, real hands-on objects, and interactive learning by science center exhibition objects.

Deci-Ryan motivation. A self-determination theory (SDT)-based SRQ-A
questionnaire was used to examine relatively stable academic motivation (32 test items, Likert scale 1-4, α = .92). The SRQ-A test includes a formula [18] based on which the relative autonomy index (RAI) was calculated. It describes the overall autonomy level experienced by the pupil. It was only applied as a pretest ( Table 1).    3. Learning context. School vs. augmented reality in a science center. Contextspecific interest was measured in the school context vs. the informal science center context by applying the semantic differential method [19] (14 pairs of adjective alternatives, Likert scale 1-5, pretest α = .81, posttest α = .88).
4. Raven test. The Raven standard progressive matrices [20] were used to test cognitive visual reasoning. The test contains 60 items divided into 5 sets (A-E). Each of these groups contains 12 tasks (α = .79, 60 items).

5.
The knowledge tests. These consisted of 31 items related to the content areas of the school curriculum of the science subjects, and these contents were combined with the AR solutions in the science exhibition. The questions were piloted 2 months before the actual preknowledge test. The post-knowledge tests were conducted 1 week after the science exhibition visit. In the test the pupils were asked to assess whether they thought the knowledge statements presented were correct or incorrect or whether they were uncertain about them (pretest, α = .72; posttest, α = .77, 31 items).
The background variables were gender and school achievement, for which we used four school grades (physics, chemistry, mathematics, and mother tongue). The students were grouped into three groups based on achievement (1 lowest quartile, 2 + 3 quartiles, 4 highest quartile).

Statistical analysis method
The pupils were clustered on the basis of their scores by applying the selforganizing maps method (SOM) [21], a neural network model [22] which is based on unsupervised learning of fuzzy logic. Compared, for example, to K-means clustering, the advantage is that within the SOM cluster, the nearer one pupil ends up to another, the closer the likeness between them is. In the K-means cluster, the neighborhood does not count, and the pupils are merely listed in the cluster, and cluster membership is the information obtained [23][24][25].
The SOM method has been widely applied internationally, especially in biotechnology, economy, and technical industries [26]. In social sciences the applications are rare, although it has been shown to have promising possibilities in educational and learning research [27,28], in the area of psychology, for example, relating in early language learning [29] and in sociological research [30].
Using the SOM method, the goal was to identify subgroups particularly benefitting or non-benefitting from AR. The data of the cluster were transferred to SPPS 25. The statistical significance of the difference between the theoretically expected and observed number of students in each cluster was tested using the chi-square test, and the adjusted residuals (criterion: absolute value ≥2) were used to pinpoint the overor underrepresentation in each cross-tabulated cell. The differences between the clusters (dummy variables: each cluster vs. all others) were compared using one-way analysis of variance. The change between the pre-and posttests results was analyzed using the general linear modeling method (GLM repeated measures) and its effect size by the partial η 2 -coefficient (interpretation: >.01 small; >.06 middle; >.14 large).

Results
Using the SOM method, five clusters were obtained. When the clusters were crosstabulated against gender and school achievement groups, the result was that boys and girls were not represented equally as expected in the clusters (χ 2 = 18.63, p < .001) and that was also true with the achievement groups (χ 2 = 25.38, p < .001). The statistical descriptives are presented in Table 2 and the 95% confidence plots of the knowledge test results (correct, incorrect, uncertain) in the two time points by cluster in Figure 5.
In order to illuminate how the science knowledge test results looked like before and after the science center visit intervention, and the change, we present the 95% confidence plots of the knowledge test results. They are divided into correct, incorrect, uncertain answers in the two time points by cluster in Figure 5.

Cluster 1 (n = 26; 18%): motivated, low school achievers, boy majority
Significantly more boys (adjusted residual = 3.2) and lowest in-school achievers (adjusted residual = 2.2) than expected. When dummy variable cluster 1 was compared to all the others, in cluster 1, cognitive reasoning was lower; situation motivation and interest in science both in the school and in the exhibition were higher. In the pretest there were more incorrect answers. The correct answers increased (η 2 = .43) and incorrect ones decreased (η 2 = .44) after the science exhibition and AR-assisted method.

Cluster 2 (n = 29; 20%): high achievers
Cluster 2 was not gendered; there were an equal number of boys and girls. However, there was a strong representation of in-school highly achieving students (adjusted residual = 3.8). In comparison, in this cluster the pupils had the highest cognitive reasoning scores, higher situation motivation, and interest in science learning at school than the others. They had more pre-and postknowledge correct answers and fewer incorrect answers at both time points. The correct answers increased significantly (η 2 = .16), and the uncertain answers decreased (η 2 = .26).

Cluster 3 (n = 32; 22%): motivated but non-learners, girl majority
In cluster 3, the different school achievers were equally represented, but gender played a role: girls were significantly more represented than boys (adjusted residual = 2.6). In the dummy comparison, students in this cluster had a higher autonomy experience and situation motivation than others. It was notable that in the post-knowledge test after the exhibition, their incorrect answers were found to even increase (η 2 = .24) and were the highest scores of all clusters but uncertainty decreased (η 2 = .26).

Cluster 4 (n = 35; 24%): non-motivated by exhibition but learning
In this cluster, the background variables made no difference: there were an equal number of boys and girls and different type of school achievers. The dummy comparison revealed that they had the lowest autonomy experience (RAI) and situation motivation. They were less interested in science learning in school and especially in the exhibition than others. In the preknowledge test, they had more incorrect but fewer uncertain answers than others. After the exhibition the correct answers increased (η 2 = .39), incorrect (η 2 = .23), and uncertain ones (η 2 = .18) decreased. In this cluster there was an overrepresentation of girls (adjusted residual = 2.1), a little less so than in cluster 3. In the dummy comparison cluster 5 versus others, these students were found to have a lower autonomy experience, lower situation motivation, and interest in science learning. In both time points, they had fewer correct and incorrect answers in the knowledge tests than others. However, the most striking difference was that both in the pre-and post-situation, they had more uncertain answers than others. In this cluster the knowledge test results did not change significantly from the pretest to the posttest.

Discussion
Previous studies [14] have shown that AR usually enhances learning in an effective way, despite some results that show that the effect seems to fade in the long run [11]. Similar results were found in our earlier study [15]: in general AR improved learning results, and this was supported by interest and situation motivation, especially among boys. In a girls' group, in turn, experienced autonomy was an important explainer of science learning results. Importantly, the lowest school achievement group especially was found to gain from the use of AR technology. By clustering the same data, we could detect deviations from these general tendencies, i.e., relating to the role of reasoning, school achievement, motivation, and science interest in knowledge learning.

Low in-school achieving pupils can also have high learning results when they act in an informal learning environment
The cluster analysis revealed that good school achievement is not the only factor leading to motivation in science and to good learning results. Although cluster 2 with an equal number of boys and girls was formed, in which everything (reasoning, school grades, interest in science at school, motivation, and knowledge results) was optimal, we also found that prior school achievement and cognitive reasoning were not totally deterministic factors, and one of the most encouraging result of the clustering study was that a deviant group (cluster 1) from the general rule could be identified. In this cluster, boys were the majority, who, in spite of low school achievement, were especially interested in science learning both at school and at the science center using AR. This led to good science learning results subsequent to the exhibition. Even though they had more incorrect knowledge answers in the beginning than the others, their interest clearly supported learning, as correct answers increased after the AR experience. This result deviates from most meta-results obtained from informal learning contexts [31,32].
It seems that the idea of using AR-supported learning was successful in its goal to introduce abstract phenomena in a concrete way. Learning by doing and personal experimentation made a crucial link between theory and practice possible, and deduction and induction were combined in a pedagogically effective way. That the pupils were interested showed up both in that they found the science-centered environment attractive (situation motivation) and in that they were deeply engaged in the theme contents (intrinsic motivation) as they learned the scientific knowledge.

Experienced lack of choice, low autonomy, and low motivation may unexpectedly lead to high results in tests-but at high costs
The somewhat puzzling and unexpected finding relates to the role of autonomy and situation motivation in learning. The general rule, which shows that a low Learning by Augmented Reality: Cluster Analysis Approach DOI: http://dx.doi.org /10.5772/intechopen.91252 autonomy experience and situation motivation usually lead to less learning and worse learning outcomes, did not materialize in the fourth cluster. In that group the students learned, although they were less autonomous than all the others, less motivated by the AR-assisted learning situation, and in general less interested in science learning overall. The number of correct knowledge answers increased, while incorrect and uncertain answers decreased. This result opens up an interesting theoretical point.
According to the SDT theory [33], nonautonomous motivation is based on the avoidance of sanctions, on hope of rewards, or on experienced pressure. It has been found that the learning results of externally motivated and, thus, less autonomy experienced students might remain more superficial and short-term than those of their more intrinsically motivated classmates. Previous research further indicates that if a student acts because she or he feels anxiety and pressure, psychological energy is consumed for defense of self, and less energy remains for learning new things [34,35].
Our earlier path-model results, according to which the girls closed the science knowledge gap between boys after using AR, became more relative, as two girldominated subgroups were identified.

Experienced autonomy and motivation usually-but not alwayscorrelate with learning
In cluster 3, with the girl majority, the pupils experienced more autonomy than others and were also motivated by the AR experience, but in both pre-and postsituation had incorrect conceptualizations-they even increased after the science center visit-in contrast with the boy-majority group just described. When, on the other hand, the low rate of uncertainty of knowing found already in the pretest further (unrealistically) decreased after the science center visit, one might wonder what the reason was: why did the experienced autonomy and being attracted by AR use in the exhibition not lead to self-correction of the wrong ideas and learning? Theoretically, AR usually seems to be a practical tool to support the Kolb learning cycle, which starts by being exposed to a concrete experience, following by reflective observation and further transferring to abstract conceptualization and finally leading to active experimentation [36].
Perhaps the phase of reflective observation remained superficial, and/or the abstract conceptualization phase failed [36]. In the research on learning concepts [37,38], it has been observed that if one builds concepts on incomplete information and only partially, these misconceptions are later very hard to change-most probably even in interesting contexts such as in the AR-connected science exhibition. The big challenge for science teaching is, thus, to identify possible misconceptions and partially formed concepts and to make a necessary return to the earliest phase of the conceptualization process.

Low autonomy, no interest in AR, much confusion showing as uncertainty,
and lowest learning results-as expected, but how to intervene?
In cluster 5, the pupils, with a girl majority, were highly uncertain both before and after the AR experience. While the students in this group were less autonomous and motivated by the exhibition situation, their lowest knowledge results were only as one could expect. Theoretically [36], interpreting the result of this cluster, these pupils (i.e., especially because there were girls in this group) have not been successful in creating meaning from their AR-and science-centered learning experience, and therefore, the whole experiential learning cycle process has been interrupted. The worst conclusion is this failure may be only one in a series of previous failures.

Concluding remarks
These results are of interest especially because they add information about different types of pupils who use AR for learning. Some of them clearly somewhat unexpectedly seem to take advantage of the use of AR as a pedagogical tool more than many of the others, even though the preconditions of low school achievement and reasoning skills would have predicted less optimistic AR-learning gains. On the other hand, the results also illuminate situations in which pedagogical intervention would be advisable: an essential notion was that unrealistically, high expectations might arise for AR usage based on superficial observation about the seemingly motivated pupils. In those cases, pupils might be autonomously and eagerly engaging in the task, but in reality, it remains unnoticed that they do not necessarily have enough guidance to make correct conclusions based on their AR experiences.
The most vulnerable group of pupils from the perspective of the big principles and ideas of science education [6] was, however, those on whom the AR experience showed to have no or little effect. Despite the fact that they were fulfilling the expectation based on low motivation and interest proposed by the multimedia learning theory [3,5], one could have hoped that AR as a novel method would have been more successful. The result is most worrying: what could be done differently, if the novelty effect, a new method to change abstract concepts to more concrete ones, and a fresh, untraditional learning environment were not working? At least we could design the AR-learning situation even more carefully [39].
Perhaps one way would be to assure that AR technology really is as easy as possible to use and that there is enough guidance at the start and support available during the whole process. Simultaneously, it should be assured that the goals are realistic and simple enough. In addition to that, Cheng and Tsai [2] suggest care should be taken that the numerous possibilities of AR do not become too overwhelming and lead to cognitive overload, as short-term memory resources are limited.
Most of the research around mixed reality has focused on technical and practical issues or an evaluation of usability. Educational research on learning aspects has already produced some useful meta-articles on the strengths and weaknesses related to augmented reality [14,40]. Digitalization has doubtless changed our everyday life-also related to learning. However, the research-based evidence, especially related to the latest, brand new technologies also tend to give false promises or create ambiguous future visions [41] (Säntti & Saari, 2017). Accurate results are needed not only for more meaningful learning but also to create cost-effective solutions and to avoid wrong, often expensive investments.
This study related to learning outcomes has produced some new evidence based on cluster analysis supplementing the earlier findings. Making an invisible phenomenon observable is clearly the strongest input of AR-especially when it offers an opportunity to learn an abstract and difficult topic in a concrete, observable way. Mixed reality combines visible elements with an already existing realistic environment and makes it more understandable. Augmented reality is applicable and works best when teaching real and restricted learning contents, and thus, it is really a challenging superficial "phenomenon-based education." Although one has to keep in mind that the process is not straightforward, there are many intervening factors relating to motivation and factors of self; encouraging possibilities emerge through the AR method. One of the most promising results is that this type of intervention and learning method really can support low-achiever students to close the gap on other students. However, it also provides opportunities and challenges for high-achieving students. It also seems to give valuable opportunities for bridging the gap between formal education and informal learning.