Industrial Heritage Education and User Tracking in Virtual Reality

Abstract

Industrial heritage provides one of the most important records of social and technological progress and has international potential for education and development. This chapter presents the potential to use the virtual reality devices for informal education in technical and natural sciences. The hypothetical virtual appearance of an industrial power plant from the nineteenth century in Slovak city of Piešťany was intricately reconstructed by a combination of identified conserved valuable parts of the building and preserved original equipment and archival plans. This practical result—interactive virtual tool—educates about the lost heritage by allowing viewers to look closer and experience the former atmosphere of industrial work. During the virtual visits, users are motion tracked and invited to take photographs to mark the most interesting motives. Gathered data from this users’ observation were analyzed to find behavioral patterns and to give feedback information about the exhibition’s attractivity, used in further presentations.

1. Introduction

In museums of technology around the world, there are innovative creations of the human spirit that are no longer interesting. The current trend, therefore, is the development of interactive models of the presentation of natural laws and technology—those that are capable of making technology museums more attractive and of enabling the inspiring use of this rich source of knowledge.

Virtual reality (VR) is one of the most progressive, quickly evolving segments in information technology. The research team at the Faculty of Architecture STU BA systematically addresses the issue of virtual reality and seeks practical applications in several engineering areas and technical education [1]. The potential of virtual and augmented reality was identified and verified in terms of derelict monumental buildings—specifically industrial heritage.

“The concept of monument presentation was extended from prevailing historical or artistic significance to other meanings, such as less important architectural examples although important in terms of social or technological concept. These events often support industrial heritage” [2]. Industrial monuments are still highly regarded by society. Raising awareness of industrial heritage could help society to protect its history.

The case study introduced in this chapter is additionally a part of the research focused on the using of virtual reality as an analytical tool of design. This way is the exploration of new simulation techniques and educational qualities of virtual spaces connected to the gathering of information about users in defined spatial conditions and subsequent utilization of these data in the further design process.

2. Theoretical scope

Several historical buildings no longer exist but historical documents—archive technical documents, drawings, or photographs—have been preserved. Some buildings remain in the living memory of earlier generations or have preserved few physical fragments. Some have been irretrievable destroyed—removed and replaced by new buildings. This historical documents and preserved parts of the building may offer data for a virtual presentation of the extinct significant building or historical monument. The presentation of a hypothetical reconstruction by virtual reality can serve to bring the history, culture, and technology closer to the public [3]. The presentation of the digital model can serve as a graspable presentation of the extinct technical and cultural heritage.

In the game industry, the degree of “realness” is referred to the notion of immersion. Immersion is meant as transportation (presence) rather than as absorption [4]. The presumption is that new virtual reality environments with their high fidelity and real-time interaction have higher degree of immersion than conventional physical 3D models. The high degree of immersion of the VR environment is substantial in increasing the effectiveness and attractiveness of education, which is closer to our innate learning by experience. Besides that, with the rising of the degree of immersion, it is easier to compare user experience to real situation, allowing to gather relevant data about user behavior.

The user tracking within historically long spatial cognition research is based on K. Lynch’s image of the city [5] and W.H. Whyte’s urban cameras [6] and continues with later explorations as A. Mallot’s experiments with orientation in virtual environments [7] and C. Ratti’s real-time Rome [8]. These studies showed that user tracking is a high-quality source of information, describing certain behavioral patterns. All these experiments tracked human behavior to use it as feedback for further design. However, this new knowledge was not utilized in any practically used analytical tool, e.g., B. Hillier’s space syntax [9]. This tool, on the other hand, works only with theoretical outcomes of the spatial cognition; it is not implementing the real tracked data [10]. The case study in this contribution is focused on utilization of tracked behavior in analytical tool for evaluating further virtual exhibition designs and presentation spaces. For that further application, it is crucial to induce natural behavior of users during the tracking phase by introducing them the right motivation and reward. A motivation is behavior translated from the real world (e.g., exploration of real machinery hall). This motivation is supported by reward, which is related to the real; that is, it has impact on the real. As our previous experiments have shown, this system induces natural behavior of users and convinces them to interact and be part of the scenario [11].

3. Methodology

3.1 Research materials

This paper reviews implemented systems and describes application of VR in presentation of old power plant in Piešťany, which is Slovak industrial heritage (Figure 1). In this process, industrial archaeology methods were used, based on the archival research. Historical documents were obtained from the Austro-Hungarian Empire period topologic project from the National State Mining Archive. Documents of used technical equipment were obtained from the National Archive in Trnava.

Figure 1.

Picture of the virtual machinery hall with machine equipment—at the first stage of the power plant in 1906 and the machinery hall in 2014 (3D model: O. Virág, M. Ganobjak, V. Hain; photo: P. Safko, 2014).

The power plant for heavy oil burning in Piešťany was built in 1906 as one of the first of its kind in the former Austro-Hungarian Empire. Later, the plant only provided distribution and energy transformation till the 1990s. The original engine equipment was sold off and the main hall became empty [12].

After conversion, the building is now used as a technical science museum, which interactively educates about the energy and electricity sector. The machinery hall, which originally had six diesel engines and generators, is now a multifunctional room for exhibitions, scientific devices, and social events. Retained documents about the original state of the machinery hall allowed the exact appearance to be replicated through VR (Figure 2).

Figure 2.

Typological power plant and archival documents of the building, changes from the National Archive in Trnava from 1906 to 1938 (photo: V. Hain, M. Ganobjak, 2010).

The materials, proportions, and details have been derived from preserved and functional historic diesel engines from the Technical Museum in Vienna through 3D scanning. Photogrammetric processes took 3 days. A 3D remodel of the historic 1906 engine was then created. Based on the interdisciplinary cooperation of STU experts and the analysis of historical documents, the historic appearance and hypothetical scene of the power plant machinery hall was hypothesized, presented via VR, and later fully animated. The movie was accompanied by sound taken from similar diesel engines recorded at the Technical Museum in Vienna (permission granted 2014). The sound was recorded using a camera Canon Eos 20D and Nikon D7000 with microphone (after permission was granted in 2014) and then optimized and purified via Adobe Premiere Pro and AGIsoft. This model serves as a 1:1 reference from which it was possible to analogically capture the proportions of the details (Figure 3) and drew them in new precise 3D model. Based on the archival research and the measurements in situ, we sought to find out whether the initial building was built according to plan in 1906. The next research identified all periods of the building’s construction additions and removals and various stages of the finished look (1920–1945). For this case study, it was decided to visualize the first and oldest period from 1906 [12].

Figure 3.

Historical diesel engine from the Technical Museum in Vienna and photogrammetry via software RealityCapture and AGISoft (photo and 3D model: O. Virág, 2016).

3.2 VR 3D model

The digital 3D model of the building was created in accordance with the current measurements and compared with historical plans and construction phases as those were identified. Some standard components of the models (Industry Props Pack, Handyman Tool Pack) are from UE marketplace and Turbosquid (screws, watering can), and graphic works have been carried with texturing, UV mapping (UV layout), animation, and programming (Textured: Quixel NDO, DDO, Substance Painter and Designer).

The final application runs via the Unreal Engine (Figure 4). Initially the scene was tested with Oculus Rift, which had delays in the synchronization of head movements and caused dizziness of VR users. Finally the new more developed version is compatible with HTC Vive as well [12].

Figure 4.

Final VR 3D model of the virtual presentation were presented by VR headset Oculus Rift in the Power Plant Piešťany, where it was possible to compare the current and historical status on-site—an overlay of physical and virtual reality (photo: O. Virág, Ľ. Dait, M. Ganobjak, 2016).

At this point, a user can see an atmosphere of characteristic historical design of space in the original, photo-realistic quality, along with animations and sounds in real time. The 3D model and VR objects were prepared in Unreal Engine 4, which provides photo-realistic images with high-quality textures and lighting. Outcomes are suitable for all these chosen devices: Oculus Rift, HTC Vive, Cyberith, etc. [13].

The VR scene for the old power plant created in 1906 (Figure 4) is designed for the visual communication of technical information, but it also ties in with the diversity of the educational and multisensory exhibition, which is more universal (e.g., for people with disabilities). The target audience represents all the visitors to the hands-on science center EP (Elektrárňa Piešťany—Power Plant Piešťany), who can be entertained but also educated by an exhibition created in this way. The project target group consists of professionals and the general public. Primary school pupils can gain additional educational support from the exhibition. Animators, tutors, lecturers, heritage methodologists, curators, artists, and culture administrators can present new findings from the interactive history in practice, in addition to mediating facts from the world of science and technology history.

The created VR 3D model of the machinery hall seeks to eliminate the extreme situations of negative emotions of the space; it is “phobia-free.” VR respects the senses and aims to eliminate negative emotions, thereby becoming universally appropriate. VR evokes feelings from this environment supplemented by authentic sounds of diesel engines that invoke an industrial atmosphere.

3.3 User tracking and data processing

The virtual machinery hall was tested at the European Researchers’ Night in Bratislava, where it was explored by tracked visitors. The mentioned motivation, inducing natural behavior, was taking photos of the old power plant machinery. Supportive reward system with the impact on the real was publishing of their photos and motions on the second screen. Additionally, the user photographing was marking the most attractive exhibition places and motives. After the visit, users were asked to fulfill short questionnaires about the exhibition’s quality and feelings in VR.

Users’ positions and gazes were tracked in VR, every 0.3 s, and were collected with photographed views to the large point cloud as a raw data to work with on the further research. Users’ positions were then extracted from the point cloud and noted via planar heatmap image by the contrast trace. The more time users spent on the specific spot, the more contrast the spot became.

This extraction enabled to visualize the attractiveness of certain places and process it by machine learning methods to create prototype of analytical tool for evaluation of new designed exhibitions (Figure 5). The created prototype of the analytical tool is a model based on the artificial neural network (ANN) trained by supervised learning. The supervised learning is teaching the network by the pair of related input and output samples [14].

Figure 5.

Users’ tracking data: left, point cloud of all users’ view locations and positions; yellow points are photographed motives; right, map of all users’ movements in plan view (R. Hajtmanek, 2019).

Based on that, the planar heatmap with user’s movements were resized to 40 × 66 pixels and sampled in 0.6 m—human module. After sampling the heatmap, every sample had 4 pixels. In the same grid of 40 × 66 positions, the 3D model of exhibition was analyzed by the isovist tool. The isovist is used for quantification of spatial openness by measuring the distances of the surrounding objects from the certain positions.

In our case, 24 distances of the objects from every position in the grid were measured. Every 24 distances were counted together, quantifying the openness of the space in every position in the grid. This measured openness of the space in the created planar heatmap, which was equally sampled into 4-pixel samples. Together with measures, the measured objects were categorized via importance in the exhibition. Objects were categorized into three groups, with different importance: (1) walls and windows, (2) secondary equipment of the hall, and (3) the main machinery of the hall. Every measure had then information about importance of the measured object, visible from certain point.

To achieve the heatmap of the exhibition importance, measures of the distances were multiplied by its importance. This way, the new heatmap of the exhibition importance was created and was also sampled to 4-pixel samples. Before training the neural network, every position in the grid had three samples: one sample of user’s behavior and two samples of spatial parameters of the exhibition—its openness and importance. In the machine learning experiments, it is usual to divide samples to training and testing set. The training set is 80% of samples, and it is for supervised learning of the ANN. Remaining 20% of the samples is used for the testing the ANN. The splitting of the samples on training and testing set is visible on Figure 6.

Figure 6.

Users’ tracking data: left, downsampled users’ movements map to 40 × 66 pixels resolution; middle, heatmap of spatial openness in the same resolution; right, heatmap of the exhibition importance. ANN was trained on the area, outside of the dashed frame. A testing set of samples was the part inside this frame (R. Hajtmanek, 2019).

Measurements of the space and sampling were made in Grasshopper, procedural modeling plug-in for Rhinoceros 6. ANN was created within this plug-in with Owl, ANN plug-in by Zwierzycki [15]. Supervised learning is based on related inputs and outputs. In this case, the inputs were the samples of importance (4 pixels) and openness (4 pixels), together as eight inputs. The outputs were the samples created from the heatmap of the user behavior, the four outputs. ANN had a structure of 8 neurons in the first input layer, 32 neurons in the three hidden layers, and, last, 4 neurons in the output layer.

4. Testing

This detailed 3D model was so interesting that Západoslovenská energetika, a.s. in cooperation with the Center for Environmental and Ethical Education “Živica” and iPARTNER followed up on this project and created an interactive application on electricity and energy for primary schools (Figure 7). The result of their cooperation with the team from FA STU is an interactive application based on VR game, through which pupils solve tasks related to the subjects of physics, chemistry, but especially electric energy. They informally educate themselves and can virtually visit the Piešťany Power Station in 1906 via the VR application.

Figure 7.

Plan of interactive game with 10 tasks for students (authors: FA STU, Živica, ZSE, a.s., iPARTNER s.r.o., CRATE, 2017).

This application has already been successfully tested at the Pavol Horov Primary School in Devínská Nová Ves—Bratislava in the last school year. It was tested by pupils from 12 to 15 years through the VR set and mobile phones. According to responses, there was a great interest of pupils in this form of education. Educators who have not had experience with the VR so far have shown interest in involving similar innovations in the education process in the future.

Experts on industrial heritage agree that the importance of presenting virtualized models of extinct historical objects is in several aspects:

• Reminder of local history and presentation of the site to the public.

• Initiating a public/professional debate.

• Reinterpreting the meaning for the present and the future.

• Effective (faster and clearer) understanding of the extinct historical object (VR, AR, 3D printing, holographic model).

• Effective nonformal learning.

Short-term use for the presentation of cultural monuments seems to be an appropriate and efficient use of the VR, which can personally approach a defunct building or site. The disadvantage is the high-input economic demands on the hardware and software equipment of the presentation, the technology (operating and explaining), and the low availability of the virtual reality headset, which each visitor uses only at the time.

The presentation in animated virtual reality with the possibility of synchronized movement in space is interactive and creates a subjective experience. It uses an audiovisual design, and in the original old power plant hall, it is sensually complemented by the historically present smell of black oil (unrefined diesel). This affects the imagination of the observer and allows his better immersion, the so-called “deep-rooted,” and the potential for long-term information storage. At the same time, the presentation of the premises through the VR is a more interesting form for a wider audience of different ages and for people with some forms of disability.

The VR is able to appeal to an age-wide and professional audience, thus ensuring the transmission of the legacy of the non-preserved cultural values of the buildings of the past. Virtual reality has proven to be a suitable tool for commemorating the extinct heritage and reinterpreting its significance for the present (Figure 8).

Figure 8.

VR application testing at the Researchers’ Night 2019 in Bratislava and testing by students of the University of the Third Age of the Faculty of Architecture in Bratislava (photo: V. Hain).

5. Results

5.1 Education and experience

The created VR 3D model for the old power plant is adapted for the visual communication of technical information and aims to diversify educational and multisensory exhibitions. The target audience represents all the visitors to the hands-on science center EP (Elektrárňa Piešťany—Power Plant Piešťany), who can be entertained but also educated by an exhibition created in this way.

Such a presentation is suitable for people with various disabilities—the possibility of virtual movement without physical movement for people in wheelchairs; for the deaf, a visual scene; for the visually impaired, intensive contrast of colors and brightness; and for the blind, a sound experience.

Animators, tutors, and presenters can showcase new findings from history interactively, as well as science and technology facts and figures. The project has a research and innovation character: arising from interactive applications with educational content [16]. The project output is aimed to be an application in the popular scientific center at the power plant in Piešťany as a new permanent part of the exhibition.

The project has a reproducible character: it can be used to support educational activities in the electricity and energy sectors, other educational institutions, and schools. This will create conditions for project results to be shared across Slovakia, inspiring repeatability for other science and tech areas.

Raising awareness about industrial heritage can bring a better understanding of its meaning in terms of the diversity of cultural heritage [17]. VR provides a quick educational effect compared to laborious videos or models. Visitors of VR spent around 2 min on average there. The opportunity to synchronize movement, sound, sensations, and walking and looking in VR delivered a unique personalized experience and memories.

The VR 3D model in the exhibition enables the creation of new exhibition attractions with virtually unlimited dimensions, the overlap of the virtual and real world, while also saving physical space in the exhibition. This is enriching exposure. The overlap of virtual and real worlds and the related synchronization of movements of the body and head enhance the immersive (recognition) personal experience [18].

The multisensory VR experience at the power plant Piešťany involves multiple senses (sight, hearing, smell, touch), applying the principles of universal design (multisensory perception, easy operation, etc.), allowing versatility to exhibit and usability by various visitors. VR has eliminated extreme negative emotions related with space—fear of heights, too much closeness or openness—and will not shock with fast-moving animated elements, i.e., it is “space-related phobia-free.” During the testing any serious health impacts of using VR or subsequent phobias have not been detected.

5.2 Questionnaires

During the European Researchers’ Night, users fulfilled together 59 questionnaires evidencing presentation popularity, after the visit of VR. Most of the respondents would visit the old power plant for the education by VR. Only smaller part of the visitors had not experienced VR yet. Seventy-five percent felt comfortable in VR, but there were also several visitors, who feel little headaches. However, most had no problem with orientation in the virtual space, and they would stay longer than 10 minutes. Based on that data, it is possible to consider this model of education as not harmful, safe, and meaningful. Answers expressed in percentages are shown in Figure 9.

Figure 9.

Answers from the questionnaire, fulfilled after the visits (V. Hain, R. Hajtmanek, 2019).

5.3 Tracking

During the visit of the virtual machinery hall, the positions of visitors and their views were tracked. Together, the data from 165 visitors were gathered. The tracking showed that visitors saw just one third of the space (33.91%) on average. There was also a presumption that visitors would mostly take photos of machines in the middle of the hall (as the main part of the exhibition), but only 28.82% of photos captured the machinery.

5.4 ANN predictions

In the subsequent research, tracked data about users’ movement trained the AAN to predict visitors in the space. Such a prediction could be useful for evaluating the suitability of similar exhibition designs. In the training phase, AAN was trained on 80% of the samples. After that it was tested on the remaining 20%. The AAN created the new heat maps of predicted users’ movement, from inputted spatial openness and importance. These predicted heatmaps were then compared to original users’ movements’ heatmap to analyze the ANN accuracy. The compared heat maps were blurred and colored to distinctly display the differences and equalities (Figures 10 and 11).

Figure 10.

Comparison of the original and predicted users’ movements’ heat maps. Left, original downsampled heatmap; right, predicted heatmap by ANN. Tested area is inside the dashed border (R. Hajtmanek, 2019).

Figure 11.

Contrasted and colored heatmaps (R. Hajtmanek, 2019).

Comparing the maps graphically, the ANN achieved sufficient outcomes with the trained set, because those areas are rather similar. Outcomes of testing set are also similar to each other but with lesser accuracy. Nevertheless, the ANN learned some patterns, as it recognized the movements around machines and the most attractive places behind them. With that prototype, it is now possible to evaluate similar exhibition spaces and directly design them without any other users’ feedback.

Tracking the users’ movements and pairing it with measured parameters of the virtual space proved as valid source of information for developing the machine learning-based evaluation tool. Further research in this direction should include even more spatial properties as illumination and materiality to offer more data to the ANN. With bigger data, precision of ANN is increased to recognize the patterns. Gathered user data from the exhibition enable to investigate this field from different perspectives as pairing the movements with their motivations (photographed views and users’ gaze properties), to teach ANN to predict the user viewing and orientation.

Further research could be also transferred to real space. Although the VR offered the ideal experimental control and sterile, laboratory environment, with machine vision technology, it is possible to track people in the real space, enabling to pair their behavior with the spatial properties. These spatial data could be achieved from its virtual representation or to be physically measured. That way, the spaces would not have only exact, measurable parameters but also statistical value of human response to them.

6. Discussion

The research raises questions about VR’s usefulness, relevance, controversy, and entertaining applications. Numerous psychologists also suggest that inappropriately applied VR may constitute a risk: being cut off from the real world and creating a brain fallacy by optical illusion is unnatural and in the long term risky. In this case study, VR as a practical tool enables the public to learn about bygone heritage. Even with the numerous controversial VR uses, this example of VR could be considered meaningful and beneficial in practice [19].

The virtual visit of the industrial space teleports the viewer into a virtual scene where it is also still possible to look around in a traditional manner. Virtual reality allows the handicapped to perform virtual movements without physical effort to places/through places where it would otherwise be impossible to go.

In this case, Oculus was more useful than HTC Vive (depends on the mobility of physically impaired persons). The same virtual scene is perceptible from the perspective of a pedestrian. The perception of users and feeling of size could be changed (the visitor is like a giant, and the scene is only a scaled model, or vice versa).

The opportunity to experience a future, fictional world, to take a walk in the past (Figure 12), or virtually teleport to other points of interest is opened up through VR presentations. Visual perception is supported with realistic materials and textures. Experience in a VR scene installed in the original Machinery Hall is supported by the real in situ scent of heavy oil which is still possible to be smelled in the existing premises.

Figure 12.

The final results of reconstructed building with realistic virtual presentation—output of Unreal Engine 4 (Project of reconstruction of old power plant Pieštany: M. Ganobjak, V. Hain, M. Paško, Z. Zacharová, 2014; 3D Model—BAT engineering 2015, VR processing—Virág, 2016).

Virtual reality with synchronized movement enables from anywhere, even from outside Pieštany, a walk in the historic yet nonexisting interior of the Machinery Hall of 1906. Synchronized movement in virtual and physical reality (Figure 12) is compelling and confirms the meaningful use of virtual reality as a vehicle for presenting the defunct cultural (industrial) heritage against the backdrop of a direct comparison of the contemporary and the original state [20].

The absence of a virtual avatar body in the VR as reported by visitors was a strange experience with feelings of disorientation and confusion, although it is disputable if the presence of an avatar body in VR would have avoided those feelings. Augmented reality, accompanied with the use of physical reality as an anchor for position and navigation, appears to be a further tool for effective education, with the brain effectively distinguishing the essence of a variety of information at a real place. Virtual reality has also shown in this case study to be useful for presentations at several events outside the industrial heritage site.

7. Conclusion

The competent management of cultural heritage requires thorough knowledge and evaluation of the subject causality—with a strong theoretical background and a target-oriented assessment perspective of the presentation and education level.

The case study through VR has reinterpreted the history of the cultural industrial heritage which was not possible to recover in physical reality and have brought it to a contemporary audience. Through this practical interactive tool, the general public can learn about lost heritage. Interactive virtual parts can be embedded in conventional channels and animations controlled by focusing on specific objects. A brief VR experience in machinery halls in an exhibition with a safe level of emotionality, and high immersion in a historical environment has a clear educational benefit about lost industrial heritage and appears an appropriate and meaningful use of VR.

User tracking and the whole principle of interdisciplinary cooperation is not only a synergistic element in a complex organized design process but also a key educational element in the protection of the local industrial heritage for involved participants.

However, each case of heritage management requires a specific and detailed study of the subject. Therefore, neither the criteria specified nor the flowchart presents absolute and conclusive results about the case studies. Therefore, the study aims to serve as an initial model for further studies on the application of virtual reality in the preservation and educational management.

Acknowledgments

This project has been supported with public funds provided by the Slovak Arts Council FPU 16-362-03415 and by the subsidy project Supportive Program for Young Researchers EPUVRSU 1304 and KEGA 038STU-4/2017 “From the laws of nature to technology by experience—a project of informal interactive learning of pupils and students encouraging interest in technical fields.”