Augmented Reality Application Areas for the Architecture, Engineering, and Construction Industry

Abstract

Augmented reality (AR) is among the technologies that have the potential to advance the Architecture, Engineering, and Construction (AEC) industry. Yet, studies show that there remain challenges in applying AR in AEC. According to the literature, the use of AR is focused on the construction phase to address performance, supervisory, and safety-related concerns. However, other phases of AEC projects could also benefit from this technology. Accordingly, this chapter provides an application-centric study to assess the state-of-the-art applications areas of AR in the AEC industry. Various applications have been identified as visualization and simulation; in-situ experience; real-time information retrieval; maintenance, inspection, and repair; project documentation; heavy equipment operation; educational training; health and safety; site navigation; and automated measurements. To further explore these application areas, a case study was conducted using the AR solution of Trimble XR10 with HoloLens 2 in a precast construction context. The results show that existing AR technologies and systems for simulation/visualization and construction quality control are still immature. The study highlighted the current use cases, the potential for technology improvements, and the obstacles that hinder the widespread AR implementation in the AEC industry. Considering these factors, further directions and future research paths for innovators are proposed.

1. Introduction

The concept of augmented reality (AR) can be traced back to the 1960s when researchers began exploring ways to create immersive experiences through computer-generated environments [1]. In 1968, Ivan Sutherland, the developer of the first head-mounted display (HMD), proposed the concept of “the Ultimate Display,” which included the idea of overlaying computer-generated graphics onto the real world. This can be considered an early precursor to augmented reality. The term “augmented reality” itself was coined in the early 1990s by Tom Caudell, a researcher at Boeing, to describe a digital display system he was working on to assist in aircraft assembly. While AR has its roots in the 1960s, it wasn’t until the 2000s that AR started to gain more attention and practical application, with advances in computer vision, wearable technology, and mobile devices [2, 3, 4].

Augmented reality (AR) is an emerging and interactive technology that can add contextual content to real life, compare the digital world with the real world, and fill the gaps between them. AR systems are generally composed of three elements, which are computing, data, and presentation [5]. By using this technology, the user can see the real world with superimposed digital and virtual data through the application of software such as SICURA and SMART, and hardware such as Microsoft HoloLens [6]. Through integrating cameras, HMDs, GPS sensors, and Internet access of smart-phones and tablets, in mobile AR applications real-world environments are overlaid with context-based, dynamic, and interactive digital content [7]. This emerging technology is currently used in applications such as education, gaming and entertainment, manufacturing, medical fields, cultural heritage, scientific visualization, and military applications [2, 3]. Due to high interest in AR, currently, researchers and developers from various domains are working collaboratively to expand knowledge and develop new augmented reality applications that can provide significant benefits of AR technologies to various fields of research and development [6].

In this Chapter, first, the AR research context (the industry, audience, and complementary technologies) is described in Section 2. Next, the prototype applications and application opportunities are identified in Section 3. In Section 4, the authors candidly identify the current challenges facing AEC industry applications. Then, the drivers and opportunities that will enable AR applications in the AEC industry are optimistically described in Section 5. Following that, a case study that demonstrates opportunities and challenges is provided in Section 6.

2. The AEC industry

The AEC industry is a significant sector that plays a vital role in economic development of countries [7]. This industry is a groundbreaking domain incorporating big data, artificial intelligence, and a wide range of knowledge fields and businesses, involved in the design, site preparation, construction of supply lines, construction of infrastructure, off-site construction, construction installation and on-site assemblies, maintenance, and repairs [1, 6]. The AEC industry is experiencing a radical change as project participants are compelling project visibility to increase construction efficiency and reduce project risks. This has enhanced the use of new technologies throughout projects’ lifecycle [8]. As technology becomes more mature and applicable, AR in turn will become an inestimable tool that can change the future of the AEC industry [7, 8, 9]. This is in accordance with the scholars which shows AR is recently gaining a momentum in the AEC industry due to the need for innovation in conducting construction-related activities.

This Chapter about AR is significant to AEC professionals, scholars, researchers, practitioners, the government, and other public agencies to expand their knowledge of how augmented reality technologies can be applied and adopted in the AEC industry. First, an application-centric literature review on the state-of-the-art AR technologies, application areas, barriers, and drivers in the AEC industry is described. Next, the results of a very recent case study are provided, which assess the state-of-the-art regarding the application of AR in AEC projects.

2.1 Industry sector and target audience

Various project types can gain advantage from AR technologies including:

• Infrastructure/public/municipal, e.g., evaluating dynamic city models and developing an model for transportation emission [10],

• Residential, e.g., virtual and augmented reality for designing and customizing mass housing [5],

• Building/commercial, e.g., visualizing high-rise building construction strategies [11], maintenance of exterior closures and interior finishes of walls and in buildings construction [12],

• Heavy/highway, e.g., developing virtual reality systems for optimized simulation of road design data [5], segmentation and recognition of highway assets using image-based 3D point clouds and semantic Texton forests [13], and

• Industrial, e.g., application areas for augmented reality in industrial construction [14].

Various target audiences can benefit from AR technologies, due to the complexity of AEC projects and the collaborative nature of this industry:

• Building systems engineers, e.g., electrical, mechanical, and structural engineers [5],

• Workers, e.g., machine operators, site supervisors, and technicians [1],

• Project end users, e.g., building occupants, residents, and office employees [2],

• Design teams, e.g., architects, engineers, and interior and exterior designers [3],

• Inspectors, e.g., project safety officers [12],

• Schedule and budget professionals, referred to as project managers [11],

• Engineering students [5],

• Other stakeholders, e.g., clients and building owners [5].

2.2 Integration with other information and communication tools and technologies

Overlaying computer graphics demonstrating object-related information to the user’s field of view requires data related to spatial coordinates and practical constraints in that application field [6]. This indicates that AR cannot work independently unless all information is provided [6]. Studies show that to adopt AR effectively in AEC projects, it should be assisted by technologies like Building Information Modeling (BIM), Computer-Aided Design (CAD), Geographic Information System (GIS), and Global Positioning Systems (GPS) [1].

BIM: Mutual application of AR and BIM can significantly impact and benefit building projects from preconstruction to post-construction stages using other technological advancements including big data and wireless sensor technologies, gathering and applying information throughout the project lifecycle [6]. Yet, the AEC industry needs adaptation to gain full advantages of combining AR and BIM tools and technologies [6, 11, 12, 13, 14].

CAD: Using CAD models, AR provides 3D images that augment the real world. Displaying images are projected directly through display devices, which requires a database that involves previously developed plans to support the AR application [6, 15, 16].

GIS: This can be used as a supporting tool to collect real-time data from the surrounding environment to be applied along with AR tools. Field-based GIS gives geo-referenced, topographic, and cartographic information, which is vital in employing virtual images in the real-world environment. A geo-referenced database is critical as it keeps the available data for analysis related to every real object of the physical space [6, 7, 8].

GPS: This can be used to acquire a user’s location whenever they change their positions in an outdoor environment. With technological advancements, smaller, light-weighted, and stronger devices equipped with GPS for self-localization, digital camera lenses, and measures to transmit information at ample bandwidths are available, which can display real-world environments augmented with GPS/GIS data and 3D CAD models [6]. These tools together with AR are essential for achieving practical functions in enhancing performance [17].

3. Application areas

The application of AR in the AEC industry can impact project performance in different fields such as providing collaborative opportunities; enhancing visualizations and simulations; monitoring progress and comparing as-built with as-planned status of projects; enhancing virtual construction site visits; pre-empting work schedule disputes; planning construction activities; and developing 3D building models and training site workers for similar projects [6]. Further explanations about these areas are provided below.

3.1 Visualization and simulation

AR enhances users’ visualization ability via adopting context-related objects, such as gazing through wall panels to see columns, or virtually walking under the ground to inspect the installation of subsurface utilities [6, 18]. AR can be also adopted to visualize the design and assist the design team during the design process [19]. In terms of simulation, AR provides authentic virtual models that can be used to simulate real-world environments. For instance, extracting site measures from 3D augmented models for evaluation of site progress, plan discrepancies, misplacements, and earthquake-induced building damage [5, 6, 7].

3.2 In-situ experience

Augmented reality (AR) can provide users with in-situ experiences such as providing virtual access, verifying the models, and giving warnings. In-situ experience can be developed by walking around a particular landscape. In developing such a virtual environment there is the possibility of adding abstract information (i.e., environmental noise contours around the building) into the augmented view [20]. The user can virtually examine the constructed structures to see whether they suitably fit in with the environment. This application can enhance the use of collaborative project delivery methods, such as integrated project delivery (IPD), in which early involvement of project participants is required [17]. Stakeholders’ early engagement in the construction process can reduce costs due to changes [6, 20]. For instance, in-situ inspections of the site to verify the as-planned status of the project is alignment with the as-is situation [6, 21], and in-situ real-time live warnings given to workers to avoid unseen dangers in the area [6, 22].

3.3 Real-time information retrieval

Augmented reality (AR) can provide project participants with real-time access to project information at various stages of the project. For instance, site inspectors can use AR technologies on-site to conduct inspection activities. Mobile AR technologies can provide professionals with adequate information on projects needed for task management activities including visualizing task locations in a virtual environment [23, 24, 25]. A mobile BIM AR system with cloud storage capabilities can improve task efficiencies by enhancing the information retrieval process [14, 15, 16]. In the transportation domain, AR users can access live information such as car maintenance data, traffic incidents, and route changes, as well as visual/auditory information about the adopted route [6, 20].

3.4 Maintenance, site inspection, and repairs

Augmented reality (AR) can help site maintenance workers to avoid obscured objects such as buried electrical wiring and structural elements that affect the outdoor environments [6]. This accelerates maintenance and renovation operations and minimizes the amount of accidental damage that mostly happens during maintenance [21, 22]. The mounting technical complexity and a high degree of diversity of components installed turn out to be great hindrances for service technicians maintaining industrial facilities [26]. Utility inspection and maintenance activities can be assisted if AR can be used to visualize underground utilities. AR enables users to gaze under the ground and inspect the subsurface utilities. In an AR-assisted inspection method, user’s normal experience is augmented with context-related or geo-referenced virtual objects [27].

3.5 Project documentation

Augmented reality (AR) can be used significantly to document the project’s progress. With recent developments in the field of AR, users can directly augment 2D project drawings with generated 3D models. AR combined with BIM and 3D modeling software, can provide elaborated, interactive models of buildings to be shared with project stakeholders at early stages of the project [6]. Clients are permitted to visualize the realistic outcomes of the project and necessary changes can be made before the construction starts. All these models and other project documents can be stored in augmented reality database, which can later be accessed by all project parties [28].

3.6 Heavy equipment operation

Augmented reality (AR) provides the possibility to give training to service technicians and heavy equipment operators in the construction process [29]. Using this training platform, the experienced workers can execute complex operations better and avoid project time delays [30, 31, 32]. Real-time tracking capability of AR tools can enhance workers’ awareness in hazardous situations at the job sites [6, 17, 18, 33]. Applying augmented models of heavy equipment, such as cranes and boom lifts, allows workers to use their headsets to rehearse operating heavy machinery in a safe virtual environment, and enhance their learning experience and development [6, 20].

3.7 Educational training

Augmented reality (AR) provides new opportunities to effectively train and educate students or novices with a higher level of cognition and fewer hazards [13]. Safety training in the construction industry is challenging and safety-related programs cost a lot of money and time [25]. Training scenarios can be provided to students with the aid of an AR tools and technologies. The virtual intuitive site-safety learning enhances students’ awareness of safety while lowering downtime and training costs [6]. Students can easily see AR images on their mobile devices due to the rapid development of AR applications [34].

3.8 Health and safety

The AEC industry is associated with highly hazardous situations that can cause danger to site workers. Thus, safety and health are one of the highest priorities for laborers. AR emerging technologies can enhance workers’ health and safety when applied correctly [35]. Vehicles equipped with AR devices can give road guidance and real-time data about dangerous site conditions to operators. Workers can use AR devices and cameras connected to their safety helmets, to see and hear through fire, smoke, heavy rain, flood, bad weather, and other dangerous conditions on-site [30, 31, 32, 34]. ICT tools integrated with AR can provide site workers with well-interpreted information to monitor the difference between standard safety requirements and unsafe site conditions [36].

3.9 Site navigation

Augmented reality (AR) coupled with GIS data with augmented visual landmarks in the real-world environment can lead users to a particular location or direction [6]. In the realm of construction site navigation activities, the data integrated into AR applications serves three primary purposes: (1) revealing obscured elements (such as obstructed objects or buried components), (2) visualizing forthcoming constructions (anticipating the future), and (3) perceiving imperceptible aspects (including site boundaries, organizational details, alignment information, or infrequent environmental incidents like rare floods) [6]. AR can assist site workers in navigating throughout the construction site or within a particular facility [20, 21, 22, 23, 24].

3.10 Automated measurements

AR can be used to extract measurements of the physical objects (width, height, and depth) on-site. This data can be integrated into 3D models to generate more accurate structural visualization of the project based on the project’s ultimate geometry [6]. AR can enable field workers to conduct automated measurements on-site during the design phase. AR allows users to capture building structures accurately, which can be adopted for real-time building simulations [3, 4, 5, 6, 7]. When an AR device is worn, workers can tap and automatically make measurements of the built elements and compare them to the planned measurements. This allows workers to find discrepancies between as-planned and as-built models and swiftly adjust them to avoid higher costs and delays in the process [37]. Another use of AR is the ability to walk through real civil infrastructure while simultaneously viewing a virtual 3D model of the same infrastructure as recorded several years prior. This is particularly valuable as a means of detecting trends in the deterioration of concrete infrastructure [38].

4. Current challenges

Augmented reality (AR) has great potential to revolutionize the AEC industry by enhancing visualization, improving communication, and streamlining project management. However, there are several challenges that need to be addressed for successful implementation of AR. Here are some of the key challenges:

4.1 High initial cost and low return on investment

Augmented reality (AR) implementation in construction requires substantial investment in hardware, software, training, and maintenance. Construction companies need to carefully evaluate the cost–benefit ratio and calculate the potential return on investment. Demonstrating the tangible benefits and long-term cost savings of AR technology can be challenging but is essential for wider adoption. Since AEC projects are typically large and complex, a prevailing notion indicates that applying AR tools and technologies would require a significant number of expensive devices, such as HMDs and other related equipment [8, 16, 17, 18, 33]. Subsequently, a major initial investment would be necessary to integrate AR into construction and infrastructure projects, which can expand the overall project cost [8]. Thus, it is essential to conduct research and development activities toward developing low-cost AR tools and technologies suitable for the AEC industries [8].

4.2 Nascent technologies

Augmented reality (AR) technologies are in nascency in the AEC industry. It requires powerful hardware and infrastructure to deliver real-time, high-quality AR experiences. Construction sites are often remote and lack stable internet connectivity, making it challenging to provide the necessary infrastructure for AR implementation. Additionally, AEC projects demand a higher degree of precision, consistency, and efficacy. However, current AR devices struggle to control the extremely sophisticated 3D information models regularly used in AEC projects [8]. Originally, AR devices were designed for the entertainment industry, thus, they might lack the on-site capabilities necessary for the AEC industry [8]. In addition, due to the complexity of this technology and a lack of technical skills and awareness, AR adoption in the AEC industry is slow [8]. Regarding the accuracy and precision, AEC projects require precise measurements and alignments. More development in AR systems is needed to accurately overlay virtual elements onto the physical environment. Achieving high accuracy and precision in AR tracking and alignment remains a challenge, especially in dynamic construction environments [20, 21, 22, 23, 24, 25].

4.3 Insufficient adoption demand

Studies show that the demand for AR in the AEC industry is currently low. Nonetheless, with the ongoing evolution of technology and the formulation of customized answers to address the distinct requirements of the AEC sector, there’s an anticipation of heightened acceptance [8]. To foster its adoption, initiatives aimed at raising awareness should be launched, highlighting the benefits of AR technology within AEC. These advantages encompass enhancing project comprehension, optimizing cost-effectiveness, and enabling efficient training [8]. Furthermore, educational establishments can contribute significantly by enticing fresh expertise in the field through the introduction of programs that offer specialized training in AR [33].

4.4 Lack of experts and insufficient training

Introducing AR technology requires training and upskilling the construction workforce. There may be resistance to adopting new technologies due to a lack of awareness or skepticism. Overcoming the learning curve and effectively training workers to use AR tools can be a challenge. A limited number of individuals are pursuing careers in AR due to its immature stage of development, complexities in adoption, and absence of standardized implementations. This lack of established norms complicates the evaluation of the knowledge and skills possessed by those working with AR technologies. Moreover, the scarcity of experts in this field poses difficulties in gauging their grasp of the technology and effectively involving them in financially significant AEC projects. To tackle this challenge, universities should introduce advanced education programs centered on AR, thereby bolstering research and development endeavors, and enhancing the technology itself. Concurrently, AR companies could support research and development initiatives within universities, thereby facilitating this progress [39].

4.5 Poor user experiences

Extended utilization of AR devices like HMDs can result in motion sickness, queasiness, perspiration, headaches, and even vomiting among users [8]. This poses a significant impediment to the widespread adoption of AR technology. Given the intricate and time-intensive nature of AEC projects, prolonged engagement with these devices can prove distressing for most users [8]. To address these concerns, endeavors should be directed toward enhancing the design and development of AR devices, aiming to minimize discomfort and alleviate the adverse effects experienced by users [8]. Additionally, incorporating frequent intervals during extended usage and restricting the duration spent in the virtual environment can aid in diminishing the occurrence of motion sickness and other related discomforts [8].

In addition, construction sites are complex and constantly changing environments. AR systems need to adapt to changing conditions and safety regulations. Ensuring that AR technology does not compromise safety and can handle various environmental factors like dust, vibrations, and lighting conditions is a significant challenge [2, 3, 4].

4.6 Integration, adaptability, and data security

Integrating AR seamlessly into existing construction workflows is crucial for adoption. AR systems need to interface with existing project management software, BIM data, and other AEC tools. Ensuring compatibility and smooth integration can be a technical and logistical challenge. The AEC industry involves multiple stakeholders, each using different software and systems. Achieving standardization and interoperability among various AR platforms, software, and hardware devices is essential to enable seamless collaboration and data exchange. AR systems in construction involve capturing, processing, and sharing sensitive project data. Ensuring data security, protecting intellectual property, and addressing privacy concerns are crucial. AEC companies must have robust data security measures in place to protect against unauthorized access or data breaches [22, 23, 24, 25].

Addressing the challenges in this section requires a collaborative effort between AEC companies, technology providers, regulators, and industry associations. As AR technology continues to evolve, these challenges are likely to be mitigated over time, leading to more widespread adoption and integration of AR in the AEC industry.

5. Drivers and opportunities

While various barriers impede the adoption of AR technologies in the AEC industry, several drivers can impact users’ motivations to apply these technologies in practice. These drivers are discussed below.

5.1 Enhancing comprehension of projects

The integration of AR technologies into the AEC sector is predominantly motivated by the advantages it brings in augmenting project comprehension [8, 10]. Through the application of AR, AEC endeavors can be virtually simulated, or tangible surroundings can be enriched with digital data, leading to a more profound grasp and visualization of the project [6, 7, 8]. This establishes a risk-free domain for the project, enabling the identification and resolution of any associated issues within a virtual realm [6]. Consequently, the utilization of AR enables the observation of diverse project stages, ultimately culminating in an enhanced holistic comprehension and reduction of project-related risks [8, 9, 10, 11, 12].

5.2 Reducing project overall costs

Augmented reality (AR) technologies have the potential to reproduce AEC projects’ overall costs using the recently introduced digital methods and approaches, such as digital twin proposed by Oke et al. [6]. By identifying and rectifying problems in the virtual environment at early stages of the project, and optimizing project designs, it becomes possible to economize expenses and diminish the likelihood of human errors [8]. The amalgamation of the physical setting and the digital realm simplifies the supervision of project tasks, enabling virtual enhancements to be incorporated and real-time observations to take place. This, in turn, enhances the project’s efficiency, resulting in superior outcomes achieved at a reduced expense [1, 6, 8].

5.3 Effective training scenarios

Leveraging AR technologies for employee training involves simulating real-life situations, affording employees the opportunity to hone their skills within a secure and regulated digital environment [8]. This methodology has the potential to considerably elevate training excellence through the provision of practical encounters within authentic contexts, ultimately fostering a more efficacious professional atmosphere [5, 6, 7, 8]. Furthermore, this expedited acquisition of new proficiencies results in noteworthy reductions in training duration, consequently curtailing expenses while concurrently elevating the caliber of training [8, 9, 39].

5.4 Reducing damage and maintenance costs

Establishing a virtual environment through AR enables the efficient monitoring and juxtaposition of project advancement [7, 8]. This strategy mitigates the prospect of harm and subsequent repairs, given the prior digital simulation of the project. Additionally, the application of AR can unearth the most effective method for executing the project, culminating in diminished developmental expenditures [8, 15]. Consequently, the incorporation of AR within the AEC sector holds the potential to curtail project risks and trim development costs [40].

5.5 Improving user experiences

Studies show that AR technologies offer a unique and immersive experience to users [8, 9, 10, 11, 12, 13, 14]. The capabilities of these technologies offer a substantial prospect for reshaping interactions within our surroundings. AR bestows upon users a world of fresh opportunities, enabling them to engage with inanimate entities, foster deeper connections with individuals and surroundings, and envision their desires with precision. Within the AEC sector, these technologies hold remarkable potential [8, 9, 10]. For example, the immersive experience, offered by AR, allows workers to simulate thoughtless or mistaken actions alongside their ensuing outcomes, thus improving their training and safety protocols [8, 41].

6. Case studies

As discussed in previous sections, AR has various potential applications in AEC projects. However, its usage in AEC projects has not been widely implemented yet. Our investigation of AR applications identified major drivers of and barriers to the adoption of AR technologies in the AEC industry, which need further investigation. This section provides case study results on the implementation of AR technologies to conduct quality control (QC) inspection tasks in a precast manufacturing plant in Canada and identifies major challenges and recommendations to address those challenges [42]. Similar to other AEC industry environments, this case study environment is particularly challenging due to the component size, precision requirements, worksite congestion, and inherent safety hazards. For the detailed information regarding this case study, you can refer to the Master thesis by ref. [42].

6.1 The application of AR in the Strescon plant

The case study is conducted in a precast concrete manufacturer, Strescon Limited, located in Saint John, New Brunswick, Canada. The plant produces a variety of products including structural and architectural panels, concrete pipes, catch basins, and bridge girders. A group of 11 quality control inspectors at Strescon were selected as the focus group in the case study. Each participant was trained to work with the AR systems and the researcher observed their performance to ensure that they can work with the features without assistance.

The Augmented Reality solution of Trimble XR10 with HoloLens 2 [43] (in short XR10) was leveraged in this case study. The Trimble AR solution was first released in 2019 including the Microsoft HoloLens 2 head-mounted display and a hardhat approved by the Canadian Standard Association (CSA). Trimble Connect for HoloLens 2 (TCH) is the software component developed for XR10 to provide an AR experience, particularly for AEC practitioners. The AR solution of XR10 incorporates features such as TCH Measure, TCH Explore, TCH To-Dos, and TCH Navigate. TCH Measure allows users to measure any two points in a model or any two points with one in the model and one in the real world. Figure 1(a) shows an example where the user is measuring the overall dimensions of a precast concrete panel in the virtual model. TCH Explore consists of Explore Visibility and Explore Info. The Visibility feature allows a user to turn the visibility of model components on and off and view only the desired components. The Info feature provides a list of properties for each individual component in the cast unit model. These properties were already incorporated in the building information model of each cast unit. Figure 1(b) shows a user looking at the properties of a lifter hook in a precast concrete panel using TCH Explore Info. The To-Do feature in the Tools menu allows a user to view work orders or manage Requests for Information (RFIs). Figure 1(c) presents a snapshot of TCH To-Do where a user is creating a To-Do to send to office users. Users can navigate through the model, create section boxes, rotate, and manipulate the model using the TCH Navigate feature. Figure 1(d) shows a snapshot of TCH Navigate where a user is using the section-box tool and navigating the virtual model to gain a better look at the components of the cast unit model.

Figure 1.

Features of Trimble Connect for HoloLens 2 in an AR test on a precast concrete slab and its virtual model.

6.2 The implementation of XR10 at Strescon

The study involved volunteer QC inspectors and focused on the manufacturing process of a large architectural wall panel. The first step is to align the building information model of the element with the cast unit. Trimble XR10 uses a marker-based spatial registration method to align models with their real-world counterparts. The proposed step-by-step instructions for model alignment is shown in Figure 2.

Figure 2.

The workflow of setting up the AR system in the Strescon shop floor.

After aligning the virtual model with the real casting bed, the researcher observed three significant challenges in using XR10 for QC inspection and recommended solutions to facilitate the use of XR10 for QC inspection scenarios. These challenges and recommendations are summarized in sections below.

6.3 Challenges and recommendations

6.3.1 Transferring BIM models to Trimble cloud platform

Two distinct methods were identified for transferring the building information model from Tekla Structures to the Trimble Connect cloud platform: (1) one approach is to export the model of the building, in which the architectural wall panel (i.e., the test subject) is modeled, and (2) the second approach is to select the cast unit model of the architectural wall panel in the building model and export it individually and as a .ifc file.

The benefits and drawbacks that were observed in each approach are described below:

Approach 1: Exporting the entire building model as an .ifc file:

• Benefits: This approach eliminates the time to export each assembly unit of the building individually. Consequently, less workload is imposed on the BIM modeler when preparing .ifc files for use on the shop floor.

• Drawbacks: When a QC inspector attempts to inspect a cast unit assembly in the shop with XR10, they need to find the cast unit model that is to be inspected, from a long list of cast unit models that are modeled in a building. All other parts of the building model should be turned off for visibility, and the target cast unit model should remain on for visibility. This procedure imposes extra waiting time for a QC inspector, at the working table in the shop floor.

Approach 2: Exporting each cast unit model as a separate .ifc file:

• Benefits: The QC inspector is not required to find the desired cast unit from a list of all cast units in the model while wearing the XR10 in the dynamic environment of the shop floor. Therefore, the inspectors would start their inspection process by launching the .ifc file of each cast unit that is scheduled for inspection. Another benefit of this approach is that while some cast unit models such as wall panels are oriented vertically in the building model, the physical casting unit of that wall unit needs to be oriented horizontally on working tables in the shop floor. Therefore, the model needs to be repositioned in the Trimble Connect 3D viewer to match the orientation of the cast unit in the manufacturing line. However, if the model is not exported as a separate .ifc file and is exported along with other cast units in the building, repositioning the cast unit model imposes complication on the workflow of QC inspectors. When repositioning the model of a wall (from vertical view to horizontal view), a precast slab will also get reoriented to vertical view rather than horizontal view. In this case, a vertical orientation of a slab in the building model would not match the orientation of its counterpart cast unit in the manufacturing shop.

• Drawbacks: For large numbers of cast units that are scheduled for daily production, the BIM modeler needs to export each cast unit individually and send them out to the scheduling department for use by QC inspectors, which may impose extra time on the workflow of modeling and scheduling departments.

To avoid adding extra complication to the workflow of QC inspectors and minimize human errors when preparing cast unit model files for QC inspection, the second approach is recommended. Figure 3 shows the repositioning of the architectural wall panel on the Trimble Connect Web platform.

Figure 3.

Model repositioning on Trimble Connect for web.

6.3.2 Creating virtual QR code markers

The QR code markers need a clean surface area around the formwork for accurate scanning. Cover sheets are suggested to prevent marker dirtiness or damage to the QR code markers, see the location of QR code markers in Figure 4. Repositioning the QR code markers in the Trimble Connect platform was challenging, as it lacked options for snapping to points surrounding the model. As shown in Figure 5, a user may need to reposition the QR code markers several times. However, when the QR code marker is placed a distance from the cast unit model, there is no option for the user to snap to a point on the QR code marker and to verify the exact position of the marker relative to the cast unit. It is recommended that users manipulate coordinates of the QR code marker with great care to avoid errors. For BIM modelers it is also recommended to model bounding boxes around the cast unit models. The modeling of a bounding box in the 3D space around the cast unit model, allows a user to snap the QR code marker to any point within the bounding box. This method may help modelers verify the position of the QR code markers in a reliable manner.

Figure 4.

Areas around the panel formwork for attaching QR code markers.

Figure 5.

Repositioning QR code markers on Trimble Connect for web.

Setting up the QR code markers on working tables posed another challenge. The use of formwork edges as reference points for marker placement could lead to misalignments between the 3D model and the as-built unit. To mitigate this issue, as shown in Figure 6, QC inspectors were advised to consider the discrepancies and consistently account for them during inspections.

Figure 6.

Unwanted misalignment of the 3D model and the cast unit.

6.3.3 Overall QC inspections

The use of the TCH Explore tool sometimes made it difficult to clearly see components in the real world because they were occluded by the superimposed virtual model. Figure 7 shows the overall inspection of a cast unit where the user identified a misplaced component in the real precast unit based on the virtual model. Adjusting the opacity of the model is recommended to improve visibility during overall inspections. Snapping for measurements between the model and the formwork in the real cast unit was also challenging, with incorrect measurements resulting from inadvertently snapping to a rebar instead of the formwork. Figure 8 shows a real-to-model measurement practice where the endpoint of the measurement is not correctly snapped and results in incorrect measurements. For tight tolerances of measurements, performing a “model-to-model” measurement and then matching it with its real measurement on the cast unit is advised to reduce errors.

Figure 7.

Inspecting misplaced components in architectural wall panel.

Figure 8.

Differences between the As-designed dimension (1′ 6″ 5/8) in the model and “real-to-model” measurement (1′ 5″ 7/8) in TCH.

Lastly, performing measurements in the depth of the formwork required the QC inspector to adopt uncomfortable positions. To overcome this, inspectors are recommended to relaunch the model from the TCH project browser, skip alignment with QR code markers, and view the model in 3D space for easier access. The TCH Navigate tool can be used to enlarge the model and to perform “model-to-model” measurements conveniently.

6.4 Observations and findings from the implementation of XR10 at Strescon

In this section, researchers’ observations on user experience and attitudes toward the TCH applications are provided. After implementing several applications of AR at the plant, Strescon QC inspectors (subsequently referred to as Participants) who were involved with the applications were asked a set of open-ended questions. These questions were designed to understand the Participants’ experience with the Trimble AR solution and their perspective toward implementation of AR in precast concrete manufacturing. Participants’ responses were recorded and transcribed. A content analysis method was carried out on the responses to identify similar codes and expressions. The results of Participants’ experience with the TCH application and XR10 are summarized below:

Participants’ experience and attitudes toward TCH Measure: Participants found the TCH Measure tool relatively hard to use compared to other features. The main reasons mentioned were difficulties in measuring irregular shapes, challenges in selecting tiny virtual objects like mesh and rebar, and problems with the TCH hand-gesture control.

Participants’ experience and attitudes toward TCH Explore: Participants found the TCH Explore (visibility) and TCH Explore (Info) features relatively easier to use. They highlighted the intuitive layout and easy navigation of the application, as well as the easy-to-learn nature of these features.

Participants’ experience and attitudes toward TCH interface: Participants had mixed opinions about the layout of information and features on the Trimble Connect application. While some found it intuitive and user-friendly, others mentioned challenging navigation. Overall, participants provided positive feedback about the layout.

Participants’ overall experience with TCH and XR10: Participants described their overall experience with the Trimble Connect application and XR10 as positive, using terms like “great,” “amazing,” and “fun.” However, they also mentioned usability issues and expressed mixed feelings such as frustration, annoyance, and confusion. Some participants emphasized the importance of training to fully utilize the technology’s potential.

Participants’ suggestions for improvements in the TCH and XR10: Participants suggested various improvements for the technology. They desired a more comfortable and lighter helmet, intuitive hand-pointer controls, automated identification of discrepancies, a bigger field of view, better organization of measurements, and more precise measurement tools.

In the research study, participants were asked a series of questions regarding their perspective on the adoption of AR technology in precast concrete manufacturing.

Participants’ perception toward the most useful applications of TCH Measure in QC inspection: Participants mentioned the TCH Measure feature can provide comfort to the inspector, knowing that they have physically measured with a tape measure and verified the accuracy of the measurements. A few participants also mentioned that this tool could be useful for formwork inspection, and initial inspection. Other participants also found that the AR technology with TCH and XR10 can be useful for checking the overall dimensions or the location of hardware components in a cast unit. Several participants also concluded that the TCH Measure tool would not be useful at all.

Participants’ perception toward the most useful applications of TCH Explore in QC inspection: Some participants mentioned that verifying the type and size of the components (e.g., rebar, welded plates, inserts, mesh, etc.), as well as finding out the location of components in the cast unit, are the most useful cases of using the TCH Explore tool. They also mentioned the “initial inspections” and the “final inspections” could also be the most useful cases of using TCH Explore but did not elaborate more on those. Another participant also mentioned the TCH Explore tool could be the most useful as a complementary tool to the conventional method of QC inspection to double-check and verify the inspections.

Participants’ perception toward the most valuable features in the TCH application for QC inspection: Regarding the valuable features of the Trimble Connect application, participants mentioned tools like TCH Explore (Visibility), TCH Explore (Info), and TCH Measure. The TCH Explore (Visibility) tool was particularly appreciated by participants as it allowed them to isolate specific model components, enabling a focused inspection. The TCH Explore (Info) tool was mentioned by four participants, who found it helpful in quickly accessing information about model properties, saving time compared to conventional methods. Some participants also highlighted the usefulness of the TCH Measure tool, which eliminated the need for manual measurements with a tape measure. Additionally, participants found the overall 3D view of the model valuable, providing an immediate assessment of component placement and identification of any discrepancies.

Participants’ perception toward the hindrances in implementing the AR application in precast concrete: When discussing hindrances or difficulties in using the Trimble Connect application with HoloLens in the production line, participants identified several concerns. The most common hindrance mentioned was the potential distraction caused by the busy and noisy shop environment, which could affect the QC inspector’s focus. Safety was another frequently mentioned concern, as wearing the XR10 device might lead to reduced awareness of surroundings, especially in areas with equipment and debris. Participants also expressed the need for continuous learning and familiarity with the device, considering the learning curve as a hindrance. Other hindrances included durability in a dynamic construction environment, battery life, limited mobility, and the need for frequent adjustments to formwork setups.

Participants’ perception toward the future of AR technology in precast concrete industry: Participants shared their expectations for the future of AR technology in precast concrete production, highlighting their overall perspective, required improvements in AR technology and Precast Concrete Manufacturing (PCM) processes, and potential benefits of implementing AR technology. While participants expressed optimism about the future adoption of AR technology in PCM, they also anticipated further development and refinement before full implementation. Improvements in AR technology mentioned by participants included enhanced accuracy in placing models on casting beds without recalibration, refined displays, improved precision, and preset options for model alignment. In terms of PCM processes, participants suggested simplifying QC tasks, expanding the level of detail in BIM software, and establishing efficient model alignment requirements. Anticipated benefits of AR technology implementation included facilitating preproduction meetings, improving efficiency and simplification of the QC process, and reducing errors.

Participants’ perception toward the current use cases of AR technology in precast concrete industry: Participants identified specific areas within the QC inspection process where they believed AR technology could be effectively utilized. These areas included formwork inspections, initial and final inspections, supplementary inspection tools, and preproduction meetings. Participants were generally optimistic about using AR technology in initial and final inspections, considering it a supplementary method alongside 2D drawings. Additionally, they saw potential benefits in using AR technology during preproduction meetings to visualize products and identify issues.

Participants’ suggestions for potential improvements in the AR technology for future of precast concrete: Regarding necessary improvements for using Trimble XR10 with HoloLens 2 in the QC inspection process, participants mentioned several areas that could be enhanced. These included user adoption, ambient light conditions in the shop and yard, maneuverability in the manufacturing line, organization of working tables, active notification systems for drawing revisions, and better time management. Concerns were raised about the impact of bright environments on model visualization, the need for sufficient space to move around while wearing XR10, and the importance of maintaining a clean and organized environment for effective use of the technology. Participants also emphasized the need for active notifications to inform individuals involved in QC inspection of drawing revisions. Moreover, participants mentioned the importance of allocating dedicated time for QC inspection with AR technology due to the presence of various tasks.

The results of the research indicated that AR users in the construction industry have a good understanding of various features in the existing technologies and their applications. In the case study, various applications of the TCH and XR10 have been explored, and users’ experiences, attitudes, and perceptions toward the AR technology were discussed. Users found that existing AR technologies and systems in construction domain are still immature and provided suggestions for improving the current state of technologies in the industry. In summary, the study highlighted the current use cases, the potential for improvements of the AR technologies in construction, and the obstacles that hinder the widespread implementation of these technologies in the AEC industry.

7. Future outlook

In previous sections, numerous factors were identified that impede the wide applications of AR systems in the AEC projects. One aspect to consider involves the early stage of development for this technology and the gradual progress of augmented reality within the construction phase. Another factor to contend with is the limited effectiveness of human interactions with these emerging technologies, which poses an additional challenge when utilizing augmented reality. Considering these factors, it becomes important to focus future research and operational strategies related to the implementation of AR technologies in the construction phase are categorized into two categories, including technology-oriented and human-oriented topics. These two categories are further discussed in this section for future research directions.

7.1 Technology-oriented development

This group includes suggestions to improve the research conditions related to AR technologies in AEC projects and the development path of this technology, as follows:

• Improving the use of 4D/5D/6D AR tools to enhance project scheduling and costs, improving monitoring of project progress, and reinforcing decision-making activities [44].

• Developing advanced AR tools to display the model with a higher level of development (LOD) leads to the representation of model elements more accurately [45].

• Focusing on the use of I4.0 technologies and cloud-based systems to achieve a real-time ability in transferring, processing, and applying changes between the BIM and AR 3D models [46].

• Testing and developing AR-based digitally-twined systems with a multifunctional capability to evaluate the effects of this technology in the construction phase of projects [47].

• Enhancing AR-based systems to adapt this technology to infrastructure projects such as bridges and tunnels, which have different construction and monitoring conditions.

7.2 User-oriented development

At the current stage of applying AR technologies in the AEC industry, several human-oriented concerns could be confronting to investigate and might require a more accurate evaluation. Accordingly, certain suggestions are put forward as follows:

• Providing sufficient training to AR users in the project, before applying the technology. This training is provided automatically by AR itself to minimize the needs, which requires further studies [46].

• Increasing the use of AR to educate students and professionals to increase interactions and familiarity with this technology between the academic environment with the AEC industry [26].

• Involving and informing the industry professionals about the features and capabilities of AR to increase its acceptance rate. Improving public acceptance can lead to an increase in the use of AR in projects [43, 47].

• Developing project delivery methods (i.e., IPD) and business models (i.e., vertical integration) that facilitate the implementation of AR technologies and other emerging digital tools to improve construction project management practices [5, 42, 48].

In summary, the authors are optimistic about the adoption of AR in the AEC industry and revolutionization of the industry by these technologies in the future. Although current AR applications tend to be under the control of a single project participant and in tightly controlled environments, it is expected that the huge potential of AR will become more evident as multiple participants adopt integrated AR technologies across multiple project phases.

8. Conclusion

This chapter provides a review of the state-of-the-art AR technologies and their applications in AEC projects. In the presented work, the authors have attempted to provide a deeper understanding of the applications of AR for the AEC industry. Ten application areas for AR technologies in the AEC industry have been identified as visualization and simulation; in-situ experience; real-time information retrieval; maintenance, inspection, and repair; project documentation; heavy equipment operation; educational training; health and safety; site navigation; and automated measurements. Following that, various challenges and drivers for application of AR in AEC projects were extracted and a case study about AR application in a manufacturing plant in Canada was provided. Eventually, according to the literature review and case study results, implementing AR in AEC projects faces problems, which shows that this technology is still immature and needs further investigation and efforts. Accordingly, the authors of this chapter have suggested a series of technology-oriented and user-oriented research and executive directions in the “future outlook” section.

Acknowledgments

The authors express their sincere appreciation for the financial support received from the New Brunswick Innovation Foundation (NBIF) Emerging Projects under the grant titled “Towards a Digital-twin-based Prefabricated Construction Environment Using Data-driven Analytics and Immersive Technologies,” with reference number EP_2022_009 and application number EP-0000000112. The contribution of in-kind support from StresCon Limited in Saint John, New Brunswick, Canada is also gratefully acknowledged for providing the testing environment essential for conducting the research case study.

Conflict of interest

The authors declare no conflict of interest.

Data availability statement

Some or all data that support the findings of this study are available from the corresponding author upon reasonable request.