Intelligent VR-AR for Natural Disasters Management

Abstract

Considering the significance of improving natural disasters emergency management and recognizing that catastrophe scenes are almost impossible to reconstruct in real life, forcing persons to experience real hazards violates both law and morality, in this research is presented an engine for Virtual Reality/Augmented Reality (VR/AR) that works enhancing human capacities for prevention, response and recovery of natural phenomena effects. The selected novel techniques have very advantageous qualities to overcome the inconveniences detected in the most recent seismic devastating experience in Mexico City, the Sept 19th, 2017, earthquake M7.2: total collapse of more than 230 buildings, partial fall of 7 000 houses, 370 people were killed, and over 6,000 were injured. VR and AR provide researchers, government authorities and rescue teams with tools for recreating the emergencies entirely through computer-generated signals of sight, sound, and touch, when using VR, and overlays of sensory signals for experiences a rich juxtaposition of virtual and real worlds simultaneously, when AR is applied. The gap between knowledge and action is filled with visual, aural, and kinesthetic immersive experiences that poses a possibility to attend to the population in danger in a deeply efficient way, never experimented before.

1. Introduction

With the increased prevalence of smartphones and popularity of artificial intelligence (AI), substantial research and development have been made in pursuit of more cost-effective and high-performance sensor technologies [1, 2] and graphical processing units, which have led to the production of affordable virtual and augmented reality devices [3]. These developments allowed researchers in many fields (e.g., astronomy, psychology, medicine) to create controlled virtual environments that permit users to interact with digitally generated stimuli [4, 5, 6]. Virtual Reality (VR) and Augmented Reality (AR) are presented here as present and future alternatives for the efficient training of human resources, not only in the industrial field, but in every aspect of real world, of real life. The virtual recreation which objective is to transport the user to fully digitized and interactive environments, has allowed for the simulation of real processes and situations, an adequate way to reduce training times, prevent errors, or even improve the quality of products and actions. In the field of environmental sciences and natural disaster management (NDM), public, scientists, decision-makers, and professionals can benefit from virtual and augmented reality applications to simulate various and complex scenarios, constituting a realistic and safe workspace for repetition, precise measurements, and improved knowledge [7, 8].

In this research it is presented how VR and AR are applied to digitally recreate the real-life setting when an earthquake hit. In the preparedness before disasters happen, the simulated scenarios allow the user to interact with buildings, houses, foundations, streets, and buried pipes, on the one hand, and with soil strata, rock basement, ground motions, fissures, and cracks, on the other. Tricking the human perceptual system into believing that is being part of the virtual city, it is guaranteed complete focus on the effects of the seismic shaking.

By the seamless blending between a real environment and computer-generated virtual objects, the resulting mixture supplements the natural environment (the phenomena, the infrastructure, and soils responses) rather than replacing it with simplified conceptions. The virtual objects incrusted in the reality display information about structures characteristics and soils properties that the users cannot directly detect with their own senses. This information conveyed by the virtual objects helps a user perform better NDM-tasks. The concepts of VR/AR activated are simulation, interaction, artificiality, immersion, telepresence, full-body immersion, and network communication. The input information is compiled from the September 19th, 2021, earthquake files (accelerograms and geotechnical zonation where the monitoring station is placed), records of large settlements, fissures, and cracks as well as total and partial collapses of buildings and damage to buried pipes. The devastating earthquake of 2017 provided the opportunity to evaluate how citizens and the government prepare and respond to emergencies due to natural phenomena, particularly the seismic phenomenon, to generate the necessary improvements at each stage of natural disaster management.

2. Why natural disasters management must be supported by VR/AR

Built environments, which refer to all physical environments constructed for human habitation and activities [9], are constantly exposed to risk from various natural and manmade disasters, such as fires, tropical cyclones (wind and storm surge), earthquakes, tsunamis, floods, and terrorist attacks, which pose a significant threat to human beings. Considering the severity of natural hazards, that become disasters when people’s lives and livelihoods are destroyed, an appropriate NDM is crucial to reduce the harmful effect of the phenomena and to facilitate a prompt reestablishment of the normal life after the emergencies [10]. When the threatened scenarios are large cities or strategic sites for nations, it is essential to have efficient and timely management programs that protect lives and properties of the inhabitants but also that quickly restore the productive and commercial capacities of population centers [11]. NDM covers i) risk prevention, ii) crisis preparedness (specific training), iii) emergency response (rescue), and iv) catastrophe recovery (reinstatement of services and lifelines) [12]. The key point in these actions is the preparation facing of the emergency so the researchers and professionals must work to improve the understanding of the situations, processes behind and analysis methods, anticipating the adverse possible situations, their worst evolution, and forecasting how they arrive to critical states.

The NDM process to save human lives, divided in pre-, during-, and post-disaster actions, still have a gap between knowledge and intelligent and effective procedures [13]. The impossibility to prepare and train under real-life conditions, limits the learning experience of first-responders, civilians, and city-planners. During disasters, communication, and visualization of possible pathways to recovery (since danger could be permanent and/or resources are scattered) is fundamental. In this sense, VR and AR technologies cannot be ignored anymore: they have grown exponentially over multiple markets and is projected to have a worldwide revenue of USD 40 billion by 2024 [14, 15]. The examples of successful applications are varied and motivating: simulation of floods, ground motions due to earthquakes, wildfires and hurricanes, around the training, the monitoring, the modeling and the early warning [16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28]. The virtual and augmented scenarios, most of the times, are developed as smart systems (based on artificial intelligence) to improve situations prediction [29, 30, 31, 32, 33, 34].

Between the various studies the use of smartphones is indispensable. The potential and benefits of using AR to access and display disaster data in its geospatial context [35, 36, 37] is normally developed through a mobile application, for example to determine the best route for evacuation from collapsed or burned areas. The Whistland system [38] is an example of retrieving crowdsourced disaster-related information from real-time Twitter feed for display as an AR overlay to the smartphone camera. Mirauda et al. [39] developed a mobile application for connecting measurements from hydrological sensors with an integrated forecasting model. Fedorov et al. [40] presented a unique approach to utilize computer vision techniques to identify mountain silhouettes and compare the extracted information to available DEM (digital elevation model) data to provide useful information (e.g., peak name, height, lithology, associated risks, etc.).

About AR/VR solutions the best models are those developed for visualizing data and analyzing information in-situ in smooth transitions between virtual and augmented environments [41]. Ready et al. [42] presented a virtual reality application for HTC Vive that recreates a 3D model of terrain and buildings (geographic context in Japan) to interact with various data resources, mainly to access hydrological time-series in an easily interpretable way for disaster management. Haynes et al. [43] developed a mobile application to visualize potential floods through integration of real-time recordings (e.g., water level, soil moisture, and humidity), being extremely important that stakeholders can observe situations from the application and make decisions consistent with what happens in the field. Macchione et al. [44] proposed a virtual environment (open-source 3D graphics app, i.e., Blender) to recreate an urban environment (e.g., buildings, streams, roads, levees, textures) to simulate hydraulic dynamics during different flood scenarios.

3. VR: AR basics

Virtual reality (VR) is a technology with which digital environments can be built for the immersion of participants. In these environments, users have the possibility of interacting with the environment and with each other in deeply realistic ways [45]. The ultimate goal of VR is for participants to experience total immersion and prevent, in every way possible, from perceiving stimuli from the outside (real) world [45]. However, VR also involves non-immersive and semi-immersive settings (i.e., when the experience is in a desktop) [46]. On the other hand, Augmented reality (AR) concentrates on the superposition of virtual or digital elements on real world scenes. The participants can “view” the real world, but it is composited with digital items [47]. AR expands the real-world rather than constructing a virtual one. The computer-generated material improves understanding or understanding of what is happening in reality [48]. Mixed Reality (MR) is a hybrid technology where virtual objects are merged into a 3D- atmosphere or real objects are positioned into a virtual creation (Figure 1).

Figure 1.

Application example for the Federal Electricity Commission in Mexico, the team of engineers works a) in a geotechnical laboratory with an application that allows them to learn more about the specimens being analyzed, share the results with supervisors (in real time) and generate the log files using the lenses; b) in discussing projects in a more immersive way in which all sources of information can be displayed to each team member can understand and discuss without bias.

Another definition of VR is a specific collection of technologies (headset, gloves and walker) that allow participants to feel that they belong to a digitally created world through high interaction between their senses and the artificial environment [49, 50, 51, 52, 53, 54]. The 3D simulation (width, height, and depth) of a real or made-up situation can be experienced visually in real-time motion (or the closest thing to this) and supported by sounds and tactile stimulus, or the necessary feedback to recreate an integral experience. With VR, users can envisage, operate, and relate multifaceted data with surprising ease [55]. Therefore, VR refers to an immersive, interactive, multi-sensory, viewer-centered, 3D computer-generated environment and the combination of technologies required to build such an environment [56, 57]. The stereoscopic ambience enables the observer experiences in deeply immersing scenes. Exploiting the human brain responses, VR dilutes the boundaries between persons and computers. Because of our ability to see the environment three-dimensionally (stereoscopic vision) VR can create right and left eyes images of an object or a scene and the observer’s brain integrates these stimuli from the presented perspectives to generate a whole sensitivity of a space. The virtuality means that an illusion is created about screen objects beyond the information on 2D displays in monitors. In VR, as in other technologies (2D-CAD even the 3D version), it is tried viewers notice distance and spatial interactions, but more convincingly and precisely.

3.1 Components of VR/AR

An augmented reality system, from a hardware perspective, consists of a sensor(s), processor(s), and display(s). AR systems superimpose computer graphics imagery on the real world, and, for this blending, the observer’s positions must be known (with extreme precision), stored, and related with the positions and geometry of the items that are to be projected on the real environment. From a general perspective, it is quite easy to draw the three-dimensional images that you want to superimpose on what the user observes in the real world. The task to be solved is to fully define a correct perspective while defining as accurately as possible the position of the observer’s eye. To offer a truly useful experience, portable and easy-to-use optical systems must be considered.

The AR processor coordinates and analyzes sensor inputs, stores, and retrieves data, carries out the tasks of the application program, and generates the appropriate signals to display. Computing systems for augmented reality can range in complexity from simple handheld devices such as smartphones and tablets to laptops, desktop computers, and workstation class machines all the way through powerful distributed systems. The scene must be updated smoothly and at a rate that the participant in the experience perceives as a constant stream of information. AR applications must sustain a frame rate of at least 15—preferably more— frames per second for the participant to perceive the display as continuous. Displays that are simulating the feel of a solid object must be updated about 1000 times per second or else the object will feel “mushy.” To superimpose information (graphics or data) stored in a computer (control center) on the actual environment, a device called a beam splitter is typically used. This divider does not divide but combines the images of the real environment with those placed in the monitor environment. As a result, the viewer is presented with a double exposure photograph. Because the optics are typically fixed, in AR systems there is only one depth at which both the computer-generated imagery and the real-world imagery are in focus. If real-world and virtual-world scenes are both in focus, it will be easier to perceive them simultaneously.

By the other hand, the components necessary for building and experiencing VR are divided into two main components, the hardware and the software mechanisms. The hardware components are composed of computer workstation, sensory displays, process acceleration cards, tracking system and input devices. The workstation is an object (computer or microcomputer) that is intended for technical or scientific work. These objects are typically used by one person at a time, but are connected to a network to facilitate multi-user operations. Sensory screens are the artifacts that are responsible for displaying virtual environments. These displays can be as basic as common computer display units to head-mounted displays (HMDs) with headphone mounts for a 3D viewing and audio experience.

When over-the-head (helmet) displays are used, the presentation of the images is given right in front of the viewer’s eyes. With the helmet’s sensors (the orientation ones) the artificial segments are controlled that make the user experience a complete virtual environment. In most cases, a set of optical lens and mirrors are used to enlarge the view to fill the field of view and to direct the scene to the eyes [58]. It is fundamental to have process acceleration cards to update the display with new sensory information and the tracking system (mechanical, electromagnetic, ultrasonic, and infrared trackers) that follows the position and orientation of a user in the virtual environment.

There are also VR software key components: 3D modeling software, 2D graphics software, digital sound editing software and VR simulation software. 3D modeling software is used in constructing the geometry of the objects in a virtual world and specifies the visual properties of these objects, and 2D graphics software permits to manipulate texture to be applied to the objects which enhance their visual details. With the digital sound editing software is mixed and edited sounds that objects make within the virtual environment and with the VR simulation software the components are put together. It is used to program how these objects behave and set the rules that the virtual world follows.

In this research is presented a mechanism of Mixed Reality (MR), which combines Virtual Reality and Augmented Reality, called also ‘Hybrid Reality’ or ‘Extended Reality’. MR [59] is understood as a proposal that can be placed anywhere between digital environments that belong to complete virtuality up to the absolute perception of the real environment. MR works with concepts and objectives of AR and VR (Figure 2). The use of helmets or glasses allows the user to enter a digital world with all the information that could be useful. The continuity between reality and virtuality is the basis for the interaction between objects in the physical world with items in the virtual world, which will finally be the future of these technologies.

Figure 2.

Reality roadmap according to Intel®, which prefers merged reality to mixed reality; the term “mixed reality” has been thrown about a lot as of late but pinning down a precise definition has proven elusive.

4. Application for earthquakes

Modeling objects and the building structure in the virtual world is the first step for environment construction. In this research the employed techniques entail software for modeling/constructing bi- and three- dimensional objects and backgrounds. Once the 3D modeling of the objective scene is built, the model brings in the VR/AR mechanisms (in this research is used Unity3D-Unreal), in this way the static model is converted into a synergistic environment. Based on the experiences modeling urban infrastructure, these alternatives are appropriate for metropolis projects. Then the virtual contents (movements, gestures, and commands) were established. Physics engines were sometimes used to assist the simulation of emergencies, i.e., for a building shaking according with the accelerations registered during an earthquake, in each virtual room the movements of the room must be experienced, but in addition to the objects that could also move and even fall.

An avatar is used to manage the commands that will make the experience in the virtual environment one with dedicated goals. The avatar is the representation of the person, and it reflects the response behavior (of him and/or the participants) to the stimuli of the virtual world. For an avatar to move and act, a device such as a mouse, keyboard, pad, and telephone (in the 2D case) and gamepad and joystick (for the 3D offer) is used. The AR phase contains, in addition to the possibilities of movements and executing commands, data and information from the real world that could be useful for the designed task. Cameras and sensors are widely used and combined with the image recognition function of an AR engine. HMDs (glasses) provides the ergonomic solution for a virtual environment task using direct vision. Users can experience high-level immersion and better react as if they were in the real world.

The September 19th, 2017, a M 7.1 earthquake hit Mexico City causing 370 people were killed, over 6,000 were injured, 230 totally collapsed buildings and more than 7 000 small houses were damaged. The situation required the intervention of hundreds of geotechnics and structural specialists to qualify conditions and to permit people to return to their homes (Figure 3). Exactly 32 years before (Sept 19th, 1985, more than 10,000 deaths, 30,000 destroyed buildings, and 68,000 injured people) a M8.1 event had showed the fact that the city is built on the unstable and sinking ground of a dried-out lakebed and this promoted a better geo-zonification of the risk.

Figure 3.

The Mexican earthquake from the 19 September 2017 led to significant building damage in the capital Mexico City and the states of Morelos and Puebla. The damage data in houses/buildings and soils characteristics highlights the correlation between damage drivers that little has been studied or they have not been fully understood.

In 1985 the functioning of the city was failed for months, the reconstruction took years and led many to relocate from the most affected areas to the city’s outskirts in search of the safety of the bedrock. The districts affected in 1985 and 2017 events were quite different and, because of this, the type of damage and the technical needs to manage rescue activities were also distinctive. These experiences have undoubtedly shown that for designing an efficient post-earthquake relief plan, it must be recognized that Mexico City is particularly exposed because of its huge population and strategic importance for the country and that this vulnerability has grown the last years due to the expansion of the urban settlements in risky areas (nodules of extreme poverty), the environmental devastation, the deterioration of life levels, the economic activities concentration that require the transport of substances (water, potable and used, gas and hydrocarbons) by underground infrastructure, and the growing complexity of transportation process.

Despite the best intentions and the enormous efforts of the governments that have attended these emergencies, the situations dangerously evolved and the period in which the city was detained, and the population subjected to chaos, spread out for months (or even years in 1985) damaging, primarily, the poorer and fragile (socially) population centers. Hereby, there is an increasing need for a holistic and more efficient natural disaster management.

The application that is explained in the following is VR-AR engine to train and to administrate the three phases of this kind of NDM: preparedness, response, and recovery. This VR instrument has the capability bridge the gap between knowledge and action with the necessary velocity and efficiency when disasters happen. Exploiting the scenarios reconstructed, the first-responders, civilians, and city-planners can be prepared to work under a series of conditions built in virtual/augmented-life, expanding the learning and response experiences. The heterogeneity of soils properties (and responses), earthquakes damage intensities, and the actions effectiveness (professional teams and their capacities) is displayed in the digital metropolis based on the results from neural topologies that predict the spatial variability of risk components (exposure and vulnerability).

4.1 Pre-emergency preparedness

For the preparation of engineers and specialists to perform i. state declaration (for soils and structures), ii. routes recognition (safest and more efficient), and iii. Provision of resources, it was necessary to construct specific VR-AR tools. The training under real-life conditions goes in two directions: geo-situations and structures. Simulations are about first-responders’ reactions for specific scenarios: 1. extreme vulnerable communities, 2. structural pathologies (residential homes and multi-family buildings) and 3. buried infrastructure and 4. soil masses.

The immersive experience of an earthquake (during the shaking and/or immediately after it has happened) is simulated in locations where fragile infrastructure coupled with danger zones (susceptible soils). In Figure 4 is shown how is recreated the structural masonry commonly used in poverty nodes in Mexico City. The structuration refers to the practice of using masonry, brick, or stone, as mass self-supporting. It is one of the oldest building methodologies, and by far the most resilient, however when self-construction (the inhabitants develop the structuring without the support of engineers or architects) does not follow basic rules for the placement of vertical and horizontal reinforcement elements (columns and beams) and the mezzanine and ceiling slabs are extremely light (even without steel reinforcement), the response of these units to ground movements is very unfavorable.

Figure 4.

A training room: after an earthquake M8, subduction zone, site Lake zone, failure conditions for a concrete frame system (1 floor). Analyzing cracks in walls and columns the user is being trained to correct classify the damage level after an extreme earthquake.

The VR-AR tool (principally the VR box) drives the user to detect first the kind of cracks, fissures or other pathologies presents in a house and to relate them with a classifying guide (Figure 5). After this, the observer must look for overgrown trees and shrubs, cracked drains, leaking rainwater goods, as some of the things that may lead to structural problems but that are not related to the seismic inputs.

Figure 5.

A training room: after an earthquake M8, a sinkhole has formed, and it is necessary to determine the sources that erode/dissolve as well as call the specific help team. The repair conditions are stored in the event DataMart.

Part of this investigation, but still in a preliminary stage, is the issue of the entry and exit routes of the emergency teams. The evacuation of threatened communities and assistance to those potentially blocked by the effects of an earthquake is a complex and vital issue. Transportation system is conceived as the role that sustain the economic and social well-being of the communities so disaster or extreme hazard such as earthquake has a major impact on the resilience of the communities. In Mexico City suburbs, road infrastructure is linked to many factors such as users, climate, economic level, material, topography, and periodic maintenance, therefore, part of the real-time analysis for the planning of entry and exit routes must be the superposition of layers on information from massive sources (traffic control systems) and the spatial variation of these susceptibilities. A comprehensive review of social infrastructure (hospitals, schools, recreation centers, markets, department stores, fire and police stations, etc.) must be done to detect temporal routes options as part of the adaptive routing solution.

4.2 Response during the emergency

The Response during Emergency refers to the actions that people may take, in the case of earthquakes, immediately after the ground movement has ended. The emergency period, when the events are extreme, can be extended depending on the size of the heavily damaged areas or in which the collapses have occurred in public places with large concentrations of people. The AR box was developed to assist technical crews which are organized according to the risk levels at each site. The entrance to housing units is categorized according to their structure and size: types of walls, types of columns, types of slabs, types of foundations and, on the other hand, single-family houses, multi-family houses, light and small buildings, large buildings.

In this stage of NDM, the personal is in real danger so it is mandatory that they have a support outside the damaged site: an automated or human control center. The superposition of information about the structures and materials permits rescue teams to take immediately decisions about evacuations or calls for additional help (Figure 6). Also, it is important that the field teams are provided with a dedicated storage for the found conditions. Once the in-situ situations are loaded, they are integrated to a set for specific analysis (feed forward neural network) that defines the risk level and spatially categorize the geographical situation. In this way, state maps are built in real time. Additional maps can be displayed using the stored information, for example by factor, by action demanded (posterior attention, evacuation, or human rescue) and by supply (call for requirement of special equipment/machines or other inspection/rescue teams) (Figure 7).

Figure 6.

Immediately after the earthquake has ended, the trained teams go out into the field and begin to verify the conditions through the lenses that are communicated with the control room, a) since street, an engineer is checking a multi-level building that has lost the walls on one side of its perimeter, b) inside a 1-story house, a user checks the cracking, catalogs it and stores it.

Figure 7.

Example of the integral maps: peak ground accelerations PGA superposed on the number of damaged houses (1 to 3 floors) -red circles- in a small region of the Mayor’s Office of Tláhuac, southeast of Mexico City.

In addition, there is a section of AR toolbox that is exclusively dedicated to the attention of leaks. Water and gas, the latter considered a priority due to the secondary effect of the explosions, are attended by specialized professionals that work in accordance with specific regulated policies (Figure 8).

Figure 8.

The status review of the pipelines that transport dangerous substances (in this case gas) is essential to qualify the inhabited regions as safe. The application created for one of the largest companies in the country is shown. The user has the information about the arrangements in the field as well as the readings that must be verified on the special monitors. At all times, the user can be assisted by his human colleagues in the control room.

In some zones of the metropolis the vulnerability of the soils to the arrival of seismic waves is very high. The most superficial layers in these areas suffer cracking and subsidence processes among the most alarming scales in the world (Figure 9). The periods of drought and torrential rains aggravate the susceptibility to the collapse/cracking. This situation maintains small buildings and buried facilities in a very susceptible state, making them prone to be more affected when an extreme earthquake hit. The coincidence between construction deficiencies (poor technical conceptions) and degradation of materials (highly deformed, cracked and collapsed soils), when the seismic waves arrive, constitutes a challenging scenario.

Figure 9.

In the southeastern of Mexico City, where the poorest neighborhoods of the metropolis are concentrated, during the 2017 earthquake manifestations such as cracks, subsidence and collapses (steps) in the most superficial soils were exacerbated. This had the effect of the uninhabitability of thousands of homes. Since then, the city government undertook one of the largest reconstruction campaigns in the modern history of the city.

The toolbox for geotechnical engineers, intend to train them to detect aspects about differential subsidence, sudden deformations, and sinkholes. In the immediate aftermath of a major earthquake, they must also learn to catalog an additional symptom of deterioration: cracks and fissures in natural masses. This is particularly important as damage to build units can develop days after the seismic event if the openings in the soils and rocks are not properly treated (Figure 10). The registered details are stored and sent to the control center where are analyzed with a CART (classification tree) to determine if the site is on the “zero set” (cases where their conditions are on the top of risk levels and demands immediate actions).

Figure 10.

The vision of the user in the field is shown, a) in the Tláhuac area, in a serious crack, he observes the information on geotechnical zoning, b) check, in one of the most damaged areas, the repair of the drainage ducts, observes the optimal filling conditions according to the regional sinking map c) at site, where a leak and step deformation are present, infers which is the damaged section and record it in the application to call a specialized team.

The machine learning analysis is based in layers of Geo-descriptors that permits to qualify the susceptibility to sink-fracture in specific Mexico City regions. With the analysis of 6 variables (Geo-position, Soil heterogeneity, Groundwater Level, Urban loads, Type of foundations, Use of nearby streets) a CART (Figure 11) determines the relative influence of each one on the cracks’ development and relates present conditions to a risk level. The user of the tool can request from the control center the result of the evaluation with CART so that he, when faced with any doubt about the state of the soils, can make a decision and qualify the situation. For example, let us examine a site in the southern poor, and very susceptible to cracking, region in the metropolis. In Figure 12 the user’s vision when entering the site is shown. The characteristics of the breaks that must be recorded are geometry, materials (under consideration of soil evaluation according to the SUCS unified soil classification system), relative movement between the flanks of the crack/ step/subsidence, among others. When the user asks for the CART response for this site, the geotechnical information (boreholes) that the AR-tool finds near the site is first shown and then the evaluation is presented with a disaggregated description of the factors that lead to that level of susceptibility. In the case exposed, the presence of a non-continuous thin layer of semi-rigid material embedded in the plastic clay, its proximity to the battery of water pumping wells (operating at relatively shallow depths) and the shape and depth of the rigid base, are the conditions that drives to the site to a high susceptibility to crack.

Figure 11.

CART for conclude about the susceptibility to crack. The model uses (a) information from maps of geo and anthropic properties and (b) survey of field damage. This tool (c) permits the user to qualify the susceptibility of a site to the development of a crack. (d).

Figure 12.

Conditions at the entrance of a trench found under a foundation (3 floor house). The case is studied with the AR tool because of a call after finding a deep crack. The user of AR can measure the geometry and send/store this data. It also has information from nearby geotechnical surveys that help him interpret what observes. The videos of what happens are recorded for later analysis.

4.3 Recovery post-emergency

When the conditions of habitability and urban services have begun the path towards normalization, government administrators and practitioners (particularly engineers dedicated to the generation of infrastructure) should consolidate the assistance programs (repair of minor damages in houses and buildings as well as attention to water pipelines) and larger-scale plans for the correction of structures that are considered not to follow what is established in the code or that require the application of complex engineering solutions (structural reinforcement, total or partial demolitions, complex restoration, control of settlements (fill), repair of sinkholes, etc.).

In this case the teams (engineers, designers, government, and urban planners) are provided with integral tools to evaluate causes-effects and directly determine how many resources the city and community needs to correct the risk situations and how to improve its resilience for future, which is one of the most effective long-term strategies. These virtual collaborative immersive spaces allow different experts from all over the instances to work together in the same virtual environment, creating enhanced and coordinated solutions. Resilience, poverty, security, among others, are social aspects that are presented in maps in superposition on the kind of damages and necessities of reparation. In this way the decisions about resources can be developed on a solidarity base that prioritizes the needs of the most marginalized communities. An additional aspect, very important for the NDM, is that one of the layers shown is the prediction of accelerations in the different geotechnical zones of the city when certain earthquakes attack the metropolis. With these interpretations and the effects, routes can be drawn for the adaptation of services and infrastructure to anticipate the potential scenarios of shutdown of functions in the event of a mega earthquake.

5. Conclusions

For gathering a more organic, equal, and inclusive world with no one left behind, there is an urgent need to transform the current unsustainable interactions within social ecological systems. The role of emerging technologies in achieving harmonious interactions is crucial. The tools presented in this investigation show how they can be used to enhance resilience to environmental disruptions, particularly earthquakes threat. It has been showed how Artificial Intelligence, VR and AR can improve natural disaster management closing the gap between knowledge and action.

VR-AR technology provides visual simulations to create a vivid first-person experience. Temporal, spatial, and social differences are lowered by immersing people into a certain location or experience. This immersion can be exploited in the three phases of NDM: preparedness, response, and recovery. Artificial Intelligence + VR-AR is an innovative addition to NDM as it provides a non-destructive and safe technique to recreate natural disasters. The simulation of future conditions using the parametric relationships found by NN improves the understanding of what kind of damage is caused, how to better prepare, and shows the possible ways to recovery.

NDM undoubtedly benefits from the adoption of VR-AR technologies. These tools are most advantageous and attractive when data, models and strategies are included from all possible sources involved in preparing for, responding to, and recovering from the effects of a natural phenomenon. The integration of geological, seismic, geotechnical, urban, service, social data, to name a few, in an interchangeable and visually stimulating format, directly impacts the effectiveness, cost and execution time of activities related to each stage of a NDM project. The proposal presented in this document was used as a prototype by some teams during the 2017 earthquake and showed its great potential to create routes that increased productivity and promoted more agile and socially supportive decision-making processes, reducing duplication of work and some traditional activities prone to big mistakes.

The requirement derived from the collaboration between actors from different branches of knowledge (from scientists, technologists, engineers, administrators, politicians to representatives of civil society) when an earthquake occurs in a large metropolis, and especially if it is of enormous complexity such as Mexico City, will increasingly force the use of tools that allow raising quality in i) training in predicted scenarios on responses of soils and rocks interacting with buildings and buried infrastructure, ii) the conceptualization of risk management and the monitoring of activities that affect the resilience of the most vulnerable populations and iii) the interpretations of behaviors, relationships, patterns and models with intelligent tools that allow exploring a multiparametric, highly dimensional, extremely complex natural/anthropic universe.

The aspect of absolute and partial immersion, the congruent and integral intelligent modeling, the exploration of the programming and the risk management tools, as well as the interoperability, interactions and exchanges between the actors that generate huge amounts of data in all types and formats, without a doubt, it is a monumental challenge not only to the exercise of those who are in charge of NDM but to the adjustments to the analytical and mental connections, with which we have historically directed the responses to the demands of emergency situations, however the recent experience has shown that the application is effective and promising.

Acknowledgments

The authors would like to express their very great appreciation to the Government of Mexico City and to the Commission for the Reconstruction. Also, we would like to thank the following company for its assistance with the collection of the data and to share experiences: LOHRE S.A. de C.V.

Conflict of interest

The authors declare no conflict of interest.