Typical datasets available on a construction project.
This chapter introduces GoogleEarthWork which is an augmented geographic information system (GIS) based on Google Earth to manage and visualize heterogeneous site information, especially 3D models, aerial and ground images, panoramas, and GIS data of the site environment. The concept is to realize a highly automated end-to-end earthwork construction planning system that is able to generate project management deliverables from heterogeneous information and enhance the usefulness and intelligence of GIS for better project planning and control in earthwork construction. With identified constraints from the augmented Google Earth, the earthwork planning problem is formulated, and an optimized executable plan can be automatically generated, including work breakdown structure and project network model. Demonstration cases are provided to prove concepts of and illustrate functionalities of GoogleEarthWork in support of earthwork construction planning in realistic settings.
- Google Earth
- Keyhole Markup Language
- earthwork construction
- automated planning
Construction project planning and control requires an integral and comprehensive understanding of the construction site. During the planning process, a large volume of data are collected and created to identify potential problems on the construction site and select proper construction methods and procedures in order to ensure safety and on-time delivery of the project. Such data include (1) as-planned information that describes the design and the scope of the project, (2) as-built information that describes the actual situation on the construction site, and (3) environmental information that can be used to evaluate the impact of the environment on the project and the impact of the project on the environment. At present, engineers and project managers can be overwhelmed with various information coming from different sources (as listed in Table 1); however, maintaining large-volume heterogeneous datasets would become a big burden unless they can be linked and managed together to enable efficient information retrieval and facilitate problem identification .
|2D drawings (as-designed)||Design|
|3D models (as-designed)||Design/construction prototyping [2, 3]|
Site layout planning 
Crane path and lift planning 
|Images/videos (as-built)||Site inspection and reporting|
As-built modeling [6, 7]
Progress monitoring 
|Laser scanning (as-built)||As-built modeling [9, 10, 11]|
Progress monitoring 
|Satellite images, topographic data, et al. in GIS (environmental)||Site layout planning [12, 13, 14, 15, 16, 17]|
Route planning 
Data management and visualization [1, 18, 19]
The adoption of advanced sensing and information management technologies in construction is greatly hindered by (1) high expenses on system development yet unclear benefits of implementation [20, 21, 22], (2) inefficient visualization and oversimplified site modeling methods for coping with complicated site environment , (3) insufficient integration and interoperability [23, 24], and (4) technology barriers and organizational difficulties in information sharing and distribution [20, 25].
Several technologies have been applied on project information management and visualization, including building information modeling (BIM) , augmented reality (AR) [26, 27, 28], the integration of BIM and AR, the integration of GIS and BIM, and Google Earth, as listed in Table 2.
|AR||3D models + images|
|BIM + AR||3D models + images|
|GIS + BIM||3D models + satellite images + topographic|
|Google Earth||3D models + images + satellite images + topographic|
BIM demonstrates great potential to model rich geometric and semantic information of a building object but lacks the capability to incorporate as-built and environmental information. AR has gained substantial attention lately due to its capability to combine site photos and as-planned 3D models. However, the absence of an accurate model of the surrounding environment, for example, those 3D site models generally provided by 3D GIS systems, makes AR less instrumental in construction engineering applications that demand the representation of frequent, intensive interactions and relationships between the facilities being built and the site environment, especially where the project is situated in crowded cities or environmentally fragile areas. Researchers have also leveraged on the benefits of integrating BIM and AR [29, 30, 31, 32]. Nonetheless, incorporating AR into BIM software is still practically infeasible due to inherent limitations of BIM software in handling large external datasets for real-time rendering .
GIS has achieved significant success in managing large-scale heterogeneous spatial information. Considerable attention has been placed on the integration of BIM models and GIS so as to integrate the indoor as-built information and the outdoor environmental information [33, 34]. To tackle unstructured data, researchers utilized variants of Extensible Markup Language (XML) to develop shared project information models thanks to its extensibility and interoperability on the web schemas [35, 36, 37]. Both the open source BIM standard of industrial foundation class (IFC) [38, 39, 40] and Web GIS formats (including LandXML [39, 41], City Geography Markup Language (CityGML) [42, 43, 44, 45], and
The integration of BIM and CityGML provides an effective means to manage indoor building information and outdoor environmental information. However, it lacks the functionality to support AR modeling based on site photos or videos. In contrast, KML—which represents a markup language specialized for data modeling in Google Earth—focuses on data integration and visualization. It provides various data models to support advanced visualization techniques including AR. In , KML was used to visualize building energy simulation results integrated with BIM. Another related endeavor  proposed the use of KML and Google Earth to generate a cost-effective site information management platform which integrated site photos, 3D models, and the building environment.
In this chapter, we introduce an augmented GIS system called
2. GIS-based site information management and visualization
Google Earth has been widely used by scientists and relevant stakeholders in addressing environmental and construction planning issues thanks to its ubiquity and rich geographic information. Diversified geographical information is presented to the user through a combination of digital elevation models, satellite imagery, 3D building models, street views, and user-uploaded images. Features such as tiling and level of detail (LOD) for images and 3D models enable Google Earth to manage large datasets with ease and efficiency, eclipsing majority of BIM software. Besides, KML enriches the extensibility of Google Earth significantly by providing users a standardized language to add data and customize analyses. With temporal and spatial information associated with each object, Google Earth enables efficient information retrieval through content navigation, 3D exploration, and time window filtering.
Photogrammetry algorithms are also used in order to align unordered images within the WGS84 coordinate system adopted in Google Earth. Panoramic views and 3D reconstruction of the construction site are produced to facilitate a better comprehension of the construction environment. The resulting 3D point cloud captures the geometry of the construction site and is thereby used for cut/fill volume takeoff, as well as measuring the hauling distance between two areas. As-planned models are converted in the KML format and time-stamped in order to visualize the construction progress. The system provides stakeholders with a visually intuitive platform to perceive the construction site and identify potential problems such as spatial limits in connection with site accesses and site layouts through integrated information visualization. By storing data on the cloud, KML enables efficient management of large volumes of data in images and models. Sharing KML documents of limited size instead of original datasets also streamlines information distribution and improves computing efficiency performances.
2.1 Data collection and preprocessing
Google Earth provides project managers with free high-resolution satellite images and topographic information of the environment around a construction site. Such information is essential to plan for site accesses, site layouts, and traffic flows. As-planned information in 2D/3D drawings is crucial for scope definition, quantity takeoff, and progress monitoring. For earthwork projects specifically, the as-designed surface is required to take off cut/fill volumes. Besides, the structures being built also affect site accessibility and traffic flows.
For as-built information, site photos have been widely used on a construction site for updating construction progress and reporting safety issues or other problems. However, images collected by different personnel are barely reused due to lack of efficient image management tools. It is desirable to automatically organize images with locations in a GIS system, but the positioning accuracy of mobile devices is inadequate for two main reasons, namely, (1) low-end localization sensors embedded in mobile devices and (2) multipath effect of radio frequency signals. In general, the camera pose obtained from a consumer-grade mobile device does not satisfy the need for geo-referencing and AR applications. Higher positioning accuracy can be obtained from aerial images taken by UAV due to high-grade localization sensors embedded and lessened multipath effects. After bundle adjustment , the camera pose can be further improved. By taking the optimized geo-location of aerial images as references, ground imageries can also be precisely aligned in the physical coordinate system. In addition, 3D reconstruction from images is instrumental in quantifying cut/fill volumes of earthmoving jobs and fixing distances and slopes of haul roads in earthwork construction planning. Most recent research endeavors  have demonstrated the cost-effectiveness of UAV photogrammetry for earthwork volume estimation.
Structure from motion (SfM)  has been well studied in photogrammetry and computer vision domains to reconstruct the 3D structure of the scene from image collections and to recover the pose of these images. Taking unordered images as inputs, SfM outputs the precise image position and orientation, plus 3D reconstruction of the site as point cloud or model. Besides, high-resolution panoramas stitched from aerial photos are cost-effective substitutes for outdated low-resolution satellite images. As an incremental approach, SfM is suitable for processing construction site photos collected on an irregular basis along the time line. However, it requires redundant images in order to ensure “realism” of the scene. This is usually not assured when ground photos are taken by different personnel on a construction site. Therefore aerial images taken by UAV are used to materialize connecting and aligning scattered ground images. With a sequence of imageries taken on the construction site, the system implements the SfM procedure, starting from the first aerial imagery and taking it as the reference in subsequent processing of images taken by cell phones on the ground.
The direct output of SfM includes the camera pose and a 3D point cloud of the object. A much denser 3D reconstruction of the object can be achieved using stereo matching subject to coplanar constraints . To visualize the 3D reconstruction in
2.2 Information integration with KML
Based on XML, KML uses a tag-based structure with nested elements to manage data and information associated with an object in a hierarchical manner. Different from CityGML which is designed to represent geometric objects, the strength of KML lies in visualization on a web-based GIS platform. It defines basic elements to represent geometric objects, raster images, as well as their visual effects. Elements predefined in KML are divided into several categories according to their functionality:
|<Model>||3D model representation and visualization||3D models|
|<GroundOverlay>||Raster data alignment and overlay on Google Earth terrain||Panoramic mosaics|
|<PhotoOverlay>||Image placement and orientation for AR visualization||Original images|
|<Camera>||Camera position and orientation for AR and navigation||Image pose|
|Associate date/time for 4D exploration of objects and activities||Schedule|
|<ExtendedData>||Customized data organization and visualization||Documents, webs, et al.|
Objects defined with elements in the
Aerial images (which are taken by UAV) provide a unique view angle of the construction site with fewer obstacles. Besides, these images can be taken on a periodical basis to capture updates and progress on site. The stitched panoramic image has much higher resolution than satellite images available in Google Earth. The <
On a construction site, ground imageries are usually taken at “random” locations and angles. Consequently, they are fragmented in nature and only used as evidence shown in documents in practice. However, by aligning the image at the exact location and orientation in relation to 3D models and the site environment, fragmented information provided by individual images can be well organized and seamlessly integrated. Different from real-time AR technologies which demand considerable computing resources and remain too expensive to implement on site, the <
An example of information integration in
2.3 Constraints identification in earthwork planning
Analytical simulation or optimization for construction operations planning requires knowledge of practical constraints on the construction site so as to make a sufficient problem definition. In rough grading, a certain volume of earth needs to be excavated at one area and filled at another. Accessibility issues during project execution become the primary concern for earthwork construction planners, especially when only limited accesses between site areas are available at the very beginning of the project. Moreover, earthmoving operations need to be executed in a safe, efficient manner, accommodating many concurring construction activities on site.
These site constraints can be categorized into quantitative constraints and qualitative constraints, as listed in Table 4. A quantitative constraint can be defined with a number; by contrast, qualitative constraints cannot be quantitatively represented in
|Quantitative constraints||Qualitative constraints|
Obviously, site photos provide valuable information to identify qualitative constraints. The accessibility issues, site layout constraints, and road conditions can also be assessed with high-resolution panoramic images and/or ground photos on computer. As these images are geo-located,
3. Earthwork optimization and planning
Given identified quantitative and qualitative constraints, the analytical method presented in [56, 57] will be introduced for automated earthwork construction planning. This method provides an analytical approach to plan rough grading operations while making problem formulation and modeling more intuitive and simplified by the use of material flow networks. To a certain extent, it can potentially eliminate temporal-spatial conflicts (such as trucks are not allowed to haul on ungraded areas) in generation of an optimized yet more practically feasible work plan. The two-phase approach splits
The architecture of the two-phase approach is illustrated in Figure 5. At the bottom, an earthwork optimizer based on a material flow network is developed to optimize earthwork operations subject to identified quantitative and qualitative constraints. The optimization result is then taken as the primary input for ensuing analysis by the earthwork planner, which generates haul jobs, defines inter-job relationships, and produces the project network model for project scheduling and control.
3.1 Earthwork optimizer
The earthwork optimizer in
Prior to delving into the core of the earthwork optimizer in
The demand is the amount of flow that is required by this vertex. If , the vertex is demanding material to flow in. It is also called a sink node. On the contrary, it is a supplying vertex also named as source node if . Otherwise, the vertex will be a transshipment node with . The capacity indicates the maximum flow allowed on each edge. The cost is the unit cost to transport each flow unit through individual edges, respectively. The flowspecifies the amount of flow on each edge.
The total cost of a flow network is defined as:
where represents the flow variables indicating the amount of flow on an edge. The optimal flow can be found by applying the minimum-cost flow algorithms  which minimize the cost function defined in Eq. (2) subject to capacity constraints and balance constraints defined in Eq. (3) and Eq. (4), respectively:
Traditional methods model haul jobs directly by adding links only between cut and fill cells. These methods require predefined hauling paths which may not be explicitly specified in earthwork planning, as hauling paths can be included as variables to be optimized in addition to earth volume assignment variables between cut and fill cells. In , a new method is introduced to deal with the issue without increasing the complexity of problem formulation. In contrast to linking cut cells to fill cells directly, this method links neighbor cells irrespective of whether they are cut or fill cells, while the exact hauling path for each haul job will be fixed by optimization along with the source cell (cut), the destination cell (fill), and the volume to handle for each haul job.
The quantitative constraints such as cut/fill volumes and the traveling speed are directly modeled as the demand for each node and the unit cost for each edge, respectively. The capacity of flow on an edge is typically unlimited unless there is a special need, for example, to limit the total amount moved to a storage area. The qualitative constraints are modeled implicitly in the network structure. They are embedded by adding or removing specific arcs at specific directions. In the following subsections, we will elaborate typical site constraints for earthwork including accessibility, reserved areas, and haul road conditions.
Once the model is established, it is optimized with established minimum-cost flow algorithms . As a result, the earthwork optimizer produces a flow network that defines the amount of flows (defined by ) between adjacent cells. Because it does not model haul jobs directly, the result cannot produce the final execution plan which defines each haul job in terms of source, destination, volume, and haul path. Next, the earthwork planner is introduced which generates the final execution plan based on the optimized earth flow network.
3.2 Earthwork planner
The optimized earth flow network specifies quantity and direction to move material along inter-cell edges (haul roads) in the site system. However, temporal or spatial constraints arising from sequencing earthmoving jobs can be missed in this representation. At the beginning of earthwork operations, only limited accessibility is available. Access to an area is enabled in the middle of the earthmoving process once its neighbor areas are graded. Thus an additional network is required to define the accessibility between areas considering the progress of the project over time. In the remainder of this chapter, the optimized earth flow network is denoted as , and the network to represent the accessibility is named as .
In this step, the classical planning model in automated planning theory is adopted for earthwork project planning in
is defined with a tuple of two directed graphs , where is the optimal earth flow network and is a directed graph representing the accessibility between cells.
is the action space defined as haul jobs. Each haul job can be represented with which specifies the cut and fill cells, together with the volume V and the hauling path P. For example, a haul job indicates 20 units of material which are transported from Cell 1 to Cell 2 passing through Cell 3 and Cell 4.
is a map from to where the optimal earth flow network and the accessibility are updated after performing an action (i.e., completing a haul job.) This includes the following:
Updating the volumes of each cell on
Updating the flow between adjacent cells on
Updating accessibility on after some cells are graded
In the classical planning model, actions are sequentially taken by selecting an action and updating the state as presented in Figure 7. The procedure consists of four steps with the first three steps corresponding to deliberation functions and the fourth step corresponding to state transition functions. Because actions are required to satisfy all material flow constraints (flow direction and flow quantity on each edge), which are already determined in the optimized flow network, the final plan is extracted from a searching space that is already optimized. The detailed explanation of the planner can be found in [56, 57].
4. Case study
In this section a campground grading project located in Northern Alberta, Canada, is used to demonstrate the application of earthwork optimization and automated planner functions. The size of the campground is around 2000 m long and 650 m wide. The total volume of material to be handled is 584,308 bank cubic meters (). The site layout is presented in Figure 8 with color bands denoting deep excavation (>3 m), medium height excavation (1.5–3 m), shallow excavation (<1.5 m), shallow fill (<1.5 m), and medium-depth fill (1.5–3 m). On the west side and east side, respectively, there are two storm water storage ponds, which also provide the two primary sources for fill material in site grading. Note during construction, only limited access to the two ponds is allowed. Pond 1 has one access point on its east side; Pond 2 has two access points on its north and west sides, respectively.
A temporary haul road aligned with a future permanent road is established to facilitate the earthmoving process. Average truck speed differs when a truck hauls on the temporary road or the rough-graded ground. A fleet consisting of a 40 T excavator with a production rate of 190 per day and CAT 740B trucks with 20 volume capacity are employed on this project. The combined loading, dumping, and waiting time is assumed to be 20 minutes. The truck hauling speed limit, irrespective of truck haul (full) and truck return (empty), is averaged at 27 km/h on temporary haul road and on rough ground, respectively. Besides, hourly rates of the excavator and the truck are and . The hourly rate for an equipment operator is around regardless of the type of the equipment.
The construction site is divided into cells (100 m × 100 m) for material flow network optimization and AON network development. The cell size is defined by the user after assessing site topology and application need. Mathematically, the smaller the cell size, the more accurate the result would be. However, too small cell size is not suitable for current application of earthwork planning and construction management. Four times the truck width is recommended as cell dimension for planning mining haul road, which was used for earthwork planning in
In this chapter, we conceptualize an augmented GIS system called
At present, application of
This research was substantially funded by Mitacs Canada and Ledcor Canada through a Mitacs Accelerate Internship grant (Project ID: IT06594). The authors sincerely acknowledge Sam Johnson, Senior Construction Manager of Graham Construction Canada, for sharing information and insight on earthwork planning and Dr. Eric Loo, the Director of Business Development of Mitacs for facilitating the research.