Open access peer-reviewed chapter

Material Classification via Machine Learning Techniques: Construction Projects Progress Monitoring

Written By

Wesam Salah Alaloul and Abdul Hannan Qureshi

Submitted: October 7th, 2020 Reviewed: February 2nd, 2021 Published: March 9th, 2021

DOI: 10.5772/intechopen.96354

Chapter metrics overview

442 Chapter Downloads

View Full Metrics


Nowadays, the construction industry is on a fast track to adopting digital processes under the Industrial Revolution (IR) 4.0. The desire to automate maximum construction processes with less human interference has led the industry and research community to inclined towards artificial intelligence. This chapter has been themed on automated construction monitoring practices by adopting material classification via machine learning (ML) techniques. The study has been conducted by following the structure review approach to gain an understanding of the applications of ML techniques for construction progress assessment. Data were collected from the Web of Science (WoS) and Scopus databases, concluding 14 relevant studies. The literature review depicted the support vector machine (SVM) and artificial neural network (ANN) techniques as more effective than other ML techniques for material classification. The last section of this chapter includes a python-based ANN model for material classification. This ANN model has been tested for construction items (brick, wood, concrete block, and asphalt) for training and prediction. Moreover, the predictive ANN model results have been shared for the readers, along with the resources and open-source web links.


  • automated progress tracking
  • artificial intelligence
  • ANN
  • construction sector

1. Introduction

The construction progress measuring practices are considered indispensable tools for effective project control [1]. Efficient and effective progress monitoring practices provide information regarding performance deviations to the execution plan and help the project management office (PMO) towards timely implementation of control actions to minimise the negative impacts [2]. Currently, instead of manual practices for construction progress assessment, the research community is fascinated by techniques such as photogrammetry, laser scanning, time-lapse photography, etc. Moreover, these strategies have also adopted 4D Building Information Modelling (BIM) as a framework to execute model-based progress tracking of construction projects [3]. In the last two decades, advancements in computer processes and digital camera technologies have allowed construction sector to effectively process [4] and retrieve valuable information from video clips and digital images. Moreover, the applications of computer vision (CV) and image processing systems are now considered in the architecture, engineering and construction (AEC) industry as an emerging field of research with steady growth [5]. Whereas, automated building material classification has gained the interest of the research community with respect to the AEC industry. Automated material classification may increase the performance output of various activities, including defect identification, on-site material control and progress monitoring [6]. The as-is BIM includes the geometric as well as non-geometric information on the building components, including the building materials, which is necessary for energy simulations and 3D structure visualisations. This evolution has led machine learning (ML) techniques to gain popularity for material classification models. Material classification can be performed via laser scan data and image-based data detection; however, the latter is more popular among the research community. The general concept of image-based approaches relies on utilising visual characteristics of building materials such as projection, roughness, colour and shape for automated detection. However, image-based approaches are highly affected by lighting environment. The varying light conditions have a substantial impact on the visual properties of the materials, which create difficulties in classifying the image-based building material. Moreover, the weak textures on surfaces and uncertain points of view often adversely affect the effectiveness and precision process of image-based content classification [7, 8]. Material classification is considered a vital activity of any vision-based framework to generate conceptual as-built 3D models for automatic progress monitoring in construction projects. In the case of construction material, related details can be obtained primarily from the appearance-based details found in 2D images. Digitalised material classification extracts the appearance-based information for construction progress tracking and perform segmentation process for the effective generation of automated 3D as-built models [9].

The implementation of ML techniques is vital for dynamic operations of the systems with continuous and automated learning [10]. Other than pattern recognition, ML technologies are adopted for the self-learning of the big-data based systems connected via the internet of things (IoT) integrated with digital technologies [11]. Likewise, for the construction progress detection technologies, the trend of integration with ML techniques for the digitalisation of the monitoring process has also been increased in recent times [12]. ML algorithms are generally divided into supervised and unsupervised types, which are analysed for learning and prediction of empiric results. Supervised learning algorithms minimise the error between the targeted data and output data, whereas unsupervised algorithms are adopted for clustering data when data training is not preferable. However, both types of ML algorithms can be utilised for the material classifications based on the site conditions and circumstances for the availability of input data [13]. Construction material classification via ML techniques has gained a lot of attention among professionals and researchers in the construction sector. Various studies can be found related to material classification for construction progress monitoring. However, still, improvements are required in the methodologies and algorithms towards effective and efficient outcomes. Therefore, this chapter aims to overview the applications of construction material classifications via ML techniques for progress monitoring/tracking/detection of construction projects by conducting a short systematic review. Moreover, a simple artificial neural network (ANN) based material classification algorithm has also been discussed at the end of this chapter, which will help the readers to understand practical implications of ML techniques in the construction sector.


2. Research methodology

For the achievement of the study objective, i.e., overview the applications of material classification via ML techniques in the construction progress monitoring domain, the literature was collected from Web of Science (WoS) and Scopus for the last ten years. Two different keywords combinations were designed for the collection of studies from the aforementioned databases. The first keywords combination was designed to explore overall automated construction project monitoring technologies, and the final results were sorted for material classification techniques via ML. The second keywords combination was designed specifically for automated construction monitoring practices using material classification via ML techniques. The study’s scope was narrowed down to journal articles and building construction projects, for the last ten years data, i.e. 2010 to 2020. Figure 1 shows the study flowchart of the adopted methodology for this chapter.

Figure 1.

Study methodology framework.

Using the first keywords combination, overall 54 studies were collected on construction automated progress monitoring technologies, out of which four studies were based on material classification via ML techniques. The summary of the data searching and collection for the first keywords combination is shown in Table 1.

DatabaseDurationKeywords combinationTotal collected papersRelevant papers
WoS2010–2020“TS = (automat* AND (construction OR project OR progress) AND (monitor* OR updat* OR track* OR detect* OR recogn*))”544
Scopus2010–2020“TITLE-ABS-KEY (automat* AND (construction OR project OR progress) AND (monitor* OR updat* OR track* OR detect* OR recogn*))”

Table 1.

Data collection summary of material classification-ML techniques with first keywords combination.

Likewise, with the second keywords combination, 48 studies were collected and out of which ten were found relevant to the designed scope of this chapter. Table 2 shows the summary of the data searching and collection for the second keywords combination.

DatabaseDurationKeywords combinationTotal collected papersRelevant papers
WoS2010–2020“TS = (automat* AND (construction OR project OR progress) AND (monitor* OR updat* OR track* OR detect* OR recogn* OR classification) AND (machine learning OR ML OR ANN OR artificial neural network))”4810
Scopus2010–2020“TITLE-ABS-KEY (automat* AND (construction OR project OR progress) AND (monitor* OR updat* OR track* OR detect* OR recogn* OR classification) AND (“machine learning” OR ml OR ann OR “artificial neural network”))”

Table 2.

Data collection summary of material classification-ML techniques with second keywords combination.

Overall, 14 studies were found relevant to the defined scope, which were further analysed for the in-depth review.


3. Discussion

Material classifications via ML techniques are popular among the research community in every domain. However, in the construction progress monitoring practices, the material classification via ML is still an emerging area. Collection of data from WoS and Scopus supports this argument as with the first keywords combination out of 54 technical journal based studies, only four were related to material classification for construction progress monitoring. Moreover, with the second keywords combination out of 48 ML studies in the construction progress monitoring domain, only ten were related to material classification. There are various ML classifiers which are being adopted by the researchers such as random forest (RF), decision tree (DT), bayesian, k-nearest neighbours (KNN), gaussian mixture modes (GMM), logistic regression (LR), support vector machine (SVM), and artificial neural networks (ANN), etc. However, ANN and SVM are the most favourite techniques among researchers, when it comes to material classification [5, 8, 14]. The material classification has been performed by researchers on various sources data input such as digital images taken with the help of a camera [15], smartphones, drones [16], and 3D point cloud models generated on collected images via structure from motion (SfM) [17], or laser scanners [8].

Table 3 illustrates a general summary of the collected studies for construction progress monitoring by adopting material classification via ML techniques. The in-depth review was performed on the collected studies, and it can be observed that material classification methodologies are being adopted for construction progress monitoring practices since before 2010. Zhu and Brilakis [18] identified concrete material regions by testing three classifiers, i.e., C-support vector classification (C-SVC), support vector data description (SVDD), and ANN, where ANN model was found with better outcomes. The model was developed using C++ and evaluated more than hundreds of building construction site digital images. Araújo et al. [19] adopted a functional SVM (FVSM) with a pearson VII function (PUK) kernel and linear functional regression for classifying granite varieties by using spectrophotometer spectrum data. Son et al. [20] developed a model for the identification of concrete structural elements in the coloured images. In the process, the red-green-blue (RGB) colour space was transformed to non-RGB colour spaces to enhance the separability between background classes and concrete. The model was tested for three ML algorithm, i.e., GMM, ANN, and SVM. However, SVM along with hue-saturation-and-intensity (HSI) colour space was found more effective. Yazdi and Sarafrazi [14] used five different classifiers, ANN, SVM, bayesian, KNN, and fisher’s linear discriminate (FLD) algorithm in combinations and as separate, i.e. Bayesian, Bayesian + FLD, KNN, KNN + FLD, SVM, SVM + FLD, ANN, and ANN + FLD on the segmented concrete images. This study found the ANN model as the better option for automatic image segmentation. Dimitrov and Golparvar-Fard [9] proposed C-support vector machine (C-SVM) algorithm combined with texture and hue-saturation-value (HSV) colour features. This study tested the developed algorithm on 20 construction materials (paving, grass, gravel, stone-limestone, formwork, soil-vegetation, marble, metal-grills, soil-mulch, soil-compact, stone-granular, wood, soil-loose, asphalt, cement-granular, brick, concrete-cast, cement-smooth, foliage, and concrete-precast) with more than 150 images for each category and compared various pixel sizes (n x n), i.e., 30, 50, 75, 100, 150, and 200. Better accuracy and effective output were reported for 200 x 200 pixel-images. Han and Golparvar-Fard [3] developed a construction material library (CML) based on C-SVM classifiers with linear x2kernels on 100 × 100, 75 × 75, and 50 × 50 pixel-images datasets of cement-based surfaces, paving, brick, asphalt, formwork, foliage, concrete, marble, gravel, insulation, metal, soil, wood, stone, and waterproofing paint. This study developed an appearance-based material classification technique for progress monitoring using daily photologs and BIM. Point cloud models were generated using SfM and multi-view-stereo (MVS) algorithms from the construction site images. These point cloud models were superimposed with 4D BIM models, and registered site images were back-projected for the BIM elements for extracting related image patches. Testing of the extracted patches was performed with multi-class material classification technique that was pre-trained with the extended CML dataset. Rashidi et al. [5] conducted a comparison study on SVM, radial basis function (RBF), and multilayer perceptron (MLP) by evaluating the performance on building construction materials, i.e. OSB boards, red brick, and concrete. The feature extraction was performed from image blocks to compare the efficiency for detecting building construction materials, and SVM classifier with RBF kernel results were found more precise in perceiving the images for material textures. Yang et al. [17] performed material recognition of windows, walls, protrusions, tile, brick, stone, and coating using image-based 3D modelling. SfM was used for generating the 3D point cloud model for site images. The building facade was modelled as combined planes of protrusions, windows, and wall. Planes were primarily detected from 3D point clouds using random sample consensus (RANSAC) and further recognised as distinct structural components by SVM classifier. Hamledari et al. [16] adopted CV integrated with shape and colour-based modules that automatically detect the interior components using 2D digital images, i.e. three states drywall sheets (plastered, installed, and painted), insulation, studs, and electrical outlets. Cumulative distribution function (CDF) used for electrical outlet module, SVM classifier has an RBF kernel type for drywall, local binary patterns (LBP) for insulation module, and progressive probabilistic hough transform (PPHT) for stud module. The method was validated by indoor construction site images captured by unmanned aerial vehicle (UAV), smartphone and internet sources. Braun and Borrmann [21] developed an automatically labelling process via construction images with 4D BIM and 3D point cloud approach. The 3D point cloud model was integrated with the BIM model, and automated labelling for structural elements was provided with the semantic information. The convolutional neural network (CNN) model was trained on this information to generate classification tasks. The accuracy of the allocated labels was checked by pixel-based field comparison to manual labels. Lee and Park [22] developed automatic reinforcing-bar image analysis system (ARIAS), which could separate the background for the bar area calculation. The model was also able to count the number of bars by testing various combinations between RF and super-pixel method (RFSP), otsu threshold for extracting the bars areas (OTSU), hough transforms (HT), and iterative voting for binary image (IVB). The combination RFSP+IVB gave better output results than other combinations. Ghassemi et al. [23] proposed the material classification model based on deep learning (DL) algorithm for classifying in various illumination conditions and different camera angles/positions. Eleven construction materials (sandstorms, paving, gravel, stone, cement-granular, brick, soil, wood, asphalt, clay hollow block, and concrete block) were classified in this study, and good accuracy was achieved by VGG16 algorithm, even for images that were hard to identify by the human eye. Yuan et al. [8] proposed an automatic material classification method with the help of 2D digital images using the graphical characteristics of building materials. A coloured laser scan data was generated using a terrestrial laser scanner (TLS) with a built-in camera, which contained the surface geometries, material reflectance and surface roughness of building materials. TLS data were classified using DT, Discriminant Analysis (DA), Naive Bayes (NB), SVM, and KNN. A laser scan database for ten common construction building materials (stone, pottery, mortar, concrete, ceramic, wood, plastic, plaster, metal, and painting) was used to train and validate the model. However, better results were achieved from one-class SVM (OC-SVM) and SVDD. Fernando and Marshall [13] performed a state of the art classification methodology for rock and gravel, by identifying force data (proprioceptive force data) acquired from load-haul-dump equipment with capacity of 14-tonne and adopting ANN, KNN, and k-mean algorithms. However, good results were obtained by ANN (5-NN) model with more realistic classification. Since the system only relies on proprioceptive sensing, it is feasible in harsh, dusty, and dark conditions that hinder the use of external sensor data. Table 4 exemplifies the technical summary of collected studies for their main features, achieved outcomes, and observed limitation if any.

RefYearData inputAdopted techniquesMaterials classified
[18]2010Site ImagesANN, SVDD, & C-SVCConcrete
[19]2010Spectral InformationFSVM with a PUK kernelGranite
[20]2012Site ImagesGMM, ANN, & SVMConcrete
[14]2014Concrete ImagesANN, SVM, KNN, Bayesian, & FLDConcrete
[9]2014Material ImagesC-SVM (SVM)Grass, Form Work, Marble, Gravel, Foliage, Soil-Loose, Soil-Compact, Paving, Soil-Vegetation, Stone-Granular, Soil-Mulch, Wood, Stone-Limestone, Brick, Cement-Smooth, Cement-Granular, Asphalt, Concrete-Precast, Concrete-Cast, & Metal-Grills
[3]2015Material ImagesC-SVM (SVM)Brick, Asphalt, Concrete, Foliage, Granular & Smooth Cement based surfaces, Gravel, Formwork, Marble, Insulation, Paving, Metal, Soil, Waterproofing Paint, Stone, & Wood
[5]2016Site ImagesMLP, ANN, & RBFConcrete, OSB Boards, & Red Brick
[17]20163D Point Cloud/Material ImagesSVMWindows, Walls, Protrusions, Tile, Brick, Stone, & Coating
[16]2017Site ImagesCDF, SVM, RBF, LBP, & PPHTInsulation, Studs, Outlets, Electrical & Three States for Drywall Sheets (Installed, Painted, & Plastered)
[21]2019Site ImagesCNN (ANN)Structural Elements
[22]2019Rebar ImagesRFSP, OTSUHT, & IVBRebar
[23]2020Material ImagesANNSandstorms, Paving, Gravel, Stone, Cement-Granular, Brick, Soil, Wood, Asphalt, Clay Hollow Block, & Concrete Block
[8]2020Laser Scan Data/Material ImagesDT, DA, NB, SVM, & KNNMortar, Concrete, Metal, Stone, Wood, Painting, Plastic, Plaster, Ceramic, & Pottery
[13]2020Proprioceptive Force DataANN, KNN, & k-meanRock and gravel

Table 3.

Summary of material classification studies for construction progress monitoring of building projects.

RefAdopted techniqueMain featuresAchieved outcomesRemarks/observed limitations
[18]ANN, SVDD, & C-SVCThe model was trained by datasets of negative & positive concrete images.ANN technique was found better with an average precision of 83.3%, average recall of 79.6%, & overall concrete detection of 80%.The accuracy of the model outcome was evaluated by manually tracing the concrete image pixels.
[19]Functional linear regression, & FSVM with a PUK kernelGranite varieties were characterised using a spectrophotometer (vectorial spectral information).FSVM with a PUK kernel was found better with 0.82% validation error rate.For assessing the total real colour of the stone, the data was to be collected from various points on the sample.
[20]GMM, ANN, & SVM108 images were collected for 50 construction projects on different timings & weather conditions.The SVM model with the HSI colour space was found better with an accuracy rate of 91.68%.The detection performance was evaluated by identifying concrete & background pixels.
[14]ANN, SVM, KNN, Bayesian, & FLDA dataset contained 31 images of concrete with the image resolution of 2 mm.ANN was the better choice for automatic image segmentation with correctly classified pixels up to 90.29%.This study covered the segmentation of concrete images.
[9]C-SVM (SVM)Various pixels sizes were tested, i.e., 30 × 30, 50 × 50, & 200 × 200. Material datasets were created for varying degrees of viewpoint, illumination, and scales.97.1% average classification rate was achieved for 200 × 200 pixel images.This study did not cover the challenge of segmentation.
[3]C-SVM (SVM)This study adopted BIM integrated daily construction photologs for extraction relevant image patches. Study adopted materials library for images with three different patch sizes of 50 × 50, 75 × 75, & 100 × 100.The accuracy of 92.4% was achieved for the dataset with 100 × 100 pixel size images.Practical limitations for adopted methodology include comprehensiveness of materials library, completeness of 3D reconstruction, and computation time.
[5]MLP, ANN, & RBFFor feature extraction, the construction materials were divided into three groups: 1) materials encompasses a very distinct colour, 2) variable colour patterns, & 3) material without a distinctive colour pattern.SVM classifier with RBF kernel was found better than other techniques.Outcomes may be affected due to lack of adequate light, varying viewpoint angle, & image capturing distance.
[17]SVMThe study performed the material detection method & image-based 3D modelling. The 3D model was generated via SfM and segmented into planar components, which were recognised as structural components via knowledge-based reasoning.RANSAC was adopted for detection in the point cloud. The model achieved an average accuracy of 95.55%.The datasets consist of 463 stone samples, 637 brick samples, 504 tile samples, & 409 coating samples.
[16]CDF, SVM, RBF, LBP, & PPHTInput source data was tested for UAV, smartphones & internet sources. A CV technique was adopted for the detection of interior components & extrapolated the existing scenario via 2D digital images.Three datasets were adopted, and the following outcomes were attained: 1) for stud module, precision above 90%, & recall above 83%. 2) for insulation module, precision above 88%, & recall above 89%. 3) for electric outlet module, precision above 86%, & recall above 87%.Practical limitations for adopted methodology include the inability to detect & metallic electrical boxes, partitions with low visibility, & improperly captured scenes.
[21]CNN (ANN)The study follows an automated construction materials’ labelling process of construction site images. The model combines the available information from the photogrammetric model & the 4D BIM. By aligning the BIM & 3D point cloud, a digital components can be projected onto the image.91% pixel-wise accuracy was validated by the sample.The construction & model inaccuracies, errors in post estimation during SfM, large scale deviations for real world coordinates, & occlusions may cause labelling errors.
[22]RFSP, OTSUHT, & IVBThe model performs analysis on the reinforcing bars of the production plant moving along a conveyor belt by accurately calculating the bar area and its number.RFSP+IVB gave better results with 0.89 as F-score.
[23]ANNThe method uses the DL model by data augmentation & prevents over-fitting of network structures for images with varying camera resolution, illumination, & small datasets.VGG16 algorithm gave the maximum accuracy of 97.35%.Raspberry Pi 3 was used with datasets taken from different construction sites with 1231 images of 11 classes for various views of materials.
[8]DT, DA, NB, SVM, & KNNThe TLS based laser scan data can provide information for building material for surface geometries such as surface roughness and material reflectance.OC-SVM and SVDD gave better results with an average classification accuracy of 96.7%.In this study, only plane target surfaces were considered.
[13]ANN, KNN, & k-meanThe study followed a material classification methodology using proprioceptive force data acquired from an digging machine integrated with ML. The model is pertinent in the dusty, dark and harsh areas.ANN model was found effective with a classification accuracy of 90%.The model only covers rock & gravel, where excavated soil may consist of various other materials.

Table 4.

Technical summary of collected articles for progress monitoring via ML of the building construction projects.

The general workflow of ML material classification model, via images, is shown in Figure 2. In the training process, the model is usually trained with the help of datasets of construction material images. The segmentation process is performed on the collected images, as segmentation increases the performance output of the model as compared to non-segmented images [24]. GMM technique can also be utilised for image segmentation. The feature extraction is performed on the segmented images for their colour, texture, compactness, contrast. Researchers mostly have adopted HSV, RGB (red, green and blue), Image patch (IP), 48-dimensional form (LM), and 18-dimensional rotationally invariant form (rLM) for features extraction and found better results with LM + HSV combination [9]. Depending upon the type of model, clustering process can be adopted, and on this data any ML classifier can be applied for model training. While testing the model for any random material image, the segmentation and feature extraction processes are performed, which are further identified with the help trained ML model for final material recognition output.

Figure 2.

General workflow of ML-material classification models.


4. Python-based ML-material classification model

In this section of the chapter, a small exercise has been performed to detect construction materials via ML algorithm for the preview of the readers to the practical application of ML in material classifications for construction progress monitoring. The exercise has been executed by using the resources available as open-source on the internet. The simple ANN model has been adopted, developed by Adrian Rosebrock [25] using Python under Keras and TensorFlow environment [26]. Moreover, the construction materials datasets for the images have been collected from the open-source GitHub repository for concrete blocks, asphalt, wood, and bricks [27]. The selected model performed the classification of construction materials in two phases. In the first phase, the model has been trained with the help of collected datasets, and in the same phase, validation of the model has been performed. The adopted ANN-based model splits 75% data for training purpose and the remaining 25% data for validation/testing. In the training phase of the model, the images of each construction item have been placed in the separate folders labelled as the name or ID of that particular construction material. The model analyses the colour, texture and geometrical aspects of images under each construction material ID and trains its memory for each construction material separately against the given name or ID. The same model validates or tests itself on the assigned images, to verify the model effectiveness for the predictions. If the model fails to give an effective validation run, either model needs to be trained more or model structure needs to be reviewed. In the second phase, the model has been applied to predict and identify construction materials on randomly selected images from the internet. The predictive model uses the memory of the trained model to predict construction item on the input image. Moreover, for improving the training model, the number of ANN hidden layers and epochs per run can be increased in the model, which enhances the output results for accuracy and decreases the data loss [28]. Therefore, to observe the effects of varying epochs per run on the predictive model, the performance of the model has been tested on two different scenarios of epochs per run, i.e., 150 and 300.

4.1 Model training and testing

The training of the ANN model was performed on the images datasets of each construction material, i.e., concrete blocks, asphalt, wood, and bricks, with each dataset comprised of 50 images. As the model has been designed to train on 75% of the provided data; therefore, for each construction item, the model was trained on 38 images. The remaining 12 images were used by the model for validation or testing. Two different models were trained on varying epochs per run, i.e., 150 and 300. The first model, with 150 epochs per run, attained the accuracy of 56% as shown in Figure 3, where its graphical representation for the attainment of model accuracy and loss can be seen in Figure 4. It can be seen that ‘Asphalt’ in this model got ‘0’ (zero) value for precision and F1 score. The maximum precision and F1 score have been attained for wood (precision = 0.75, F1 score = 0.80). Whereas, the lowest precision and F1 score have been achieved by concrete block (precision = 0.35, F1 score = 0.56).

Figure 3.

Training/testing model output for 150 epochs per run.

Figure 4.

Graphical representation of model accuracy and loss for 150 epochs per run.

In the second model, the epochs per run for the model was set to 300, and 64% model accuracy was attained. The attained model accuracy, data loss along with precision, recall and F1 score can be seen in Figure 5, where the graphical representation of the training data for the attainment of model accuracy and loss for 300 epochs can be seen in Figure 6. The accuracy and F1 score of the second model, with the increased epochs, was better than the first model, which validates the enhanced performance of the ANN on increasing the epochs per run.

Figure 5.

Training/testing model output for 300 epochs per run.

Figure 6.

Graphical representation of model accuracy and loss for 300 epochs per run.

It can be seen that maximum precision and F1 score have been attained for wood (precision = 0.93, F1 score = 0.93), and brick (precision = 0.88, F1 score = 0.78. Whereas, the lowest precision has been achieved by concrete block (precision = 0.35) and F1 score by asphalt (F1 score = 0.42).

4.2 Predictive model

The output of the predictive model is much dependent on the training of the model. For the testing of the predictive model, random images were collected from the internet for each. i.e., concrete block, asphalt, wood and brick. The selected predictive model utilised the memory output of the trained model and predicted the material along with its probability. The model predictions were observed for both trained models, i.e., 150 epochs and 300 epochs. Table 5 shows the summary and comparison of the predictive model against the input images for both trained models (150 epochs and 300 epochs) along with the model prediction probability percentage.

Table 5.

Summary of predictive model.

It can be illustrated from the results that ANN model performance has been enhanced and increased by increasing epochs per runs. Model predictions for the construction materials against the training model with 150 epochs were mostly inaccurate, which can be seen in ‘Brick’ and ‘Asphalt’. Whereas, for the same input images, the model predictions were accurate and reliable with more prediction probability against the trained model with 300 epochs. Thus, the outcome of the predictive model is dependent on the structure of the trained ANN model.


5. Conclusions

The emergence of automation and IoT, in the construction sector, have regained the interest of professional and research community towards ML techniques. The ML technologies are now being adopted in many construction processes and one of which is construction progress monitoring. The theme of this chapter was designed to overview the application of material classification via ML techniques and their implication in the construction automated progress monitoring. For the achievement of this study objective, a small structured review was performed to collect relevant studies from WoS and Scopus. Overall, 14 studies were found relevant, where the majority of studies were performed for the multi-classification of construction materials using digital images. Moreover, the classification of material has also been performed based on proprioceptive force data. ANN and SVM models have been found most effective ML techniques for classification, and these techniques have also been integrated with BIM for effective construction processes control. For the better understanding of the readers to the practical implementation of ML techniques, a small experiment for multi-classification of construction materials (brick, asphalt, concrete block, and wood) using Python has also been included in this chapter. A simple ANN-based model was trained on the dataset of the aforementioned construction materials, and predictions were performed on random images. The utilised resources were collected from the open-source repositories, and details of their parent websites have also been shared for the readers’ interests.

In this era of automation, the construction sector is inclined towards the adoption of artificial intelligence, and ML is playing a vital role in the enhancement of construction processes in various ways. Likewise, material classification is one of the favourite techniques when it comes to ML, especially for project progress monitoring. Although various studies have been conducted, however, still, there is a need for dedicated research to improve algorithms and methodologies to make these construction monitoring processes more effective and feasible for construction stakeholders.



The resources and program codes used in this chapter for the Python-based material classification model are available as open-source, and their access links are provided for the readers.



RANSACRandom sample consensus
ARIASAutomatic reinforcing-bar image analysis system


  1. 1. Pazhoohesh M, Zhang C. Automated construction progress monitoring using thermal images and wireless sensor networks. GEN. 2015;101:01
  2. 2. Lin Z, Petzold F, Ma Z. A Real-Time 4D Augmented Reality System for Modular Construction Progress Monitoring. In: ISARC. Proceedings of the International Symposium on Automation and Robotics in Construction. vol. 36. IAARC Publications; 2019. p. 743–748
  3. 3. Han KK, Golparvar-Fard M. Appearance-based material classification for monitoring of operation-level construction progress using 4D BIM and site photologs. Automation in Construction. 2015;53:44–57. Available from:
  4. 4. Li Y, Liu C. Applications of multirotor drone technologies in construction management. International Journal of Construction Management. 2019;19(5):401–412
  5. 5. Rashidi A, Sigari MH, Maghiar M, Citrin D. An analogy between various machine-learning techniques for detecting construction materials in digital images. KSCE Journal of Civil Engineering. 2016;20(4):1178–1188
  6. 6. Meroño JE, Perea AJ, Aguilera MJ, Laguna AM. Recognition of materials and damage on historical buildings using digital image classification. South African Journal of Science. 2015
  7. 7. Lu Q, Lee S, Chen L. Image-driven fuzzy-based system to construct as-is IFC BIM objects. Automation in Construction. 2018
  8. 8. Yuan L, Guo J, Wang Q. Automatic classification of common building materials from 3D terrestrial laser scan data. Automation in Construction. 2020;110(October 2019):103017. Available from:
  9. 9. Dimitrov A, Golparvar-Fard M. Vision-based material recognition for automated monitoring of construction progress and generating building information modeling from unordered site image collections. Advanced Engineering Informatics. 2014;28(1):37–49
  10. 10. Hadidi R, Cao J, Woodward M, Ryoo MS, Kim H. Real-time image recognition using collaborative iot devices. In: Proceedings of the 1st on Reproducible Quality-Efficient Systems Tournament on Co-designing Pareto-efficient Deep Learning; 2018. p. 1
  11. 11. Jameel SM, Hashmani MA, Rehman M, Budiman A. An adaptive deep learning framework for dynamic image classification in the internet of things environment. Sensors (Switzerland). 2020;20(20):1–25
  12. 12. Bhaddurgatte RC, Vijaya Kumar BP, Kusuma SM. Machine learning and prediction-based resource management in IoT considering Qos. International Journal of Recent Technology and Engineering. 2019;8(2):687–694
  13. 13. Fernando H, Marshall J. What lies beneath: Material classification for autonomous excavators using proprioceptive force sensing and machine learning. Automation in Construction. 2020;119(June):103374. Available from:
  14. 14. Yazdi M, Sarafrazi K. Automated segmentation of concrete images into microstructures: A comparative study. Computers and Concrete. 2014;14(3):315–325
  15. 15. Penumuru DP, Muthuswamy S, Karumbu P. Identification and classification of materials using machine vision and machine learning in the context of industry 4.0. Journal of Intelligent Manufacturing. 2020;31(5):1229–1241. Available from:
  16. 16. Hamledari H, McCabe B, Davari S. Automated computer vision-based detection of components of under-construction indoor partitions. Automation in Construction. 2017;74:78–94
  17. 17. Yang J, Shi ZK, Wu ZY. Towards automatic generation of as-built BIM: 3D building facade modeling and material recognition from images. International Journal of Automation and Computing. 2016;13(4):338–349
  18. 18. Zhu Z, Brilakis I. Parameter optimization for automated concrete detection in image data. Automation in Construction. 2010;19(7):944–953. Available from: 1016/j.autcon.2010.06.008
  19. 19. Araújo M, Martínez J, Ordóñez C, Vilán JAV. Identification of granite varieties from colour spectrum data. Sensors. 2010;10(9):8572–8584
  20. 20. Son H, Kim C, Kim C. Automated Color Model–Based Concrete Detection in Construction-Site Images by Using Machine Learning Algorithms. Journal of Computing in Civil Engineering. 2012;26(3):421–433
  21. 21. Braun A, Borrmann A. Combining inverse photogrammetry and BIM for automated labeling of construction site images for machine learning. Automation in Construction. 2019;106(June):102879. Available from:
  22. 22. Lee JH, Park SO. Machine learning-based automatic reinforcing bar image analysis system in the internet of things. Multimedia Tools and Applications. 2019;78(3):3171–3180
  23. 23. Ghassemi N, Mahami H, Darbandi MT, Shoeibi A, Hussain S, Nasirzadeh F, et al. Material Recognition for Automated Progress Monitoring using Deep Learning Methods. Journal of Advanced Engineering Informatics. 2020:1–30. Available from:
  24. 24. Siddula M, Dai F, Ye Y, Fan J. Classifying construction site photos for roof detection. Construction Innovation. 2016;16(3):368–389
  25. 25. jrosebr1 (Adrian Rosebrock);. Available from:
  26. 26. Keras Tutorial: How to get started with Keras, Deep Learning, and Python - PyImageSearch;. Available from:
  27. 27. ralizadehsani/material_recognition;. Available from:{_}recognition
  28. 28. Salah Alaloul W, Hannan Qureshi A. Data Processing Using Artificial Neural Networks. Dynamic Data Assimilation - Beating the Uncertainties [Working Title]. 2020 may. Available from:

Written By

Wesam Salah Alaloul and Abdul Hannan Qureshi

Submitted: October 7th, 2020 Reviewed: February 2nd, 2021 Published: March 9th, 2021