Developing a Bridge Condition Rating Model Based on Limited Number of Data Sets

This chapter utilizes artificial neural network (ANN) and multiple regression analysis (MRA) to model bridge condition rating based on limited number of data sets. Since data sets are very limited and there is a gap in range of rating scale, two conditions of data sets are used in this study, namely complete data sets and data set with bridge component condition rating data are missing. Five methods are then used to handle the missing bridge component condition rating data. Three commonly used methods and two new methods are explored in this study. It seems that the performance of the model using data sets after handling missing bridge component data to fill the gaps in the range scales of the bridge condition rating improved the performance of the model. In addition, a han - dling method that substitutes missing data of bridge component ratings with available bridge rating data is favorable. Based on the values of root mean square error (RMSE) and R 2 , the ANN models perform slightly better than MRA to map relationship between bridge components and bridge condition rating. This concluded that ANN is suitable to model bridge condition rating compare to MRA method.


Introduction
The bridge condition rating is the most important part of the bridge management system (BMS) because, historically, bridge condition rating data were found to affect approximately 60% of the BMS analysis modules [1]. For BMS requirements, bridge inspectors generally use visual inspection as the first step toward condition assessment procedure unless a structure cannot be visually assessed. In through visual inspection, bridge inspectors evaluate the condition of a bridge using their personal experience and following guidelines such as those found in inspector's manuals. The visual inspection incorporates many parameters and human judgments that may produce slightly uncertain and imprecise results [2]. It would be ideal to conduct physical structural tests on each bridge component, but it would be impractical and economically prohibitive to implement them, given the large number of bridges to be inspected within a given period. As an alternative, the collective judgment of the inspectors can be used to develop unified, coherent bridge inspection procedures [3]. Thus, a mathematical method such as an artificial neural network (ANN) and multiple regression analysis (MRA) can be useful to handle this uncertainty, imprecision, and subjective judgment.
The limited inspection reports maintained by the Public Works Department (PWD) of Malaysia are used as the initial data, which contain valuable information about the condition ratings of the bridge components and whole bridges. The condition rating is a numerical system, where a number from one to five is assigned to each component of the structure based upon observed material defects and the resulting effect on the ability of the component to perform its function in the structural system, as described in Ref. [4]. Table 1 shows the condition rating system used by PWD Malaysia. The bridge components that are inspected and rated contribute to the overall bridge rating, as shown in Table 1. The bridge components involved are the beam/girder, deck slab, pier, abutment, bearing, drainpipe, parapet surfacing, expansion joint, and slope protection.
The original data sets are very limited and many shortcomings exist in the records such as unavailable or missing bridge component condition rating data, the whole bridge condition rating is not distributed in complete range of rating scale and many outlier data sets are exist. The outlier data are where the whole bridge rating value is larger than the maximum component rating data. The missing data problem requires a method to improve the accuracy and efficiency of the modeling of the condition rating of a bridge that utilizes a mathematical model such as MRA and ANN. In terms of BMSs, inappropriate treatment of missing data affects the performance of the bridge condition rating model and the bridge deterioration predictive model. The common methods used by previous researchers to deal with the missing data include either deleting the missing features or replacing the missing data with zero or with the mean data of the training set [5,6]. However, as there is a certain level of uncertainty associated with a particular case, such as the bridge condition rating data, the above method must be Rating scale General definition 1 No damage found and no maintenance required as a result of inspection 2 Damage detected, and it is necessary to record the condition for observation purposes 3 Damage detected is slightly critical, and thus it is necessary to implement routine maintenance work 4 Damage detected is critical, and thus it is necessary to implement repair work or to carry out detailed inspection to determine whether any rehabilitation works are required or not 5 Being heavily and critically damaged, and possibly affecting the safety or traffic, it is necessary to implement emergency temporary repair work immediately, rehabilitation work without delay after the provision of a load limitation traffic sign, or replacement work Table 1. The condition rating system based on severity of defect. quantified in certain ways during the data-mining process. Moreover, the simple methods mentioned earlier are often not suitable for improvement of the quality of the data.
The performance of the mathematical model of bridge condition rating is also dependent on the quantity and the quality of data set used in constructing relationship between bridge component and whole bridge condition rating. Thus, the purpose of this study is to develop bridge condition rating model based on limited available data and different rates of missing bridge component condition rating data. MRA and ANN are utilized to map relationship between bridge component ratings and whole bridge condition rating. Four conditions of data sets are used: complete data sets (M0), data sets with one component missing (M1), data sets with one or two components missing (M2), and data sets with one, two, or three components missing (M3). Five methods are then used to handle the missing bridge component condition rating data. This study explores three commonly used methods and two new ones. The best method is then applied to substitute the missing data of bridge component condition ratings so that the bridge condition rating model can be developed from various conditions of data sets. Furthermore, the handling of the missing data also increases the size of data sets and provides a more complete range for the bridge condition rating distribution.

Previous works
Condition rating data have the potential to provide tremendous value to the bridge management. The condition rating data can be used to help prioritize maintenance work and decide on allocation of available budgets based on engineering and financial considerations [7]. Hence, the appropriate procedure is needed in bridge condition ratings data gathering for managing bridges under constrained resources.
ANNs are widely used as an attractive alternative to handle complex and non-linear systems that are difficult to model using conventional modeling techniques such as MRA. ANNs have been widely applied in engineering, science, medicine, economics, and environmental applications. The most common applications are function approximation, pattern classification, clustering, and forecasting [1,[8][9][10]. Various forms of ANNs (i.e., feed-forward neural networks: FENN, recurrent networks, radial basis functions, wavelet neural networks, Hopfield networks, etc.) have been applied in various disciplines. However, in the context of function approximations, such as bridge condition rating, the FFNN is generally chosen as the network architecture [9] and back propagation (BP) as the learning algorithm [10][11][12].
Chen [10] categorized and evaluated a beam bridge condition into four main bridge components, namely, substructure, bearing, beam, and accessory structure. These four components are evaluated based on 20 assessment criteria that can be inspected by close visual inspections according to Chinese Bridge Maintain Codes. Five neural network models are then developed to model substructure, bearing, beam, accessory structure, and whole bridge status. The input parameters for substructure, bearing, beam, accessory structure, and whole bridge status are 7, 2, 6, 5 and 4, respectively. He concluded that the proposed approach improves the efficiency of bridge state assessment.
Li et al. [13] utilized ANN to evaluate a bridge conditions based on substructure, superstructure, deck, and channel conditions. In their proposed model, the training cases converged very well, but for the test cases, the prediction from the network is consistent with the target in about 60%. They concluded that the low prediction accuracy affected by the data used in training the network is not sufficient for the network to generate proper weights to precisely model the inputoutput relationship. Another reason was inconsistency in the evaluation results due to subjective factors observed in the inspection data, which are used to train and test the neural network.
In most countries, there exists a large time gap between the dates of the construction of the bridge and the adoption and implementation of the relevant BMS [14]. There is a general leaking in such BMSs' database such as inconsistency of data sets and much bridge component condition rating is unrecorded. Another example of how bridge condition rating data can go missing is the difficulty in obtaining information and expensive testing of some bridge components. The missing data constitute the largest fraction of the difficulties in analyzing the data, making constructing predictive ratings, and other decision-making processes that depend on these data. Furthermore, it is impossible to build a convictive classification model with missing data because the missing data affect the integrity of the dataset [6]. Zhimin et al. [6] used five methods to handle missing data in their classification problem. The methods are as follows: deleting missing data, replacing missing data with zero, replacing missing data with the mean value of all the data of the training set, replacing missing data with the mean value from the same label data of the training set and predicting the missing value using a feed-forward backpropagation ANN. Markey et al. [5] compared three methods for estimating missing data in the evaluation of ANN models for their approximation problems. These methods are as follows: simply replacing the missing data with zero or the mean value from the training set or using a multiple imputation procedure to handle the missing value.

Methodology
In this study, the bridge condition ratings data were provided by the Bridge Unit, Roads Branch, PWD Malaysia. The ratings based on the Annual Bridge Inspection Manual [4] by the PWD classified the state of a bridge and bridge component conditions into five numerical systems on a rating scale of 1-5, from "no damage" (rating 1) to "heavily and critically damaged" (rating 5). The bridge components are classified as either primary or secondary elements. The primary components include the surface, deck slab, beam/girder, piers, and abutment, whereas the secondary components include the parapets, expansion joints, bearings, slope protection, and drainpipes. The bridge condition rating can be evaluated by processing the ratings and important sets of the bridge components [15]. Suksuwan [16] evaluated an overall bridge condition rating based on the condition of the superstructure and substructure. The superstructure consists of two components, namely the bridge deck and accessories. Meanwhile, the substructure is divided into three components: pier, abutment, and foundation. The relationship between the bridge condition rating and its components can be drawn as Y = f {X 1 , X 2 , …, X p }, where X 1 , X 2 , …, X p are p bridge component condition rating variables and Y represents the bridge condition rating.
Since the availability of data sets with complete condition data of bridge component were very limited, utilizing incomplete sets by handling missing value with the appropriate treatment is expected to improve the performance of the model. Furthermore, upon observation of the available data sets, it was found that there is a big gap in the bridge condition rating distribution, where there were no complete data sets available for a bridge condition with a rating of 4 and only four pairs of complete sets of bridge conditions with a rating of 5 available. Consequently, finding the best method to handle this problem is an important step prior to constructing the model of the bridge condition ratings.

Data preparation
The original data sets include 1244 data sets from the last 4 years of inspection records of 311 single-span concrete bridges. Among these data sets, only 579 sets have complete rating data for components and the whole bridge condition. From these 579 data sets, a large number of the bridge condition rating data sets have repeated data. When constructing the model of bridge condition ratings through an ANN and MRA, they provide nothing new as the information is redundant. To avoid this redundancy, only one of the data sets with the same data is retained and the others are deleted. Furthermore, these available data sets are also adjusted to remove the outlier data sets where the bridge condition rating value cannot be larger than the maximum component rating data. After deleting the redundant and outlier data sets, there are 157 data sets left with almost all of them having a rating of 1, 2, or 3. However, bridges with a condition rating of 4 do not have complete component rating data, and only three pairs with a rating of 5 are available, as shown in Figure 1.
As explained in the previous section, all data sets for bridges with a rating of 4 are incomplete. In this study, the data sets that have 1-3 components with missing value are considered and handled with different methods to provide more data and fill the gap in the bridge rating scale distribution. The number of data sets for M0, M1, M2, and M3 are 157, 226, 252, and 267, respectively.   Table 2 illustrates data sets that contain complete data and missing data of bridge component condition ratings. Five methods are proposed to handle these missing data of bridge component ratings. These five methods are as follows: substituting with the local mean (SM), substituting with the local minimum (SMN), substituting with the local mode (SMD), substituting with the local mean value of the same component class (SMC), and substituting with the available bridge condition rating value (SBR) from the same label of data set.
In SM method, the mean value of the data label n (xn) is calculated using Eq. (1). The missing value is then substituted with the xn value. Furthermore, the local minimum value (xmin) is defined as the smallest value appearing in the same label of data set. Meanwhile, the local mode value (xmode) is defined as the value that appears most often in the same label of data set.
Meanwhile, in the substitution with SMC method, the missing value is substituted based on the criteria of the bridge components. If the primary component rating data are missing, the data will be substituted by the average value of other available primary component ratings. The same method is also applied to unavailable data of secondary components. By substitution with the SBR method, the missing component rating value is simply substituted with the available bridge condition rating value from the same label of data set.
After the missing data are handled with the above methods, the data are then checked to remove the redundant data that appear in the list. Furthermore, the remaining data sets show that there is no significance different in the number of data sets (M2 and M3) in comparison to M1, hence only data sets M0 and M1 are chosen in study. The distribution of data sets after removing the redundant data is shown in Figure 2.

Multiple regression analysis model
In this case, MRA deals with one output parameter (dependent variable), which is the bridge condition rating value and nine input parameters (independent variables) which are the bridge components condition rating. If the bridge condition rating is y and bridge components condition rating is x 1 , x 2 , x 3 , …, x n , then the model is given by:

No. of data Bridge component (X)
Bridge rating (Y) Since the data sets used in this modeling problem are more than one, then Eq. (2) can be written as follow: where β is the coefficient of bridge components condition rating, n the number of bridge component considered, i = 1, 2, 3, …, N is the number of data sets, and ε represents the error of the model.

Artificial neural network model
Five conditions of data sets are then trained with a choice of the network architecture, namely complete data sets (M0), which number 157 data sets in total; data sets with one component missing (M1), which number 226 data sets in total; data sets with one or two components missing (M2), which number 252 data sets in total; and data sets with one, two, or three components missing (M3), which number 267 data sets in total.
A feed-forward neural network with a single hidden layer that varies the number of hidden neurons in the range from 1 to 28 neurons and one output layer, as shown in Figure 3, was selected to train the data sets. The networks are trained with a variation of the back-propagation training algorithm, namely the Levenberg-Marquardt algorithm (trainlm). The Trainlm algorithm is utilized as it has the fastest convergence in function approximation problems such as the bridge condition rating problem [17]. Bridge component condition rating data are used as input variables, which consist of nine inputs (surface, expansion joint, parapet, drainage, slopes protection, abutment, bearing, deck/slab, and beam/girder), and the whole bridge condition rating is used as the output variable Y.  During the training process, there is a possible risk of overfitting or overtraining the network. In this situation, the error on the training set is driven to a very small value, but when new data are presented to the network, the error becomes large. The network has memorized the training examples, but it has not learned to generalize to new situations [17,18]. Therefore, in this work, the early stopping technique was used to monitor the training process to handle the over-training problem. In the early stopping technique, there is a need to divide the data set into three subsets: training, validation, and testing data sets. The training set is used to train the network, and the validation data set is required to validate the network according to the early stopping technique. The testing data sets are used to test the performance of the trained network. In this study, 60, 20, and 20% were used as the training, validation, and testing data sets, respectively.
Prior to training with the data sets, the network inputs and targets are normalized using the functions [pn, ps] = mapstd(p) and [tn, ts] = mapstd(t) in MATLAB software so that they had mean of 0 and standard deviation of 1. The original network inputs and targets are given in the matrices p and t. The normalized inputs target pn and tn that are returned to have mean of 0 and standard deviation of 1. The settings structures ps and ts contain the means and standard deviations, respectively, of the original inputs and original targets. After the network has been trained, these settings are then used to transform any future inputs that are applied to the network. To convert these outputs back into the same units that were used for the original targets requires ts. The following functions simulate the network that was trained on the previous functions and then converts the network output back into the original units an = sim(net, pn) and a = mapstd("reverse," an, ts) [19].
A transfer function is used to produce the neuron output and limit the amplitude of the output of the neuron. It determines the relationship between the inputs and outputs of a neuron and a network [17]. In this study, the tangent sigmoid transfer function (tansig) and linear transfer function (purelin) are used in the hidden and output layer, respectively. The tansig function, as given in Eq. (4), produces outputs in the range of −1 to +1, and the purelin function, as given in Eq. (5), produces outputs in the range of −∞ to +∞.
The root mean squared error (RMSE) of the training set is used to measure the performance of the network, where the typical performance function that is used for training ANNs is the mean sum of the squares of the network errors. The coefficient of determination (R 2 ) of the linear regression line between the ANN outputs and the bridge condition rating targets is also used to measure the response of the trained network. The error of data label k (e k ) and RMSE of all the training sets are calculated using Eqs. (6) and (7).
Here, t k is the target of data label k, a k is the network output of data label k, and Q is the number of data sets used in the network training process.
The number of epochs for all the training algorithms is fixed at 1000. At the same time, the initial values of the weights and biases are always initiated from random values; therefore, each run might produce different output values [20]. Therefore, each ANN is made to run 30 times, and the average values for RMSE and R 2 are reported.
The process of training using the incomplete data is virtually the same as that for the training data with none of the features missing. Once the substitution has been made, the remaining steps of the training algorithm ensure the use of the handled features in the process of updating the weight and bias of the network, while the missing features are automatically handled based on the proposed solution. The purpose of the training process is to map the relationship between the input and output parameters using the ANN, as given in Eq. (8).
Here, x is the input vector, y is the output vector, w is the weight matrix for the connections between the input and hidden layer, v is the weight matrix for the connections between the hidden and output layer, b 1 is the bias in the hidden layer, b 2 is the bias in the output layer, t 1 is the transfer function for the neurons in the hidden layer, and t 2 is the transfer function for the neurons in the output layer [21]. Table 3 presents the results of MRA models for data set with complete bridge components condition rating data and data set after substituting missing data with the proposed methods.

Models MRA
The results show that the performance of a network trained with the entire range scale of data sets (M1) seems better than the network trained with missing range scale ratings as in data sets M0, as indicated by the R 2 values. For data sets M1, the handling method with SMD, SMC, and SBR yields almost similar R 2 values that are better in comparison to SM and SMN methods. Furthermore, in term of RMSE value, SBR method shows better performance in comparison to other methods. Referring to RMSE and R 2 value of training, validation, and training data sets, the performance of SBR method appears to yield slightly better results than other methods.
The typical MRA models linking bridge condition rating to its explanatory components for data sets M1-SBR are presented in Table 4. The significance of each coefficient was determined by t-value and P-value. The F-ratio presented in Table 3 measures the probability of chance departure from a straight line. On review of the output found in Table 3 shows that the overall model was found to be statistically significant as (F = 53

Models ANN
The training process is evaluated based on plotting the training, validation, and Mean Squared Errors (MSEs) versus number of epochs. Figure 4 shows the training progress of the selected ANN for the data sets M0. The network seems unstable, as indicated in Figure 4, where the characteristics of the validation and test error are not similar, which may indicate that there is a gap in the data set distribution or a poor division of the data set [17].  Figure 5 shows the typical training progress of data sets where the missing value is substituted with the bridge condition rating value (SBR). Figure 5 shows the decrease of the MSE versus the number of epochs during the training process of the ANN. Indeed, Figure 5 shows that the validation and testing set errors show similar characteristics, which provide reasonable evidence of network training, and it does not appear that any significant overfitting has occurred. If the error in the test set reaches a minimum at a significantly different epoch number than the validation set error, it may indicate a poor division of the data set [17].  Regression analysis between the bridge condition rating predicted by the ANN model and the corresponding bridge condition rating target is performed using the routine postreg using MATLAB software. The format of this routine is [m,b,r] = postreg (a,t), where m and b correspond to the slope and the intercept, respectively, of the best linear regression that relates the targets to the ANN outputs. If the fit is perfect, the ANN outputs are exactly equal to the bridge condition rating targets, and the slope is 1 and the intercept with the Y-axis is 0. The third variable, r, is the correlation coefficient between the ANN outputs and targets. It is a measure of how well the variation in the predicted bridge condition rating is explained by the target. If r is equal to 1, then there is perfect correlation between the targets and ANN outputs [22].
The performance of ANN models for complete data sets (M0) and data sets (M1) after the missing data are handled by the above-mentioned methods are presented in The linear regression analysis of all data sets is then also performed to evaluate the developed models' response on all the data sets. Here, all the data sets, namely training, validation, and test sets, are introduced through the models, and a linear regression between the model outputs and the corresponding targets is performed. Table 6 shows the R 2 values of all data sets for all handling methods.
The result shows that the MRA model and ANN model trained with the missing data substituted by the bridge condition rating value (SBR) has a higher accuracy of prediction in comparison to other methods, as shown in Table 6. The SBR method yields R 2 values of 0.9100 and 0.9328 for all data sets by MRA and ANN methods, respectively. This indicates that, for the bridge condition rating model with these data conditions, the SBR method is a more reasonable method for handling the missing value prior to training the network. The performance comparison of ANN models and MRA models is also made in terms of R 2 values of predicted value versus target value as shown in Table 6. It can be seen that all ANN models provide better agreement with the target of bridge condition rating.

Conclusions
Bridge condition rating models are developed based on very limited inspection data records using MRA and ANN techniques. Since the data sets are limited, utilizing all the available data sets by handling missing data with the appropriate methods is expected to improve the performance of the models. The method, where the missing value is substituted with the bridge condition rating value, performs better than the other methods. This method is able to determine the bridge condition rating with R 2 values of 0.9553, 0.8922, and 0.9057 for the training, validation, and testing data sets, respectively, by ANN technique. It can be concluded that constructing a model with a complete range of the rating scale is more reasonable for bridge condition rating problems compared with constructing the model using only the available rating scale. Furthermore, there was no significant difference between R 2 value of validation and testing set for treated data sets in comparison to data sets M0. The R 2 values for validation and testing set of data sets after missing data that are substituted by SBR are 0.8922 and 0.9057, respectively. Meanwhile, the R 2 values for validation and testing sets of data sets M0 are 0.7515 and 0.8115, respectively. It can also be concluded that the ANN models perform slightly better than MRA in mapping relationship between bridge components and bridge condition rating.