Reliability Analysis Based on Surrogate Modeling Methods

Various surrogate modeling methods have been developed to generate approximate functions of expensive numerical simulations. They can be used in reliability analysis when integrated with a numerical reliability analysis method such as a first-order or second-order reliability analysis method (FORM/SORM), or Monte Carlo simulations (MCS). In this chapter, a few surrogate modeling methods are briefly reviewed. A reliability analysis approach using surrogate models based on radial basis functions (RBFs) and successive RBFs is presented. The RBF surrogate modeling method is a special type of interpolation method, as the model passes through all available sample points. Augmented RBFs are adopted to create approximate models of a limit state/performance function, before the failure probability can be computed using MCS. To improve model efficiency, a successive RBF (SRBF) surrogate modeling method is investigated. Several mathematical and practical engineering examples are solved. The failure probabilities computed using the SRBF surrogate modeling method are fairly accurate, when a reasonable sample size is used to create the surrogate models. The method based on augmented RBF surrogate models is useful for probabilistic analysis of practical problems, such as civil and mechanical engineering applications.


Introduction
The probabilistic analysis of practical engineering problems has been a traditional research field [1][2][3]. The first category of engineering reliability analysis methods are the most probable point (MPP) methods [4][5][6][7]. In this category of methods, a design point, or the so-called most probable point in the design space is sought. The limit state function is often transformed into a standard Gaussian space and approximated using Taylor series expansions. Depending on the order of approximation used, FORM/SORM are available [4][5][6][7]. These methods require the derivatives of system responses, i.e., sensitivity analysis. For complex engineering systems that require expensive response simulations such as nonlinear explicit finite element (FE) analysis, the integration of the MPP-based methods and a commercial FE code is not straightforward. An alternative category of methods are the direct sampling-based methods, including MCS and some other simulation methods [8][9][10][11][12]. These methods can be integrated fairly easily with an existing simulation program because they do not require the derivation or calculation of gradient information. However the direct application of MCS can be computationally prohibitive in complex engineering problems that require expensive response simulations.
To reduce the complexity of implementation and improve the computational efficiency, various approximate modeling techniques have been applied to the reliability analysis of practical engineering systems [13,14]. These approximate models are referred to as surrogate models. There are abundant literature that presented surrogate models and their applications to numerical optimization and reliability-based design optimization. However, the focus of this chapter and the review of literature here is primarily on the applications of surrogate models to engineering reliability analysis. In surrogate modeling methods, the analysis software is replaced by approximate surrogate models, which have explicit functions and are very efficient to evaluate. FORM/SORM or a sampling method can then be applied using the explicit surrogate model instead of the original implicit numerical model. In all the surrogate models developed, the most basic and popular surrogate model is the conventional polynomial-based response surface method (RSM). The RSM has been shown to be useful for different engineering reliability analyses and applications [15][16][17][18][19][20][21][22][23][24][25]. The entire response space is approximated using a single quadratic polynomial function in a global RSM model. To improve model accuracy for reliability analysis using a global RSM model, different techniques were proposed such as efficient sampling methods [26,27] and inclusion of higher order effects [28,29]. When combined with gradient-based search methods, it is more efficient to use RSM in an iterative manner or a local window of the response space [30]. Local RSM methods such as the moving least square technique were developed to handle highly nonlinear limit state functions [31]. Other commonly used surrogate modeling methods have also been developed over the years, such as artificial neural networks (ANN) [32][33][34][35][36][37], Kriging [38][39][40][41][42][43][44][45][46], high-dimensional or factorized high-dimensional model representation [47][48][49][50][51], support vector machine [52][53][54][55][56][57], radial basis functions (RBFs) [58], and even ensemble of surrogates [59][60][61][62]. An RBF surrogate model is a multidimensional interpolation approach using available scattered data. Due to their characteristics in global approximation, RBFs could create accurate surrogate models of various responses [63,64]. An RBF model provides exact fit at the sample points. In the studies by Fang and coauthors [65,66], various basis functions were investigated including Gaussian, multiquadric, inverse multiquadric, and spline functions. Some compactly supported (CS) basis functions developed by Wu [67] were also studied. Mathematical functions and practical engineering responses were tested and their surrogate models were created using different basis functions. Augmented compactly supported functions worked well and were found to create more accurate surrogate models than non-augmented models.

Aims and objectives
It can be seen from literature review that accurate and efficient surrogate models are useful tools when integrated with expensive response simulations for practical reliability analysis and design problems. The objective of this research is to study efficient and accurate RBF models, such as adaptive or successive RBF models based on the augmented basis functions, and their application in engineering reliability analysis. Note that the accuracy of RBF surrogate models depends on the sample size used. If the sample size is too small, the model may not be accurate. On the other hand, a large number of sample points will improve the model accuracy, but some sample points and associated response simulations may not be necessary.
Since the most appropriate sample size is not known before the creation of the surrogate models, it remains a challenge to determine the appropriate sample size to use. One viable approach is to create and test a few different sample sizes, and the best sample size for the problem can be determined. To improve this process, the concept of SRBF surrogate models is developed and it is intended to automate this process and find the proper sample size iteratively and automatically for the augmented RBF surrogate models that can be used for reliability analysis of practical engineering systems.
This chapter presents an engineering reliability analysis method based on a SRBF surrogate modeling technique. In each iteration of the new method, augmented RBFs can be used to generate surrogate models of a limit state function. Three accurate augmented RBFs surrogate models, which were identified from a previous study, are adopted. The failure probability can be calculated using the SRBF surrogate models combined with MCS. Section 3 describes the general concept of engineering reliability analysis. Section 4 briefly reviews some surrogate modeling methods, and explains the augmented SRBF surrogate modeling technique. Sections 5 and 6 presents the MCS method and the overall reliability analysis procedures. In Section 7, the proposed approach is applied to the probability analysis of several mathematical and practical engineering problems. The failure probabilities are compared with those computed based on the direct implementation of MCS without surrogate models. The numerical accuracy and efficiency of the proposed approach using MCS and SRBF surrogate models is studied.

Engineering reliability analysis
A time-invariant reliability analysis of an engineering problem is to compute the failure probability, P F , using the following integral [1][2][3]: where x is an s-dimensional real-valued vector of random variables, g x ð Þ is the limit state function, and p X x ð Þ is the joint probability density function. Eq. (1) is difficult to obtain for practical engineering applications, since p X x ð Þ is unknown and g x ð Þ is usually an implicit and nonlinearity function. A detailed response analysis model, such as the FE analysis of the engineering system is often required to evaluate function values of g x ð Þ.

Design of experiments
An implicit function g x ð Þ is considered, where x = x 1 ⋯ x s ½ T is an input variable vector and s is the number of input variables. Before a surrogate model of function g x ð Þ can be created, some sample points shall be generated using design of experiments (DOE). Some routinely used DOE approaches include factorial design, Latin hypercube sampling (LHS) [68], central composite design, and Taguchi orthogonal array design [69]. Assume x i is the input variable vector at the ith (i = 1,…n) sample point, the limit state function g x ð Þ needs to be evaluated at all the sample points to obtain the function values, i.e.,

Response surface method using quadratic polynomials
Using linear or quadratic polynomials, a response surface model can be developed. The most commonly used quadratic polynomial response surface model is expressed as [63]: where the β's are the unknown coefficients. Using the function values at n sample points, a total of n linear equations can be written in a matrix form, as: where e β k Â 1 ð Þis the least-square estimation of the unknown coefficients in Eq. (2), and X n Â k ð Þis a matrix of input variables at sample points. Apply the least squares method to solve for e β, as:

Least squares support vector machine
The support vector machine (SVM) uses a nonlinear mapping technique and solves for a nonlinear input-output relationship. For n sample points, a commonly used least squares SVM model is given as [52,53]: where α i (i = 1,… n) are Lagrange multipliers, b is the scalar threshold, and K x; x i ð Þis a kernel function. Available kernel functions include polynomial, radial, and sigmoid kernels [53]. A system of (n + 1) equations can be written as: where γ is a tolerance error, and Ω n Â n ð Þ is a matrix of kernels based on the sample points. α and b can be calculated from:

Kriging
The Kriging model is an interpolation technique that combines two parts, i.e., a linear regression part and a stochastic error, as [38,39]: Ã T are the p basis functions, and β ¼ β 1 ⋯ β p Â Ã T are the corresponding regression coefficients. The first part of Eq. (8) approximates the global trend of the original function, in which β can be estimated using the least squares method. The second part, z x ð Þ, represents a stochastic process with zero mean and covariance where σ 2 is the process variance, and R is a correlation matrix. If Gaussian function is used as the correlation function, R x i ; x j À Á is written as: where x k i and x k i are the kth (k = 1,… s) component of sample points x i and x j , respectively, and θ k are unknown correlation parameters to fit the model.

Augmented radial basis functions
Developed for fitting topographic contours, an RBF surrogate model e g x ð Þ is written as: where ϕ is the basis function, x À x i k kis the Euclidean norm, and λ i is the unknown weighted coefficient that need to be determined. Table 1 lists commonly used RBFs.
Using the n available sample points and function values, a total of n equations can be written, as: Write all the n equations in a matrix form, as: Function name Radial basis function where λ ¼ λ 1 ⋯ λ n ½ T , and A is given as: Solve the linear system of Eq. (14) to calculate coefficients λ, as: Since highly nonlinear basis functions are used, the RBF surrogate models in Eq. (11) can approximate nonlinear responses very well. However, they were found to have more errors for linear responses [58]. In order to overcome this drawback, the RBF model in Eq. (11) can be augmented by polynomial functions, as: where the second part represents p terms of polynomial functions, and c j (j = 1,… p) are the unknown coefficients to be determined. There are more unknowns than available equations; therefore the following orthogonality condition is required to solve for all unknowns, as: Eqs. (17) and (18) consist of (n þ p) equations in total, and they can be rewritten, as: where c ¼ c 1 ⋯ c p Â Ã T , and F is given as: Solve the linear system of Eq. (19) to get λ and c, as: For augmented RBFs, either linear or quadratic polynomial functions can be used. In this study, only linear polynomial functions were added to Eq. (17). For the rest of the paper, a suffix "-LP" is used to represent linear polynomials added to RBFs. The following RBF models were studied: SRBF-MQ-LP: sequential multiquadric function with linear polynomials. SRBF-CS20-LP: sequential compactly supported function ϕ 2, 0 with linear polynomials.

Estimation of failure probability
Eqs. (11) and (17)  can be efficiently calculated in each iteration of the SRBF approach. Based on the surrogate model e g x ð Þ, the failure probability P F can be computed using a sampling method, such as MCS, as: where N is the total number of MCS samples, x i is the ith realization of x, and Γ is a deciding function, as: The reliability index β can be further determined, as [49]: where Φ is the standard normal cumulative distribution function.
6. Reliability analysis based on successive RBF models Figure 1 shows a flowchart of reliability analysis using SRBF-based surrogate modeling technique and MCS. Once the explicit augmented RBF surrogate model is generated in one iteration of the proposed method, MCS is applied to efficiently estimate the failure probability for any sample size. If the convergence criterion is not satisfied in the current iteration, more sample points will be added and another iteration starts. As the sample size increases, the SRBF surrogate models in general become more accurate, a reduction was observed in the failure probability estimation errors. However this results in more function evaluations. Since the number of response simulations is determined by the sample size used to create a surrogate model, the majority of the computational cost is from the response simulations. The detailed procedure is as follows: 1. Determine initial and additional sample sizes, n and m, and convergence criterion. In this study, the initial sample size n is suggested be 5-10 times of the number of random variables s. The additional sample size m in each subsequent iteration can be typically taken as one third to one half of the initial sample size, n.
2. Generate the initial sample set with n sample points; set the iteration number k ¼ 1. A commonly used LHS was applied to generate samples for RBF surrogate models.
3. Evaluate limit state function g x ð Þ for the initial sample set n generated in Step 2. Numerical analyses such as FE analyses may be required for practical problems.
4.Update sample set n to include all sample points, n ¼ n þ m. For the first iteration (k ¼ 1), m ¼ 0, and no additional sample points are added.
5. Construct augmented RBF surrogate models e g x ð Þ of function g x ð Þ based on Eq. (17) using all available sample points. 7 Reliability Analysis Based on Surrogate Modeling Methods DOI: http://dx.doi.org/10.5772/intechopen.84640 6. Calculate failure probability P F for iteration k using MCS. 7. Check the convergence criterion. If the convergence criterion is satisfied, stop; otherwise go to Step 8. In this study the convergence criterion is that the relative error of the failure probability P F between two successive iterations is less than the tolerance. A tolerance value of 1% was applied in this study. For practical applications, another convergence criterion may be defined, e.g., the maximum number of response simulations has been reached. This will help control the total number of iterations performed in the reliability analysis.
8. Generate additional sample set with m sample points; set the iteration number k ¼ k þ 1.
9. Evaluate limit state function g x ð Þ for the additional sample set m generated in Step 8, then go to Step 4.

Numerical examples
Four numerical examples were solved using the proposed reliability analysis method. These include both mathematical and engineering problems found in literature. In this study, the proposed method based on three SRBFs, i.e., SRBF-MQ-LP, SRBF-CS20-LP, and SRBF-CS30-LP, is referred to as the SRBF-based MCS. The Direct MCS refers to MCS without using surrogate models. In the Direct MCS, the number of response simulations was determined by the MCS sample size. However, in the SRBF-based MCS, the number of response simulations was based on the surrogate modeling sample size. A total of N = 10 6 samples was adopted in MCS when surrogate models were used.

Example 1: a nonlinear limit state function
A nonlinear limit state function was studied in literature, as [21,49,50]: where x 1 and x 2 are independent random variables following standard normal distributions (mean = 0; standard deviation = 1). The failure probability P F = 0.009372 was obtained based on Direct MCS and used to compare with other solutions. The RBF surrogate models were constructed using the two variables sampled in the range of À3.0 to 3.0. All three surrogate models started with 10 sample points in the first iteration. With 10 sample points, the error of the estimated failure probability was 7.0, 3.0, and 1.8% for SRBF-MQ-LP, SRBF-CS20-LP, and SRBF-CS30-LP, respectively. In each subsequent iteration 10 more sample points were added. At convergence, the accuracy of SRBF models was improved; the error was reduced to 0.9, 0.8, and 1.3% for SRBF-MQ-LP, SRBF-CS20-LP, and SRBF-CS30-LP, respectively. Adequate accuracy of reliability analysis was achieved for all three SRBF surrogate models. The failure probability values obtained based on three surrogate models and the associated errors as compared to the solution obtained using Direct MCS are listed in Table 2. It took 4, 3, and 2 iterations for SRBF-MQ-LP, SRBF-CS20-, and SRBF-CS30-LP methods to converge, corresponding to 40, 30, and 20 sample points, respectively. A total of 40, 30, and 20 function evaluations (original limit state function) were required for the three SRBF-based MCS, respectively.

Example 2: a cantilever beam
The reliability analysis of a cantilever beam with a concentrated load is conducted in this example [50]. The beam has a rectangular cross section. The performance requirement is the displacement at tip should be <0.15 in. Therefore, the limit state function is.
where P is the concentrated load, l is the beam length, b and h are the width and depth of the beam cross-section, and E = 10 7 psi is the Young's modulus. In this example P = 80 lb. was considered. Table 3 lists the three random variables in this problem, i.e., l, b, and h.
All three SRBF surrogate models started with 20 sample points in the first iteration, with 10 more samples generated in each following iteration. The reliability analysis results and the corresponding sample sizes required for SRBF surrogate models were examined, as listed in Table 4. The failure probability estimated based on Direct MCS using Eq. (26) was 0.02823, which was regarded as the actual solution. It took 4, 7, and 5 iterations for SRBF-MQ-LP, SRBF-CS20-LP, and SRBF-CS30-LP to converge, respectively. With the initial 20 samples, the error of the estimated failure probability was 35.9, 19.4, and 9.7% for SRBF-MQ-LP, SRBF-CS20-LP, and SRBF-CS30-LP, respectively. With 50, 80, and 60 sample points, the error was reduced to 9.7% for SRBF-MQ-LP, 0.3% for SRBF-CS20-LP, and 1.7% for SRBF-CS30-LP. The errors in estimating the failure probability by SRBF surrogate models decreased as the sample size increased. The SRBF-MQ-LP model did not produce as accurate estimation of P F as SRBF-CS20-LP and SRBF-CS30-LP, when the same sample size was used. In all three SRBF surrogate models, SRBF-CS20-LP provided the most accurate estimate of P F , and the surrogate model SRBF-MQ-LP did not converge close to the actual solution. In this example, 60-80 sample points were required for SRBF-CS20-LP and SRBF-CS30-LP to achieve reasonably accurate surrogate models and estimates of the failure probability.

Example 3: a reinforced concrete beam section
This example is the reliability analysis of a singly-reinforced concrete beam section [51,70]. Based on static equilibrium, the following nonlinear limit state function can be developed, as: Table 3. Example 2: random variables [50].
Eq. (27) included six independent random variables: x 1 is the total crosssectional area of rebars, x 2 is the yield strength of rebars, x 3 is the effective depth of section, x 4 is a dimensionless factor related to concrete stress-strain curve, x 5 is the compressive strength of concrete, and x 6 is the width of the concrete section. The limit state was for the ultimate bending moment strength of the section, and a bending moment limit M n ¼ 211:20 Â 10 6 N-mm was adopted in this study. Table 5 lists the six input random variables and their statistical properties.
To start the reliability analysis, 30 sample points were used in the first iteration of all three SRBF surrogate models, and 10 additional samples were included in each subsequent iteration. Table 6 lists the failure probability P F values obtained using different methods, in addition to the required number of original function evaluations, representing the associated computational effort. Compared with P F = 0.01102 obtained by Direct MCS, the errors of SRBF-MQ-LP, SRBF-CS20-LP and SRBF-CS30-LP were 0.8, 1.1, and 0.9%, respectively. Figure 2 is the plot showing failure probability estimation versus sample size. All three SRBF models worked well and smooth convergence histories can be observed. The three SRBF models produced similar failure probabilities. The results by SRBF-CS20-LP and SRBF-CS30-LP were shown to be better than that using SRBF-MQ-LP when the sample size was small. Among the three SRBF models, SRBF-CS30-LP generated the most accurate approximation with the same sample size. As expected, more sample points resulted in reduced SRBF approximation errors. With the increase of the number of sample points or function evaluations (i.e., computational effort), a reduction in estimation error of the failure probability using the proposed SRBF models was observed. For example, the estimation error of P F was reduced from 10.7 to 0.8% for SRBF-MQ-LP, 4.9-1.1% for SRBF-CS20-LP, and 4.1-0.9% for Table 5. Example 3: random variables [70]. SRBF-CS30-LP, respectively. SRBF-CS20-LP and SRBF-CS30-LP created with 40 samples and SRBF-MQ-LP created with 50 samples could provide fairly accurate reliability analysis results (<2% error of P F ).

Example 4: burst margin of a rotating disk
This example is the reliability analysis of a disk with an angular velocity of ω, as shown in Figure 3 [50,51]. The inner and outer radii of disk are R i and R o , respectively. The burst margin, M b , of the disk refers to the safety margin before overstressing the disk, which is expressed as: If a lower bound value of 0.37473 is used, the limit state function of M b can be written as: where S u is the ultimate material strength, α m is a dimensionless material utilization factor, and ρ is the mass density of material. Table 7 lists the six random variables used in the example. Similar as Example 3, all three surrogate models started with 30 sample points. In each subsequent iteration, 10 sample points were added. Table 8 lists the  estimated failure probability P F in this study based on different SRBF surrogate models and the associated errors as compared to the solution obtained using Direct MCS. The augmented SRBF-based methods required 60-70 original function evaluations to converge. Figure 4 illustrates the variation of the failure probability P F versus number of sample points. In general with the increase of the sample size, a reduction was observed in the estimation errors of the failure probability P F , from 67.1, 6.6, and 12.8% when 30 sample points were used, to 5.6, 0.8, and 0.5% at convergence for SRBF-MQ-LP, SRBF-CS20-LP, and SRBF-CS30-LP, respectively. The reliability analysis results based on surrogate models SRBF-CS20-LP and SRBF-CS30-LP were shown to be better that using SRBF-MQ-LP. It showed that with around 50 sample points very accurate SRBF-CS20-LP and SRBF-CS30-LP surrogate models could be created for reliability analysis. Table 7. Example 4: random variables [50,51].

Concluding remarks
Augmented RBFs are suitable for creating accurate surrogate models for linear and nonlinear responses. When combined with a sampling method such as MCS, they can be used in reliability analysis and provide accurate estimation of the failure probability. In spite of their excellent model accuracy, the most appropriate number of sample points is not known beforehand. To provide an improved and automated approach using the RBF surrogate models in reliability analysis, a SRBF surrogate modeling technique was developed and tested in this study, so that the RBF surrogate models could be used in an iterative yet efficient manner. In this chapter, three augmented RBFs, including multiquadric function and two compactly supported basis functions were considered. To evaluate the proposed SRBF surrogate modeling method for reliability analysis, its numerical accuracy and computational efficiency was examined.
Numerical examples including existing mathematical and engineering problems were studied using the proposed method. Accurate failure probability results were achieved using a reasonable sample size within a few iterations. The required number of response simulations or function evaluations was relatively small. All three SRBF models produced similar accuracy, and the surrogate models based on SRBF-CS20-LP and SRBF-CS30-LP produced more accurate reliability analysis results, especially when a smaller sample size was adopted. This study shows that the proposed reliability analysis method is efficient and has a promising potential for application to complex engineering problems involving expensive simulations. Further research includes efficient sequential sampling methods that can be combined with the SRBF methods, and the optimal approach to determine the sample sizes used in each iteration of the SRBF methods.