Reliability Assessment with a Small Amount of Data

Abstract

This chapter describes various methods for reduction of uncertainties in the determination of characteristic values of random quantities (quantiles of normal and Weibull distribution, tolerance limits, linearly correlated data, interference method, Monte Carlo method, bootstrap method).

Many quantities necessary for reliability assessment are obtained by observation or measurement (load and material characteristics). Often, the number of tests n is low. As a consequence, the distribution parameters are only estimated. Their true values can be different and thus also the other characteristics, such as quantiles. This could be dangerous especially if n is very low (tens or less). Uncertainty arising from a small amount of data should be taken into consideration in any reliability assessment. This chapter presents four methods that can increase the safety of these assessments. The first method is related to the determination of the guaranteed lowest or highest values (i.e. low probability quantiles of normal or Weibull distribution). The second method tries to mitigate the uncertainties related to the use of the S–R interference method. The third method is devoted to the uncertainties of the Monte Carlo simulations if the input distributions were obtained from a small amount of measurements. The last method explains briefly the principle of the so-called bootstrap technique.

1. Guaranteed values of quantiles

Guaranteed or safe value of a random quantity x is such value that will be exceeded (e.g. load) or not-attained (strength) only with a very low probability α. This value corresponds to α-quantile in the latter case and to α-critical value in the former case; note that the α-critical value corresponds to (1 – α)-quantile.

A quantile can be calculated as the inverse value of the distribution function. However, because usually only the estimates of the parameters of a population are known, only an estimate of the quantile is obtained in this way. Its true value can be different. As a consequence, in some cases, the actual reliability will be lower than assumed, which is dangerous. The difference can be high especially if the statistical parameters of the population were obtained from a very small number of samples. For this reason, confidence interval should always be determined for the quantiles. Two very important cases are normal and Weibull distribution.

Quantiles of normal distribution

The α-quantile of a quantity x is calculated as

$$x_{\alpha }=\mu +u_{\alpha }\sigma ,$$E1

μ and σ are the mean and standard deviation and u_α is the α-quantile of the standard normal distribution. The uncertainty in the determination of the distribution parameters μ and σ can be reduced by means of confidence limits for the quantile. For quantities with normal distribution, usually tolerance limits x_α,tol are used instead. The lower (upper) tolerance limit demarcates the fraction α of the population, which will be lower (higher) than x_α,tol only with probability α, the risk of a wrong estimate being γ. The tolerance limits are determined using the formula [1, 2]

$$x_{\alpha ,tol}=\overline{x}\pm k_{\alpha }s,$$E2

and s are the sample average and standard deviation, and k_α is the one-sided tolerance factor, which depends on α, the number of measurements n, and the risk γ that the prediction will be wrong. The plus sign pertains to the upper limit, whereas the minus sign pertains to the lower limit.

The difference between the values obtained using Eq. (1) or (2) is large especially for low n, α and γ. For example, the 0.1%-quantile according to (1) is μ – 3.09σ, while the 0.1% lower tolerance limit (for γ = 10%) is – 3.44s for n = 100, – 3.79s for n = 30, and – 4.63s for n = 10. If, for example, strength tests were made only on 10 specimens, and the standard deviation was obtained as s = 0.1, then the strength guaranteed with 99.9% is by (4.63–3.09)s = 0.15, i.e. by 15% of the average strength lower than the less conservative value according to Eq. (1)

The tolerance factors k can be found in statistical tables, e.g. [2]. Their exact determination is difficult. An approximate formula was proposed by Wallis [1, 3]:

$$k_{\alpha }=\left(u_{\alpha }+u_{\gamma }\sqrt{\frac{1}{n}+\frac{u_{\alpha }{}^2}{2(n-1)}-\frac{u_{\gamma }{}^2}{2n(n-1)}}\right)/\left(1-\frac{u_{\gamma }{}^2}{2(n-1)}\right),$$E3

where u_α and u_γ denote the α- and γ-critical values of the standard normal distribution. The approximation (3) coincides with the exact solution for n ≥ 500, and the difference slightly increases with decreasing sample size. For the confidence level 1 – γ = 0.9, the difference exceeds 1% for n < 20 and 2.5% for n < 10 regardless the value of α.

This method can also be used for quantities with log-normal distribution. They can be transformed (by logarithms) into variables with normal distribution. Then, tolerance limits for quantiles of this new quantity can be calculated using the above approach. Finally, the tolerance limits for the original quantity are found by inverse transformation.

Quantiles of three-parameter Weibull distribution

Weibull distribution has a very flexible character and is used very often to characterize the strength or time to failure. The α-quantile x_α of a three-parameter Weibull distribution is determined as

$$x_{\alpha }=x_0+a\text{}{\left[-\text{ln}(1-\alpha )\right]}^{1/b},$$E4

where a and b are the scale and shape parameters of the distribution, respectively, and x₀ is the parameter of its position (threshold value). Again, only estimates of the true parameters can be obtained from a sample of limited size and thus also only an estimate of the quantile x_α. Especially for small sample sizes (n equal several tens or less), the sample parameters can differ significantly from those of the population [4]. Sometimes, the threshold value x₀ is assumed zero for safety reasons. However, this can yield unreasonably low values of low probability quantiles. The three-parameter distribution can be better, but confidence limits should always be given with quantiles, especially for small samples and low probability quantiles. These limits (L - lower; U - upper) can be computed as

$$x_{\alpha ,L,U}=x_{\alpha }\pm {\mathrm{\Delta }}_{\alpha ,\gamma ,n},$$E5

where ∆_α,γ,ν is a certain function of the distribution shape, scatter of the measured values, probability α, confidence level 1 – γ, and number of measurements n. Mann et al. [5] proposed a method for the determination of the confidence limits. Unfortunately, the procedure is complicated and tabulated values must be used. Menčík [4] has proposed a simple approximate expression for ∆ based on the variation of the position and slope of the distribution function:

$$\mathrm{\Delta }=\sqrt{t_{\gamma ,n-1}{}^2\frac{s^2}{n}+{\left[1-{\left(\frac{2n}{{\chi }^2{}_{\gamma ,2n}}\right)}^{1/b}\right]}^2{\left(x_{\alpha }-\overline{x}\right)}^2},$$E6

where and s_x are the sample average and standard deviation, respectively; n is the number of measurements from which the distribution parameters were estimated; t_{γ, n-1} is the one-sided γ-critical value of the Student’s distribution for n – 1 degrees of freedom; b is the shape parameter; and χ²_γ,2n is the γ-quantile of the χ²–distribution for 2n degrees of freedom. The probability that the true value of x_α will be lower than the lower confidence limit is γ. The use of confidence intervals for quantiles may strongly be recommended if n < 100.

Linearly correlated data

If fatigue or wear processes occur, the relationship between characteristic load S and cycles to failure N can usually be described by a power-law function:

$$N=A{S}^{-m},$$E7

where A and m are constants. This equation corresponds to a straight line

$$y=a+bx,$$E8

in coordinates x = log S and y = log N. The constants a = log A and b = – m are obtained by testing several specimens under various stress levels S, measuring the cycles to failure N, and fitting the transformed y(x) data by linear regression function. Typical of fatigue is the large scatter of times to failure. This must be taken into account when determining the guaranteed time to failure for a given service stress or the allowable stress for the required lifetime.

The situation is easier if 10 or more specimens were used for each stress level: the pertinent N-values in each level can be ordered so that they (approximately) correspond to quantiles. For example, if N = 10, then the lowest value corresponds to the 10% quantile, the second lowest corresponds to 20% quantile, etc. Then, the S–N curves may be constructed for various probabilities of survival by fitting only the pertinent quantiles. This is the best solution. Unfortunately, fatigue tests are time and cost demanding, so that often only several specimens are tested and Equation (8) is fitted to all data, thus representing the mean line. In this case, 50% probability exists that the true times to failure will be lower than those predicted via this line. Therefore, confidence intervals are also needed. The confidence interval for the points on the regression line (Fig. 1) is

$$y=a+bx\pm t_{\alpha ,\nu }{s}_{res}\sqrt{\frac{1}{n}+\frac{{(x-\overline{x})}^2}{(n-1)s_x{}^2}},$$E9

Figure 1.

Confidence interval around regression curve (a schematic).

where and s_x are the average value and standard deviation of x, calculated from all n values x_j used for the determination of regression constants a and b; t_α,ν is one-sided α-critical value of t-distribution for ν = n – 2 degrees of freedom, and

$$s_{res}=\sqrt{\frac{{\displaystyle \sum {(y_j-a-b{x}_j)}^2}}{n-2}}$$E10

is the residual standard deviation, characterizing the scatter around the regression line; the summation is done over all measured values of y_j.

The modification of Equation (9), as proposed in [4], gives the approximate expression for tolerance limits for single values of y,

$$y=a+bx\pm t_{\alpha ,\nu }{s}_{res}\sqrt{{\left(\frac{k_{\alpha }}{t_{\alpha ,\nu }}\right)}^2+\frac{{(x-\overline{x})}^2}{(n-1)s_x{}^2}};$$E11

the minus (plus) sign pertains to the lower (upper) limit. In Equation (11), k_α is a one-sided tolerance factor, which can be calculated using Equation (3). The probability that y(x) will be lower or higher than the tolerance limit (11) equals α, whereas the probability that this estimate is wrong is γ.

The intervals (11) for all y(x) form a tolerance band around the regression line (9). The tolerance limits for the actual number of cycles (or time) to failure can be obtained using the inverse transformations S = 10^x, N = 10^y, A = 10^a, and m = – b.

2. Interference method for normally distributed stress and strength

The interference method, suitable if random “load” acts on an “object” whose “resistance” also exhibits random scatter, was explained in Chapter 14. Failure occurs if the load effect S is higher than the resistance R. If the distributions of R and S interfere, the distribution of reliability margin G = R – S can be found, and the probability of failure is determined as the value of the distribution function for G = 0. The solution is easy if R and S are normally distributed and their parameters are known, because here the distribution of G is also normal, with parameters

$${\mu }_G={\mu }_R-{\mu }_S,{\sigma }_G=\sqrt{{\sigma }_R^2+{\sigma }_S^2};$$E12

μ and σ are the mean value and standard deviation; the subscript denotes the pertinent quantity. G can be transformed to standard normal variable u using the relationship

$$G={\mu }_G+u{\sigma }_G.$$E13

Using the failure condition, G = 0, the probability of failure P_f, can be found as the value of the distribution function for u = – μ_G/σ_G.

However, instead of the parameters μ and σ of the distributions R and S, only their estimates x and s are usually inserted into Eq. (13), which were obtained from samples of limited size n_R and n_S. As a consequence, one obtains only the estimates x_G and s_G of the reliability margin. In such case it is necessary to use one-sided tolerance factor k instead of quantile u in Eq. (13). Otherwise the actual probability of failure can differ from the forecasted one by more than one order [6, 7]. The tolerance factor k should be determined for the confidence level γ of the forecast and for the equivalent size n_G of the sample G, which must be calculated from the sample sizes and standard deviations of the samples R and S via the relationship

$$G={\mu }_G+u{\sigma }_G.$$E14

The probability of failure is found as that corresponding to the lower tolerance limit

$$k={\overline{x}}_G/s_G.$$E15

When dimensioning for given probability of failure, k is determined first via Eq. (3). For this value, x_G is calculated from Eq. (15). Finally, the nominal size x of cross-section is determined from x_G and the known mean value of the load using Eq. (12). Diagrams for this purpose are given in [6, 7].

3. Monte Carlo method

If the investigated quantity x is a function of random input variables, its distribution can be obtained easily by the simulation Monte Carlo method (Chapter 15). In the simulation trials, random values are assigned to input quantities so that their distribution corresponds to the probability distribution of the pertinent variable. However, the distribution parameters used in the simulations were obtained from samples of limited size. This means that the actual distributions can differ more or less from those used in the simulation. The corresponding uncertainties and errors persist in the results regardless of the number of Monte Carlo trials but can be reduced in two ways.

The first approach [8] uses random variation of distribution parameters. If the distributions of parameters of a random quantity x are known, this quantity can be generated so that, in each trial, random values are also assigned to its parameters, so that they vary randomly in their probable range. For example, the random quantity x of normal distribution with the known mean value μ and standard deviation σ can be generated using formula (1) with u_α replaced by the random value u of the standard normal distribution. If, instead of true parameters μ and σ, only the sample estimates m and s are known, the probable values of μ and σ can be generated in individual trials using modified expressions for their confidence limits. The corresponding formula for random values of x is

$$x=m+t_{n-1}\frac{s}{\sqrt{n}}+us\sqrt{\frac{n-1}{{\chi }^2{}_{n-1}}};$$E16

u, t, and χ² are random values of normal, t, and χ² distribution, respectively (t, χ² for n – 1 degrees of freedom). One value of x thus needs three random numbers to be generated. The quantiles of t and χ² distributions can be expressed approximately by means of quantiles of standard normal distribution; a review of various approximations can be found in [1].

The second method [4] adds a random component to each generated number. The basic random values x₀ of a quantity x are created (via the inverse probabilistic transformation F^–1) from random values F uniformly distributed in interval (0; 1). Then, a random component ∆ is added to each value x₀ created so that the result

$$x(F)=x_0(F)+\mathrm{\Delta }(F)$$E17

has the same probability distribution around x₀ like the quantiles of the genuine variable x. The obtained x values create a blurred confidence band around their distribution function.

4. Bootstrap method

This method, which also uses the Monte Carlo simulations, was originally used for finding the statistical characteristics of random quantities from a relatively small number of data n [9]. It creates its own population, consisting only of the experimental values. From this population, n values are chosen randomly, and the characteristic X of interest is calculated (e.g. mean or a quantile). This procedure is repeated many times. Then, the α-confidence interval for X is determined by one of the following ways. In the first approach, the average value m_x and standard deviation s_X of the pertinent quantity are calculated, and the lower (L) and upper (U) confidence limits are found using the expression

$$X_{L,U}=\overline{x}\pm u_{\alpha }{s}_X,$$E18

where u_α is the α-quantile of standard normal distribution; the probability of X being outside the limits (18) is 2α. This approach assumes that X has normal distribution. This needs not always be true, and various improvements were later proposed. (For more details, see [10].)

Another approach generates a large number of simulated data sets (at least 100). Then, the values of the characteristic of interest, calculated for each data set, are ranked into ascending order. The confidence bounds, corresponding, for example, to the 90% confidence interval, are obtained using the 5th and 95th values of the ordered values. However, when determining the confidence bounds for quantiles, this approach may only be used for quantiles sufficiently far from the tails of the distribution.

The bootstrap method can also be used in reliability assessment by the Monte Carlo technique, the more so that each simulation set gives a different value of X (e.g. the probability of failure P_f). Thus, the Monte Carlo simulation sets are repeated many times. In each set, P_f, is determined. Then, its probable highest value is found by one of the above approaches.

5. Concluding remarks

Reliability assessment based on a small number of experimental values always means risk. A very important condition for the use of any probabilistic method is that the experimental sample adequately represents the whole population. The situation can be very dangerous if the population is not homogeneous, for example if several kinds of flaws and other defects can be responsible for the strength of a brittle material [11]. All characteristic kinds of defects must be present in the experimental sample (including the largest but rare ones), otherwise the predicted values of low probability strength can be wrong despite the determination of their confidence limits. The probability that a defect of probability of occurrence (e.g. 1:1000) will be found in a small sample consisting of only 10 pieces is really very low (only 1% compared to the probability 99% that such flaw will not be revealed). A similar situation exists, for example, when the maximum height of water waves at the sea coast should be predicted. Statistical characteristics can be obtained from long-term measurements, but if the waves at extremely rare tsunami events have not been included into the evaluation, the new coast structures will not be sufficiently protected against them.

When dealing with reliability assessment of some kind of structures, it is thus reasonable to gradually gather the data of all measurements (for the pertinent material or load) and to combine the newer data with older ones. For this purpose, Bayesian methods may be suitable, explained briefly in Chapter 22.