Bayesian Methods

Abstract

Probabilistic Bayesian methods enable combination of information from various sources. The Bayes theorem is explained and its use is illustrated on several examples of practical importance, such as revealing the cause of an accident or reliability increasing of non-destructive testing. Also its use for continuous quantities and for increasing the reliability of the parameters of normal or Weibull distribution is shown.

The term “Bayesian methods” denotes probabilistic methods that enable the combination of information on some event or quantity with previous information from measurement or experience. The use of additional information can increase the reliability of our information or reduce the extent of measurements needed for making conclusions on certain event. Examples of application are the determination of the most probable cause of a failure, increasing the reliability of diagnostic methods or increasing the accuracy of the determination of distribution parameters of random quantities.

Bayesian methods are based on the so-called Bayes theorem [1 – 6]. It was originally formulated for discrete quantities, but extended later for continuous quantities as well. These methods have also been included into standards. In this chapter, their principle will be explained, and the use is shown on several practical examples.

1. Bayes theorem

Let us assume that an event (B) can occur if another event (A) has occured. The event A, however, could occur by several ways (A₁, A₂,..., A_n), which are mutually exclusive. The probability of simultaneous occurence of both events A_j and B is calculated as

$$\left(B{A}_j\right)=P\left(A_j\right)\times P\left(B|A_j\right),$$E1

where P(A_j) is the probability of event A_j, and P(B|A_j) is (conditional) probability that event B can occur provided that event A_j has happened. The total probability of event B is

$$P\left(B\right)=\sum P\left(B{A}_j\right);$$E2

the summation is done for all possible cases j = 1, 2,..., n. Bayes theorem looks at the issue from the opposite side: ”If event B has happened, what is the probability that it was as a consequence of (or after) event A_j?“ With the use of Equations (1) and (2) and the fact that P(BA_j) = P(A_jB), this probability can be expressed as [2 – 6]:

$$P\left(A_j|B\right)=P\left(A_j\right)\times P\left(B|A_j\right)/P\left(B\right),$$E3

where the total probability P(B) in the denominator is calculated from individual probabilities via Equations (2) and (1). Equation (3) is the simplest form of Bayes theorem. Its use will be shown on three examples. The first example, adapted from [4], does not solve a reliability problem, but is very instructive.

Example 1. Identification of origin of a sample from several possible sources.

The materials for road building are delivered from two plants with daily capacities 300 t (plant 1) and 700 t (plant 2). The long-term monitoring of quality shows that plant 1 has 2% of all batches faulty and plant 2 has 4% faulty batches. If now a sample is chosen at random at the building site, and if this sample is faulty, which plant is the batch from?

From the total amount of 300 + 700 = 1000 t/day, plants 1 and 2 produce 30% and 70%, respectively. Let us denote event A₁: the sample is from plant 1; event A₂: the sample is from plant 2. The corresponding probabilities are P(A₁) = 0.3; P(A₂) = 0.7. Event B: the sample is defective. The probability of defective sample from plant 1 is P(B|A₁) = 0.02, and from plant 2, it is P(B|A₂) = 0.04. The total fraction of faulty production is: P(B) = 0.3 × 0.02 + 0.7 × 0.04 = 0.034 = 3.4%. The defective material from plant 1 represents 0.02 × 0.3 = 0.006 from the total production of both plants. This is 0.006/0.034 = 0.176 = 17.6% from the total faulty production. Similarly, plant 2 produces 82.4% of the scrap. These numbers also say that if the randomly chosen sample was faulty, a probability of 17.6% exists that it is from plant 1 and 82.4% that it is from plant 2.

Using Bayes rule (3), one can express the probability that the defective specimen is from plant 1 as P(A₁|B) = P(A₁) × P(B|A₁)/P(B). The values 0.3 × 0.02/0.034 yield the same result 0.176 as above. Similarly, the probability that the faulty sample is from plant 2, P(A₂|B) = 0.7 × 0.04/0.034 = 0.824 (=1 – 0.176).

If the quality is not considered, the probability that a randomly chosen sample comes from plant 1 equals 30% (i.e. the fraction of production from plant 1). If, however, additional information ”the sample was defective“ was used together with the information on quality in both plants, this probability has dropped to 17.6%. The same information has increased the probability of the sample being from plant 2 from 70% to 82.4%. Although the probability that a sample is from plant 2 was higher even without the Bayes rule (70%), the strengthening of this hypothesis is obvious.

Further strengthening of the hypothesis by using more tests

The hypothesis ”the material is from plant 1 (or 2)“ can be strengthened (or mitigated) by checking more specimens. If n specimens are taken from one batch, and if all appear to be defective (= event B’), then the expression P(B|A_i) in Bayes rule (3) must be replaced by the expression P(B’|A_i) = P(B|A_i)ⁿ. For example, if three specimens were taken from a batch from the above example, and if all were faulty, then P(B’|A₁) = 0.02³, P(B’|A₂) = 0.04³, P(B’) = 0.3 × 0.02³ + 0.7 × 0.04³ = 0.0000472, and P(A₁|B’) = 0.3 × 0.02³/0.0000472 = 0.05. Similarly, P(A₂|B’) = 0.95. In such case, it is nearly sure that the batch was from plant 2.

Example 2. Revealing the most probable cause of an accident.

This example is adapted from [2]. An explosion occurred during a repair of a tank for liquid natural gas. The accident could have happened due to (1) static electricity, (2) fault in the electric equipment, (3) work with open flame during the repair, or (4) intentional act (sabotage). Engineers for risk analysis estimated that the accident could happen with a probability of 25% due to static electricity, 20% due to a fault in the electric equipment, 40% due to work with open flame, and 75% due to a sabotage. The discussion with them also gave the following subjective assessment of probability of individual causes: 0.30, 0.40, 0.15, and 0.15. What is the most probable cause of the explosion in view of all this information?

Solution. Event A: presence of conditions for explosion: P(A₁) = 0.30; P(A₂) = 0.40; P(A₃) = 0.15; P(A₄) = 0.15 (note: ΣP(A_i) = 1.00). Event B: explosion. The probabilities of explosion under particular conditions are P(B|A₁) = 0.25; P(B|A₂) = 0.20; P(B|A₃) = 0.40; P(B|A₄) = 0.75. Total probability of the accident: P(B) = 0.30 × 0.25 + 0.40 × 0.20 + 0.15 × 0.40 + 0.15 × 0.75 = 0.3275. Probability that the explosion has happened due to: (1) static electricity: P(A₁|B) = 0.30 × 0.25/0.3275 = 0.229 = 22.9%, (2) electric appliance: P(A₂|B) = 24.4%, (3) open flame: P(A₃|B) = 18.3%, and (4) sabotage: P(A₄|B) = 34.3%. [Compare these updated probabilities with the original estimates P(A_i).]

Example 3. Increasing the reliability of nondestructive testing.

Welded components are tested for the occurrence of defects (cracks). The device used for nondestructive testing is not perfect. It classifies defect correctly (as defect) only with probability 98%, whereas, in 2% of all cases, it does not recognize the crack and classifies the component as good. On the contrary, the device marks 96% of good parts as good, but 4% classifies as with a crack. According to long-term inspection records, 3% of all tested components contain cracks. The questions are: If the tested part was classified as ”wrong“ (i.e. with a defect), what is the probability that it is actually (a) wrong or (b) good? And what about if the component was classified as ”good“?

Solution. Event A₁: Component contains a defect, A₂: component is good. P(A₁) = 0.03; P(A₂) = 0.97. Event B: component is classified as wrong. P(B|A₁) = 0.98; P(B|A₂) = 0.04. The fraction of tested components marked as wrong: P(B) = 0.03 × 0.98 + 0.97 × 0.04 = 0.0682.

Case 1a. Probability that the component marked as wrong is actually wrong, is P(A₁|B) = P(A₁) × P(B|A₁)/P(B) = 0.03 × 0.98/0.0682 = 0.431 = 43.1%. Case 1b. Probability that the component marked as wrong, is actually good, is P(A₂|B) = 0.97 × 0.04/0.0682 = 0.569 = 56.9%. (Remark: Due to the high proportion of good parts (98%), the proportion of good but rejected parts is high.)

Event B’: Component is classified as good. P(B’|A₁) = 0.02; P(B’|A₂) = 0.96. The total fraction of components, denoted as good, is P(B’) = 0.03 × 0.02 + 0.97 × 0.96 = 0.9318. Case 2a. Probability that the component marked as good is actually wrong, is P(A₁|B’) = 0.03 × 0.02/0.9318 = 0.00064 = 0.06%. Case 2b. Probability that the component marked as good is actually good is P(A₂|B’) = 0.99936 = 99.94%.

Recommendation: All rejected components could be tested once more to reduce the number of discarded good components.

A similar approach can be used in medicine (e.g. in cancer screening).

2. Bayes rule for continuous quantities

If the probability of event B depends on the value of a continuous quantity A, described by the probability density f(A), it is possible to calculate the total probability of this event as

$$P\left(B\right)=\int \left[P\left(B|A\right)f\left(A\right)\right]\text{ d}A;$$E4

the integration is performed over the whole domain A. [The integration has replaced the summation in Equation (2).] An example is the nondestructive detection of cracks in welds: P(B) is the probability of crack detection, f(A) is the probability distribution of cracks of size A, and P(B|A) is the probability of detection of a crack of size A, the so-called ”probability of detection“ curve (shortly POD curve) of the device.

Now, a question can be asked: If event B (result of the test) has occurred, what is the actual distribution of random variable A? Bayes rule (3) can be modified also for this case; the formula for updated distribution of quantity A is [1, 3]:

$$f\left(A|B\right)=f\left(A\right)\times P\left(B|A\right)/P\left(B\right),$$E5

where P(B) is given by Equation (4). For example, the updated distribution of crack lengths can be used for the estimation of time to fatigue failure by the Monte Carlo method [1, 2].

2. Other applications

Bayesian methods can also be used for the improvement of parameter estimate of various probability distributions. Three examples follow.

Parameters of normal distribution

The mean value μ and standard deviation σ of a population with normal distribution are usually unknown, so that they are replaced by their estimates m and s from a sample of size n. The estimate of the mean value can be refined via confidence interval:

$$m–t_{\alpha }{}_{,\nu }\surd \left(s/n\right)\le \mu \le m+t_{\alpha }{}_{,\nu }\surd \left(s/n\right),$$E6

where t_α,ν is α-critical value of Student’s t-distribution for ν = n – 1 degrees of freedom.

The estimate can be made more accurate if additional information is available (e.g. estimates of m₀ and s₀ from previous measurements or records). If the number n₀ of these values is known, and if the assumption can be made that all samples (new and old) belong to the same population, the following procedure may be used. The updated average is calculated as the weighted average of both sample averages:

$$m_u=(nm+n_0{m}_0)/n_u;n_u=n+n_0,$$E7

where n_u is the updated number of values. The updated standard deviation is

$$s_u=\sqrt{\frac{(n-1)s^2+(n_0-1)s_0{}^2+n{m}^2+n_0{m}_0{}^2-n_u{m}_u{}^2}{n_u-1}}.$$E8

Then, the updated confidence interval for μ can be calculated with m, s, and n in (9) replaced by the updated values m_u, s_u, and n_u. If n₀ is unknown, the literature on Bayesian methods recommends an approximate formula [1, 5, 6]:

$$n_0=s^2/s_0{}^2,$$E9

based on the idea that m₀ and s₀ carry information corresponding to a fictitious sample of certain size n₀. The smaller the scatter s₀² compared to s², the more important are the original results and the larger is the size of the fictitious sample. (An important condition for this estimate is that the ”a priori“ values of m₀ and s₀ were obtained from large samples.)

Quantiles of normal distribution

The ISO 12491 standard ”Statistical methods for quality control of building materials and components“ recommends the following formula for the Bayesian estimate of p-quantile of normal variable x:

$$x_{p,B}=m_u+t_p{s}_u\surd \left(1+1/n_u\right),$$E10

where t_p = t_p(α, p, ν_u) is p-quantile of Student’s t-distribution for ν_u = n_u – 1 degrees of freedom. If no additional information is available, the standard recommends the original values m, s, and n.

Weibull distribution

Some quantities, such as the strength or time to failure due to fatigue, can often be approximated by Weibull distribution:

$$\left(x\right)=1–\text{ exp}\left\{–{\left[\left(x–x_0\right)/a\right]}^b\right\}.$$E11

The parameters a, b, and x₀ are determined from tests. (The threshold value x₀ is often assumed equal zero.) Sometimes, the number of tests is too low for obtaining reliable values of all parameters. Fortunately, the investigated component or structure is often not a quite new solution but rather an improvement of the current conception. In such case, one can expect that the failure mechanism will be similar as in the previous construction. As the parameter b is closely related to the character of failures, one can assume that the value b will be approximately the same as for the previous components and use it as a known constant. Under this assumption, the finding of the remaining parameters a and x₀ from small amount values is more reliable. This approach is called ”Weibayes“ [7]. The assumed value b is more reliable if it was determined from many tests. It is thus suitable to keep records from all tests – for possible use in the future!

4. Software for Bayesian methods

The problems from the above first three examples can be solved easily using Excel and standard Bayesian notation. Some simple programs can be found in the literature, for example [2, 3]. At ETH Zürich, a PC program Combinfo was created, which enables the combination of data from various sources [8], including vague information, such as probability estimates by experts or by judgment. The program allows assigning various weights to individual information. Bayesian methods are also incorporated into software packages for reliability analysis, such as www.reliability.com, www.weibull.com, www.reliasoft.com, or www.itemsoft.com.