An important subject in mathematical science that causes new improvements in data analysis is sequential analysis. In this type of analysis, the number of required observations is not fixed in advance, but is a variable and depends upon the values of the gathered observation. In sequential analysis, at any stage of data gathering process, to determine the number of required observations at the next stage, we analyze the data at hand and with respect to the obtained results, we determine how many more observations are necessary. In this way, the process of data gathering is cheaper and the information is used more effectively. In other words, the data gathering process in sequential analysis, in contrast to frequency analysis, is on-line. This idea caused some researches to conduct researches in various statistical aspects (Basseville and Nikiforov).
In this chapter, using the concept of the sequential analysis approach, we develop an innovative Bayesian method designed specifically for the best solution in selection problem. The proposed method adopts the optimization concept of Bayesian inference and the uncertainty of the decision-making method in dynamic programming environment. The proposed algorithm is capable of taking into consideration the quality attributes of uncertain values in determining the optimal solution. Some authors have applied sequential analysis inference in combination with optimal stopping problem to maximize the probability of making correct decision. One of these researches is a new approach in probability distribution fitting of a given statistical data that Eshragh and Modarres  named it Decision on Belief (DOB). In this decision-making method, a sequential analysis approach is employed to find the best underlying probability distribution of the observed data. Moreover, Monfared and Ranaeifar  and Eshragh and Niaki  applied the DOB concept as a decision-making tool in some problems.
Since the idea behind the sequential analysis modeling is completely similar to the decision-making process of a human being in his life, it may perform better than available methods in decision-making problems. In these problems, when we want to make a decision, first we divide all of the probable solution space into smaller subspaces (the solution is one of the subspaces). Then based on our experiences, we assign a probability measure (belief) to each subspace, and finally we update the beliefs and make the decision.
2. An application to determine the best binomial distribution
In the best population selection problem, a similar decision-making process exits. First, the decision space can be divided into several subspaces (one for each population); second, the solution of the problem is one of the subspaces (the best population). Finally, we can assign a belief to each subspace where the belief denotes the performance of the population in term of its parameter. Based upon the updated beliefs in iterations of the data gathering process, we may decide which population possesses the best parameter value.
Consider n independent populations , where for each index , population is characterized by the value of its parameter of interest . Let denote the ordered value of the parameters . If we assume that the exact pairing between the ordered and the unordered parameter is unknown, then, a population with is called the best population.
There are many applications for the best population selection problem. As one application in supply chain environments, one needs to select the supplier among candidates that performs the best in terms of the quality of its products. As another example, in statistical analysis, we need to select a distribution among candidates that fits the collected observations the most. Selecting a production process that is in out-of-control state, selecting the stochastically optimum point of a multi-response problem, etc. are just a few of these applications.
The problem of selecting the best population was studied in papers by Bechhofer and Kulkarni  using the indifference zone approach and by Gupta and Panchapakesan  employing the best subset selection approach.
2.1. Belief and the approach of its improvement
Assume that there are n available Binomial populations and we intend to select the one with the highest probability of success. Furthermore, in each stage of the data gathering process and for each population, we take an independent sample of size m. Let us define and to be the observed number of successes and failures of the ith Binomial population in the tth stage (sample) and and to be the cumulative observed number of successes and failures of the ith Binomial population up to the kth stage (sample) respectively. In other words, . Then, in the kth stage defining to be the estimated probability of success of the ith population obtained by , referring to Jeffrey’s prior (Nair et al.), for , we take a Beta prior distribution with parameters αi,0=0.5 and βi,0=0.5. Then, using Bayesian inference, we can easily show that the posterior probability density function of is
At stage k of the data gathering process, after taking a sample and observing the numbers of failures and successes, we update the probability distribution function of for each population. To do this, define as a probability measure (called belief) of the ith population to be the best one given and as
We then update the beliefs based on the values of for each population in iteration k. If we define as the prior belief for each population, in order to update the posterior belief , since we may assume that the data are taken independently in each stage, we will have
From equation (3) we see that to update the beliefs, we need to evaluate in each decision-making stage. One way to do this is to use
Then, the probability given in equation (3) will increase when a better population is selected. In the next theorem, we will prove that when the number of decision-making stages goes to infinity this probability converges to one for the best population.
If the ith population is the best, then .
In order to prove the theorem first we prove the following two lemmas.
Define a recursive sequence as
where are different positive constants, , and Then, if , there exist at most one non-zero .
Suppose there are two nonzero . Taking the limit on as k goes to infinity we have
Now since , then by equation (6) we have
In other words, we conclude , which is a contradiction.
Sequence converges to one for and converges to zero for , where g is an index for the maximum value of .
That is a contradiction because . So
Now we are ready to prove the convergence property of the proposed method. Taking limit on both sides of equation (3), we will have
From the law of large numbers, we know that , where is the probability of success of the jth population. Hence, using equation (7) we have . Then assuming population i is the best, i.e., it possesses the largest value of ’s, by lemma 1 and 2 we conclude that and . This concludes the convergence property of the proposed method.
In real-world applications, since there is a cost associated with the data gathering process we need to select the best population in a finite number of decision-making stages. In the next section, we present the proposed decision-making method in the form of a stochastic dynamic programming model in which there is a limited number of decision-making stages available to select the best population.
2.2. A dynamic programming approach
The proposed dynamic programming approach to model the decision-making problem of selecting the best Binomial population is similar to an optimal stopping problem.
Let us assume that to find the best population there is a limited number of stages (s) available. Then, the general framework of the decision-making process in each stage is proposed as:
Take an independent sample of size m from each population.
Calculate the posterior beliefs in terms of the prior beliefs using Bayesian approach.
Select the two biggest beliefs.
Based upon the values of the two biggest beliefs calculate the minimum acceptable belief.
If the maximum belief is more than the minimum acceptable belief, then we can conclude that the corresponding subspace is the optimal one. Otherwise, go to step 1.
In step 3 of the above framework, let populations i and j be the two candidates of being the best populations (it means that the beliefs of populations i and j are the two biggest beliefs) and we have s decision-making stages. If the biggest belief is more than a threshold (minimum acceptable belief), we select the corresponding subspace of that belief as the solution. Otherwise, the decision-making process continues by taking more observations. We determine the value of such that the belief of making the correct decision is maximized. To do this suppose that for each population a new observation, , is available at a given stage k. At this stage, we define to be the expected belief of making the correct decision in s stages when two populations i and j are the candidates for the optimal population. In other words, if we let CS denote the event of making the correct decision, we define , where is the belief of making the correct decision. Furthermore, assume that the maximum of occurs at . Then, we will have
We denote this optimal point by . In other words, . Moreover, let us define and to be the state of selecting population i and j as the candidates for the optimal population, respectively, and as the state of choosing neither of these population. Then, by conditioning on the above states, we have
In order to evaluate, in what follows we will find the belief terms of equation (11).
These are the beliefs of making the correct decision if population i or j is selected as the optimal population, respectively. To make the evaluation easier, we denote these beliefs by and . Then, using equation (2) we have
These are the beliefs of selecting population i or j as the optimal population, respectively. Regarding the decision-making strategy, we have:
Hence, we define event as
Since and that the beliefs are not negative we conclude . Furthermore, since the decision making is performed based upon the maximum value of the beliefs, without interruption of assumptions, we can change the variation interval of from [0,1] to [0.5,1]. Now by considering implicitly, we have . By similar reasoning . Hence
In which, .
To evaluate in equation (16), let and to be the probability distributions of and , respectively. Then,
By change of variables technique, we have:
For we have
Having all the belief terms of equation (11) evaluated in equations (12), (13), (14), (15), and (16), and knowing that by partitioning the state space we have , equation (11) can now be evaluated by substituting.
2.2.1. Making the decision
Assuming that for the two biggest beliefs we have , equation (23) can be written as
For the decision-making problem at hand, three cases may happen
In this case, both and are negative. Since we are maximizing , then the two probability terms in equation (24) must be minimized. This only happens when we let , making the probability terms equal to zero. Now since , we continue to the next stage.
In this case, and are both positive and to maximize we need the two probability terms in equation (24) to be maximized. This only happens when we let . Since , we select population i as the optimal subspace.
In this case, one of the probability terms in equation (24) has positive coefficient and the other has negative coefficient. In this case, in order to maximize we take the derivative as follows.
Thus following is obtained,
For determining , first using an approximation, we assume that is a constant number equal to its mean, then we have:
Also is obtained as follows,
Now following can be resulted,
Now the approximate value of say is determined.
Second using another approximation, we assume that is a constant number equal to its mean thus with similar reasoning, following is obtained:
Therefore the approximate optimal value of can be determined from following equation,
3. An application for fault detection and diagnosis in multivariate statistical quality control environments
In this section, a heuristic threshold policy is applied in phase II of a control charting procedure to not only detect the states of a multivariate quality control system, but also to diagnose the quality characteristic(s) responsible for an out-of-control signal. It is assumed that the in-control mean vector and in-control covariance matrix of the process have been obtained in phase I.
In a multivariate quality control environment, suppose there are correlated quality characteristics whose means are being monitored simultaneously. Further, assume there is only one observation on the quality characteristics at each iteration of the data gathering process, where the goal is to detect the variable with the maximum mean shift. Let be the observation of the ith quality characteristic, , at iteration , , and define the observation vector and observation matrix . After taking a new observation, , define , the probability of variable to be in an out-of-control state, as
where stands for out-of-control. This probability has been called the belief of variable to be in out-of-control condition given the observation matrix up to iteration and the observation vector obtained at iteration .
Assuming the observations are taken independently at each iteration, to improve the belief of the process being in an out-of-control state at the kth iteration, based on the observation matrix and the new observation vector , we have
Then, using the Bayesian rule the posterior belief is:
Since the goal is to detect the variable with the maximum mean shift, only one quality characteristic can be considered out-of-control at each iteration. In this way, there are remaining candidates for which quality characteristics are in-control. Hence, one can say that the candidates are mutually exclusive and collectively exhaustive. Therefore, using the Bayes' theorem, one can write equation (34) as
When the system is in-control, we assume the characteristics follow a multinormal distribution with mean vector and covariance matrix
In out-of-control situations, only the mean vector changes and the probability distribution along with the covariance matrix remain unchanged. In latter case, equation (35) is used to calculate the probability of shifts in the process mean at different iterations. Moreover, in order to update the beliefs at iteration one needs to evaluate .
The term is the probability of observing if only the ith quality characteristic is out-of-control. The exact value of this probability can be determined using the multivariate normal density, , where denotes the mean vector in which only the ith characteristic has shifted to an out-of-control condition and is a known constant.
Since the exact value of the out-of-control mean vector is not known a priori, two approximations are used in this research to determine . Note that we do not want to determine the exact probability. Instead, the aim is to have an approximate probability (a belief) on each characteristic being out-of-control. In the first approximation method, define to be the event that all characteristics are in-control, and let be the conditional probability of observing given all characteristics are in-control. Further, let in the aforementioned multivariate normal density, so that can be approximately evaluated using , where . Note that this evaluation is proportional to , and since it is assumed that characteristic i is under control, no matter the condition of the other characteristics, this approximation is justifiable.
In the second approximation method, we assume . Although it is obvious that is not equal to , since we only need a belief function to evaluate and also we do not know the exact value of out-of-control mean vector, this approximation is just used to determine. Moreover, it can be easily seen that the closer the value of the ith characteristic is to its in-control mean the smaller is as expected. We thus let
where is a sufficiently big constant number to ensure the above definition is less than one.
The approximation to in equation (37) has the following two properties:
It does not require the value of out-of-control means to be known.
The determination of a threshold for the decision-making process (derived later) will be easier.
Niaki and Fallahnezhad  defined another equation for the above conditional probability and showed that if a shift occurs in the mean of variable , then. They proposed a novel method of detection and classification and used simulation to compare its performances with that of existing methods in terms of the average run length for different mean shifts. The results of the simulation study were in favor of their proposed method in almost all shift scenarios. Besides using a different equation, the main difference between the current research and Niaki and Fallahnezhad  is that the current work develops a novel heuristic threshold policy, in which to save sampling cost and time or when these factors are constrained, the number of the data gathering stages is limited.
3.3. The proposed procedure
Assuming a limited number of the data gathering stages, , to detect and diagnose characteristic(s), a heuristic threshold policy-based model is developed in this Section. The framework of the proposed decision-making process follows.
Define as the set of indices for the characteristics, all of which having the potential of being out-of-control.
Using the maximum entropy principle, initialize as the prior belief of the ith variable to be out-of-control. In other words, at the start of the decision-making process all variables have an equal chance of being out-of-control. Set the discount rate, the maximum probability of correct selection when decision making stages remains, and the maximum number of decision making stages.
Obtain an observation of the process.
Estimate the posterior beliefs, (for), using equation (35).
Obtain the order statistics on the posterior beliefs such that
Furthermore, let and .
Assume the variables with the indices and are the candidates of being out-of-control, where decision-making steps are available. Define the probability of correct choice between the variables and , where is the acceptable belief. Also, define the event of correct selection and event the existence of two out-of-control candidate variables and . Then, we have:
where " " means "defined as."
Assuming the maximum point of , called the minimum acceptable belief, we have
Let and be the event of selecting and as the out-of-control variables, respectively, and be the event of not selecting any. Then, by conditioning on the probability, we have:
At the kth iteration, the conditional bi-variate distribution of the sample means for variables and , i.e, , is determined using the conditional property of multivariate normal distribution given in appendix 1. Moreover, knowing and evaluating the conditional mean and standard deviation (see appendix 1) results in
Based on the decomposition method of Mason et al. , define statistics as
Thus, when the process is in-control, the statistics follow a standard normal distribution .
Now, let denote the probability of correct selection conditioned on selecting as the out-of-control variable. Hence,
Then, the probability measure is calculated using the following equation,
The probability measure is defined as the probability of selecting variable to be out-of-control. Regarding to the explained strategy, we have:
Since and the value of beliefs are not negative, we conclude
Without interruption of assumptions, we can change the variation interval of from [0,1] to [0.5,1]. Hence,
By similar reasoning, we have:
The term denotes the probability of correct selection conditioned on excluding the candidates and as the solution. In other words, the maximum belief has been less than the threshold (minimum acceptable belief) and the decision making process continues to the next stage. In terms of stochastic dynamic programming approach, the probability of this event is equal to the maximum probability of correct selection when there are stages remaining. The discounted value of this probability in the current stage using the discount factor equals to . Further, since we partitioned the decision space into events , we have:
Now we evaluate as follows,
In other words,
The method of evaluating the minimum acceptable belief is given in Appendix 2.
Step VIII: The Decision Step
If the belief in the candidate set is equal to or greater than then choose the variable with index to be out-of-control. In this case, the decision-making process ends. Otherwise, without having any selection at this stage, obtain another observation, lower the number of remaining decision-stages to , set , and return to step V above. The process will continue until either the stopping condition is reached or the number of stages is finished. The optimal strategy with decision-making stages that maximizes the probability of correct selection would be resulted from this process.
In what follows, the procedure to evaluate of equation (53) is given in detail.
3.4. Method of evaluating
Using as the minimum acceptable belief, from equation (53) we have
Then, for the decision-making problem at hand, three cases may occur
In this case, both and are negative. Since we are maximizing , the two probability terms in equation (54) must be minimized. This only happens when , making the probability terms equal to zero. In other words, since , we continue to the next stage.
In this case, and are both positive and to maximize we need the two probability terms in equation (54) to be maximized. This only happens when . In other words, since , the variable with the index is selected.
In this case, one of the probability terms in equation (54) has a positive and the other a negative coefficient. Then, in order to maximize , the first derivative on must be equated to zero. To do this, define and as follows:
We first present the method of evaluating as follows.
Then, the method of evaluating probability terms in equation (57) is given in appendix 2.
With similar reasoning, we have,
The method of determining the minimum acceptable belief is given in appendix 2.
4. An application for fault detection in uni-variate statistical quality control environments
In a uni-variate quality control environment, if we limit ourselves to apply a control charting method, most of the information obtained from data behavior will be ignored. The main aim of a control charting method is to detect quickly undesired faults in the process. However, we may calculate the belief for the process being out-of-control applying Bayesian rule at any iteration in which some observations on the quality characteristic are gathered. Regarding these beliefs and a stopping rule, we may find and specify a control threshold for these beliefs and when the updated belief in any iteration is more than this threshold, an out-of-control signal is observed.
In Decision on Beliefs, first, all probable solution spaces will be divided into several candidates (the solution is one of the candidates), then a belief will be assigned to each candidate considering our experiences and finally, the beliefs are updated and the optimal decision is selected based on the current situation. In a SPC problem, a similar decision-making process exits. First, the decision space can be divided into two candidates; an in-control or out-of-control production process. Second, the problem solution is one of the candidates (in-control or out-of-control process). Finally, a belief is assigned to each candidate so that the belief shows the probability of being a fault in the process. Based upon the updated belief, we may decide about states of the process (in-control or out-of-control process).
4.1. Learning — The beliefs and approach for its improvement
For simplicity, individual observation on the quality characteristic of interest in any iteration of data gathering process was gathered. At iteration k of data gathering process, was defined as the observation vector where resemble observations for previous iterations 1, 2, …, k. After taking a new observation, Ok-1 the belief of being in an out-of-control state is defined as . At this iteration, we want to update the belief of being in out-of-control state based on observation vector and new observation . If we define as the prior belief of an out-of-control state, in order to update the posterior belief , since we may assume that the observations are taken independently in any iteration, then we will have
With this feature, the updated belief is obtained using Bayesian rule:
Since in-control or out-of-control state partition the decision space, we can write equation (60) as
Assuming the quality characteristic of interest follows a normal distribution with mean μ and variance σ2, we use equation (61) to calculate both beliefs for occurring positive or negative shifts in the process mean μ.
Positive shifts in the process mean
The values of , showing the probability of occurring a positive shift in the process mean, will be calculated applying equation (61) recursively. is defined by the following equation,
For positive shift, the probability of being a positive shift in the process at iteration k, , is calculated using equation (63).
where is the cumulative probability distribution function for the normal distribution with mean μ and variance σ2. Above probabilities are not exact probabilities and they are a kind of belief function to ascertain good properties for
Therefore is determined by the following equation,
Negative shifts in the process mean
The values of denotes the probability of being a negative shift in the process mean that is calculated using equation (61) recursively. In this case, is defined by the following equation,
Also is calculated using equation (66).
Thus is determined by the following equation,
4.2. A decision on beliefs approach
We present a decision making approach in terms of Stochastic Dynamic Programming approach. Presented approach is like an optimal stopping problem.
Suppose n stages for decision making is remained and two decisions are available.
A positive shift is occurred in the process mean
No positive shift is occurred in the process mean
Decision making framework is as follows:
Gather a new observation.
Calculate the posterior Beliefs in terms of prior Beliefs.
Order the current Beliefs as an ascending form and choose the maximum.
Determine the value of the minimum acceptable belief (is the minimum acceptable belief for detecting the positive shift and is the least acceptable belief for detecting the negative shift)
If the maximum Belief was more than the minimum acceptable belief, , select the belief candidate with maximum value as a solution else go to step 1.
In terms of above algorithm, the belief with maximum value is chosen and if this belief was more than a control threshold like , the candidate of that Belief will be selected as optimal candidate else the sampling process is continued. The objective of this model is to determine the optimal values of . The result of this process is the optimal strategy with n decision making stages that maximize the probability of correct selection.
Suppose new observation is gathered. (k is the number of gathered observations so far). is defined as the probability of correct selection when n decision making stages are remained and we follow strategy explained above also denotes the maximum value of thus,
CS is defined as the event of correct selection. S1 is defined as selecting the out-of-control condition (positive shift) as an optimal solution and S2 is defined as selecting the in-control condition as an optimal decision and NS is defined as not selecting any candidate in this stage.
Hence, using the total probability law, it is concluded that:
denotes the probability of correct selection when candidate S1 is selected as the optimal candidate and this probability equals to its belief, , and with the same discussion, it is concluded that
is the probability of selecting out of control candidate (positive shift) as the solution thus following the decision making strategy, we should have and that is equivalent to following,
With the same reasoning, it is concluded that,
denotes the probability of correct selection when none of candidates has been selected and it means that the maximum value of the beliefs is less than and the process of decision making continues to latter stage. As a result, in terms of Dynamic Programming Approach, the probability of this event equals to maximum of probability of correct selection in latter stage (n-1), , but since taking observations has cost, then the value of this probability in current time is less than its actual value and by using the discounting factor α, it equals
Since the entire solution space is partitioned, it is concluded that
By the above preliminaries, the function is determined as follows:
In terms of above equation, is obtained as follows:
Calculation method for :
and are defined as follows:
Now equation (73) is rewritten as follows:
There are three conditions:
In this condition, both and are negative, thus we should have in order to maximize . Since , we don’t select any candidate in this condition and sampling process continues.
In this condition, both and are positive, thus we should have in order to maximize . since , we select the candidate of belief as the solution.
In this condition, one of the probabilities in equation (10) has positive coefficient and one has negative coefficient, to maximize , optimality methods should be applied.
Definition: is defined as follows:
First the value of is determined as follows:
Since is a cumulative distribution function thus it follows a uniform distribution function in interval [0, 1], thus the above equality is concluded.
With the same reasoning, it is concluded that:
Now equation (73) can be written as follows:
And equation (79) can be written as follows:
Since thus it is sufficient to maximize the real value function , therefore; we should find the function value in points where first derivative is equated to zero as follows,
The optimal threshold is determined by the above equation. Since the optimal value of should be in the interval [0.5, 1] thus it is concluded that the optimal value of will be determined as follows:
The above method is presented for detecting the positive shifts in the process mean and can be adapted for detecting the negative shifts with the same reasoning.
The general decision making algorithm is summarized as follows:
Set k=0 and the initial beliefs .
Gather an observation and set .
If , then no shift is occurred in the process mean and decision making stops.
Update the values for the beliefs by equation (61).
If , then if , it is concluded that a positive shift is occurred in the process mean and decision making stops, also if , then no positive shift is occurred in the process mean and decision making stops.
If , then data is not sufficient for detecting the positive shift and go to stage 2 after checking the occurrence of negative shift in the rest of the algorithm.
If then if it is concluded that a negative shift is occurred the process mean and decision making stops and if , then no negative shift is occurred in the process mean and decision making stops.
If , then data is not sufficient for detecting the negative shift and go to stage 2.
If , then determine the value of (minimum acceptable belief for detecting the positive shift) by the following equation:
If , then determine the value of (minimum acceptable belief for detecting the negative shift) by the following equation:
If , then a positive shift is occurred and decision making stops, and if , then no positive shift is occurred and decision making stops, else go to stage 2 after checking the occurrence of negative shift in rest of the algorithm.
If , then a negative shift is occurred and decision making stops, and If , then no negative shift is occurred and decision making stops, else go to stage 2.
The approximate value of based on the discount factor in the stochastic dynamic programming approach is .
In this chapter, we introduced a new approach to determine the best solution out of m candidates. To do this, first, we defined the belief of selecting the best solution and explained how to model the problem by the Bayesian analysis approach. Second, we clarified the approach by which we improved the beliefs, and proved that it converges to detect the best solution. Next, we proposed a decision-making strategy using dynamic programming approach in which there were a limited number of decision-making stages.
Conditional Mean and Variance of the Variables
Conditional mean of variables and can be evaluated using the following equation.
: The covariance matrix of the process
: Submatrix of the covariance matrix for variables
: Submatrix of the covariance matrix between variables and
: Submatrix of the covariance matrix for variables
Further, the conditional covariance matrix of variables on variables , is obtained as .
Evaluating the Optimal Value of
Assume . Then,
Now since follow a standard normal distribution , hence and follow a distribution with one degree of freedom. Then using an approximation, if we assume that is approximately equal to its mean, we have
Now, since , we have
Replacing the above equations in equation (53) results in
Now by solving the equation , the following equation is obtained.
Finally, the approximate value of say is determined by solving this equation numerically or by a search algorithm.
Now using another approximation, if we assume that is approximately equal to its mean, the approximate value of say is determined by solving following equation,
The approximate optimal value of is obtained as follows,