Mean squared error for

## Abstract

In this chapter, Erlang distribution is considered. For parameter estimation, maximum likelihood method of estimation, method of moments and Bayesian method of estimation are applied. In Bayesian methodology, different prior distributions are employed under various loss functions to estimate the rate parameter of Erlang distribution. At the end the simulation study is conducted in R-Software to compare these methods by using mean square error with varying sample sizes. Also the real life applications are examined in order to compare the behavior of the data sets in the parametric estimation. The comparison is also done among the different loss functions.

### Keywords

- Erlang distribution
- prior distributions
- loss functions
- simulation study
- applications

## 1. Introduction

Erlang distribution is a continuous probability distribution with wide applicability, primarily due to its relation to the exponential and gamma distributions. The Erlang distribution was developed by Erlang [1] to examine the number of telephone calls that could be made at the same time to switching station operators. This distribution can be expressed as waiting time and message length in telephone traffic. If the duration of individual calls are exponentially distributed then the duration of succession of calls is the Erlang distribution. The Erlang variate becomes gamma variate when its shape parameter is an integer (for details see Evans et al. [2]). Bhattacharyya and Singh [3] obtained Bayes estimator for the Erlangian queue under two prior densities. Haq and Dey [4] addressed the problem of Bayesian estimation of parameters for the Erlang distribution assuming different independent informative priors. Suri et al. [5] used Erlang distribution to design a simulator for time estimation of project management process. Damodaran et al. [6] obtained the expected time between failure measures. Further, they showed that the predicted failure times are closer to the actual failure times. Jodra [7] showed the procedure of computing the asymptotic expansion of the median of Erlang distribution.

The probability density function of an Erlang variate is given by

where *k* an integer number.

### 1.1 Graphical representation of pdf for Erlang distribution

In this chapter, Erlang distribution is considered. Some structural properties of Erlang distribution have been obtained. The parameter estimation of Erlang distribution is obtained by employing the maximum likelihood method of estimation, method of moments and Bayesian method of estimation in different sections of this chapter. In Bayesian approach, the parameters are estimated by using Jeffrey’s and Quasi priors under different loss functions (Figure 1).

### 1.2 Relationship of Erlang distribution with other distributions

The gamma distribution is a generalized form of the Erlang distribution.

If the shape parameter k is 1, then Erlang distribution reduces to exponential distribution.

If the scale parameter is 2, then Erlang distribution reduces to Chi-square distribution with 2 degrees of freedom.

Thus from the above descriptions, we can say that exponential distribution and Chi-square distribution are the sub-models of Erlang distribution.

## 2. Methods used for parameter estimation

In this chapter, we have used different approaches for parameter estimation. The first two methods come under the classical approach which was founded by Fisher in a series of fundamental papers round about 1930.

The alternative approach is the Bayesian approach which was first discovered by Reverend Thomas Bayes. In this chapter, we have used two different priors for the parameter estimation. Also three loss functions are used which are discussed in their respective sections. A number of symmetric and asymmetric loss functions used by various researchers; see Zellner [8], Ahmad and Ahmad [9], Ahmad et al. [10], etc.

These methods of estimations are elaborated in their respective sections accordingly.

### 2.1 Maximum likelihood (MLH) estimation

The most general method of estimation is known as maximum likelihood (MLH) estimators, which was initially formulated by Gauss. Fisher in the early 1920 firstly introduced MLH as general method of estimation and later on developed by him in a series of papers. He revealed the advantages of this method by showing that it yields sufficient estimators, which are asymptotically MVUES. Thus the important feature of this method is that we look at the value of the random sample and then select our estimate of the unknown population parameter, the value of which the probability of getting the observed data is maximum.

Suppose the observed data sample values are

Similarly the definition applies when the random sample comes from a continuous population but in that case

Since the principle of maximum likelihood consists in finding an estimator of the parameter which maximizes the likelihood function for variation in the parameter. Thus if there exists a function

Since

a form which is more convenient from practical point of view.

The MLH estimation of the rate parameter of Erlang distribution is obtained in the following theorem:

**Theorem 2.1:** Let

**Proof:** The likelihood function of random sample of size n having Erlang density function Eq. (1) is given by

The log likelihood function is given by

Differentiating Eq. (3) w.r.t.

### 2.2 Method of moments (MM)

One of the simplest and oldest methods of estimation is the method of moments. The method of moments was discovered by Karl Pearson in 1894. It is a method of estimation of population parameters such as mean, variance, etc. (which need not be moments), by equating sample moments with unobservable population moments and then solving those equations for the quantities to be estimated. The method of moments is special case when we need to estimate some known function of finite number of unknown moments.

Suppose

In general

e.g.,

where

Then by the method of moments

The MM estimation of the rate parameter of Erlang distribution is obtained in the following theorem:

**Theorem 2.2:** Let

**Proof:** If the numbers

where

The rth moment of two parameter Erlang distribution about origin is given by

Using Eq. (5) in Eq. (6), we have

If r = 1 in Eq. (7), we get

If r = 2, then Eq. (7) becomes

Thus the variance is given by

When we divide *k* only and is given by

On taking the square roots of Eq. (8), we have the coefficient of variation

The rate parameter

If r = 1, in Eq. (5), then

Also if r = 1 in Eq. (7), then

Thus,

## 3. Bayesian method of estimation

Nowadays, the Bayesian school of thought is garnering more attention and at an increasing rate. This thought of statistics was given by Reverend Thomas Bayes. He first discovered the theorem that now bears his name. It was written up in a paper “An Essay Towards Solving a Problem in the Doctrine of Chances.” This paper was found after his death by his friend Richard Price, who had it published posthumously in the Philosophical Transactions of the Royal Society in 1763. Bayes showed how inverse probability could be used to calculate probability of antecedent events from the occurrence of the consequent event. His methods were adopted by Laplace and other scientists in the nineteenth century. By mid twentieth century interest in Bayesian methods was renewed by De Finetti, Jeffreys and Lindley, among others. They developed a complete method of statistical inference based on Bayes’ theorem. Bayesian analysis is to be used by practitioners for situations where scientists have a priori information about the values of the parameters to be estimated. In everyday life, uncertainty often permeates our choices, and when choices need to be made, past experience frequently proves a helpful aid. Bayesian theory provides a general and consistent framework for dealing with uncertainty.

Within Bayesian inference, there are also different interpretations of probability, and different approaches based on those interpretations. Early efforts to make Bayesian methods accessible for data analysis were made by Raiffa and Schlaifer [11], DeGroot [12], Zellner [13], and Box and Tiao [14]. The most popular interpretations and approaches are objective Bayesian inference and subjective Bayesian inference. Excellent expositions of these approaches are with Bayes and Price [15], Laplace [16], Jeffrey’s [17], Anscombe and Aumann [18], Berger [19, 20], Gelman et al. [21], Leonard and Hsu [22], De-Finetti [23]. Modern Bayesian data analysis and methods based on Markov chain Monte Carlo methods are presented in Bernardo and Smith [24], Robert [25], Gelman et al. [26], Marin and Robert [27], Carlin and Louis [28]. Good elementary introductions to the subject are Ibrahim et al. [29], Ghosh [30], Bansal [31], Koch [32], Hoff [33].

In Bayesian statistics probability is not defined as a frequency of occurrence but as the plausibility that a proposition is true, given the available information. The parameters are treated as random variables. The rules of probability are used directly to make inferences about the parameters. Probability statements about parameters must be interpreted as “degree of belief.” We revise our beliefs about parameters after getting the data by using Bayes’ theorem. This gives our posterior distribution which gives the relative weights we give to each parameter value after analyzing the data. The posterior distribution comes from two sources: the prior distribution and the observed data. This means that the inference is based on the actual occurring data, not all possible data sets that might have occurred.

In this section, posterior distribution of Erlang distribution is obtained by using Jeffrey’s prior and Quasi prior. The rate parameter of Erlang distribution is estimated with the help of different loss functions. For parameter estimation we have used the approach as is used by Ahmad et al. [34], Ahmad et al. [35], etc. Some important prior distributions and loss functions which we have used in this article are given as below:

### 3.1 Prior distributions used

Prior distribution is the basic part of Bayesian analysis which represents all that is known or assumed about the parameter. Usually the prior information is subjective and is based on a person’s own experience and judgment, a statement of one’s degree of belief regarding the parameter.

Another important feature of the Bayesian analysis is the choice of the prior distribution. If the data have sufficient signal, even a bad prior will still not greatly influence the posterior. We can examine the impact of prior by observing the stability of posterior distribution related to different choices of priors. If the posterior distribution is highly dependent on the prior, then the data (the likelihood function) may not contain sufficient information. However, if the posterior is relatively stable over a choice of priors, then the data indeed contains significant information.

Prior distribution may be categorical in different ways. One common classification is a dichotomy that separated “proper” and “improper” priors.

A prior distribution is proper if it does not depend on the data and the value of integral

In this chapter, we have used two different priors Jeffrey’s prior and Quasi prior which are given below:

#### 3.1.1 Jeffrey’s prior

An invariant form for the prior probability in estimation problems is given by Jeffery’s [36].

The general formula of the Jeffreys prior, which is defined by

where

In this situation, the Jeffreys prior is given by

Jeffrey suggested a thumb rule for specifying non-informative prior for parameter

Rule 1: if

Rule 2: if

Under linear transformation, rule 1 is invariant and under any power transformation of

#### 3.1.2 Quasi prior

When there is no more information about the distribution parameter, one may use the Quasi density as given by

### 3.2 Loss functions used

The concept of loss function is old as Laplace and was reintroduced in statistics by Abraham Wald [37]. In statistics, typically a loss function is used for parameter estimation, and the event in question is some function of the difference between estimated and true values for an occurrence of data. In the context of economics, loss function is usually economic cost. In optimal control, the loss is the penalty for failing to achieve a desired value.

The word “loss” is used in place of “error” and the loss function is used as a measure of the error or loss. Loss function is a measure of the error and presumably would be greater for large error than for small error. We would want the loss to be small or we want the estimate to be close to what it is estimating. Loss depends on sample and we cannot hope to make the loss small for every possible sample but can try to make the loss small on the average. Our objective is to select an estimator that makes this error or loss small, also which makes the average loss (risk) small and ideally select an estimator that has the small risk.

In this chapter, we have used three different Loss Functions which are as under:

#### 3.2.1 Precautionary loss function (PLF)

The concept of precautionary loss function (PLF) was introduced by Norstrom [38]. He introduced an alternative asymmetric loss function and also presented a general class of precautionary loss functions as a special case. These loss functions approach infinitely near the origin to prevent underestimation, thus giving conservative estimators, especially when low failure rates are being estimated. These estimators are very useful when underestimation may lead to serious consequences. A very useful and simple asymmetric precautionary loss function (PLF) is

#### 3.2.2 Al-Bayyati’s loss function (ALF)

The loss function proposed by Al-Bayyati [39] is an asymmetric loss function and is given by

where

#### 3.2.3 LINEX loss function (LLF)

The idea of LINEX loss function (LLF) was founded by Klebanov [40] and used by Varian [41] in the context of real estate assessment. The formula of LLF is given by

where

### 3.3 Posterior density under Jeffrey’s prior

Let

and the likelihood function Eq. (2) given as below

Jeffreys’ non-informative prior for

where

On solving the above expression, we have

By using the Bayes theorem, we have

Using Eqs. (2) and (10) in Eq. (11), we get

where *ρ* is independent of *λ* and

Using the value of *ρ* in Eq. (12)

### 3.4 Posterior density under Quasi prior

Let

The Quasi for

By using the Bayes theorem, we have

Using Eqs. (2) and (14) in Eq. (15), we have

where

By using the value of

## 4. Estimation of parameters under Jeffrey’s prior

In this section, parameter estimation of Erlang distribution is done by using Jeffreys’ prior under different loss functions. The procedure of calculating the Bayesian estimate is already defined in Section 3. The estimates are obtained in the following theorems:

**Theorem 4.1:** Assuming the loss function

**Proof:** The risk function of the estimator

Using Eq. (13) in Eq. (18), we get

On solving the above expression, we get

Minimization of the risk with respect to

**Theorem 4.2:** Assuming the loss function

**Proof:** The risk function of the estimator

On substituting Eq. (13) in Eq. (20), we have.

On solving the above expression, we get

Minimization of the risk with respect to

**Theorem 4.3:** Assuming the loss function

**Proof:** The risk function of the estimator

Using Eq. (13) in Eq. (22), we have

On solving the above expression, we get

Minimization of the risk with respect to

## 5. Estimation of parameters under Quasi prior

In this section, parameter estimation of Erlang distribution is done by using QUASI prior under different loss functions. The procedure for obtaining the Bayesian estimate is available in Section 3. The estimates of parameter are obtained in the following theorems.

**Theorem 5.1:** Assuming the loss function

**Proof:** The risk function of the estimator

Using Eq. (17) in Eq. (24), we have

On solving the above expression, we get

Minimization of the risk with respect to

**Remark:** Replacing d = 1 in Eq. (25), the same Bayes estimator is obtained as in Eq. (19).

**Theorem 5.2:** Assuming the loss function

**Proof:** The risk function of the estimator

By using Eq. (17) in Eq. (26), we have.

On solving the above expression, we get

Minimization of the risk with respect to

**Remark:** Replacing d = 1 in Eq. (27), the same Bayes estimator is obtained as in Eq. (21).

**Theorem 5.3**: Assuming the loss function

**Proof:** The risk function of the estimator

Using Eq. (17) in Eq. (28), we have.

On solving the above expression, we get

Minimization of the risk with respect to

**Remark:** Replacing d = 1 in Eq. (29), the same Bayes estimator is obtained as in Eq. (23).

## 6. Entropy estimation of Erlang distribution

The concept of entropy was introduced by Claude. Shannon [42] in the paper “A Mathematical theory of Communication.” This concept of Shannon’s entropy is the central role of information theory, sometimes referred as measure of uncertainty. Shannon entropy provides an absolute limit on the best possible lossless encoding or compression of any communication, assuming that the communication may be represented as a sequence of independent and identical distributed random variables. Entropy is typically measured in bits, when the log is to the base 2, and nats, when the log is to the base n.

Shannon’s definition of entropy, when applied to an information source, can determine the minimum channel capacity required to reliably transmit the source as encoded binary digit. The entropy of a random variable is defined in terms of its probability distribution and can be shown to be a good measure of randomness or uncertainty. For deriving the entropy of probability distributions, we need the following two definitions that are more discussed in Cover et al. [43].

**Definition (i):** The entropy of the discrete random variable defined on the probability space is given by

It is obvious that

**Definition (ii):** The entropy of the continuous random variable defined on the real line is given by

In this section, entropy estimation of two parameter Erlang distribution is discussed which given as below.

**Theorem 6.1:** Let

**Proof:** Shannon’s entropy for a continuous random variable is defined as

Using Eq. (1) in Eq. (30), we have

Now

Also

Substitute the value of Eqs. (32) and (33) in Eq. (31), we have.

## 7. AIC and BIC criterion for Erlang distribution

For model selection the approach of Akaike information criterion (AIC) and Bayesian information criterion (BIC) based on entropy estimation are used. The Akaike information criterion (AIC) was introduced by Hirotsugu Akaike [44] and proposed it as a measure of goodness of fit of an estimated statistical model. It is a measure of the relative quality of a statistical model for a given set of data. It has been found in information theory that it offers a relative estimate of the information lost when a given model is used to represent the process that generates the data.

The AIC is not a test of the model in the sense of hypothesis testing; rather it is a test between models—a tool for model selection. Given a data set, several competing models may be ranked according to their AIC, with the one having the lowest AIC being the best.

The formula for AIC is given by

where *K* is the number of parameters and

AICC was first introduced by Hurvich and Tsai [45] and its different derivations were proposed by Burnham and Anderson [46]. AICC is AIC with a correction for finite sample sizes and is given by

Burnham and Anderson [47] strongly recommended that we should use AICC instead of AIC when the sample size is small or if *K* is large. Since AICC converges to AIC as the sample size is getting large. Using AIC, instead of AICC, when the sample size is not many times larger than *K*^{2}, increases the probability of selecting models that have too many parameters, i.e., of over fitting. The probability of AIC over fitting can be substantial, in some cases.

The Bayesian information criterion (BIC) also known as Schwarz Criterion is used as a substitute for full calculation of the Bayes’ factor since it can be calculated without specifying prior distribution. In BIC, the penalty for additional parameters is stronger than that of the AIC.

The formula for the BIC is given by

where *K* is the number of parameters, n is the sample size and

The AIC and BIC of two parameter Erlang distribution are obtained in this section, which are given below.

The Shannon’s entropy of two parameter Erlang distribution is given by

Also

Comparing Eqs. (35) and (36), we have

The AIC and BIC methodology attempts to find the model that best explains the data with a minimum of their values, we have.

or

and

or

## 8. Simulation study of Erlang distribution

We have generated the data for Erlang distribution of different sample sizes (15, 30 and 60) in R Software for each pairs of _{1} = −1, 1) and (a = 0.5, 1.0). The values of extension are (C = 0.5, 1.0). The estimates of rate parameter for each method are calculated. The results are presented in the following tables.

## 9. Comparison of Erlang distribution (ED) with its sub-models

The flexibility and potentiality of the Erlang distribution is compared with its sub models, which is examined by using different criterions like AIC, BIC and AICC with the help of the following illustration.

**Illustration I:**

We provide the compatibility of the Erlang distribution (ED) with their sub-models; Chi square and exponential distributions. For this purpose, we generated the data set for Erlang distribution of large sample size (i.e., 200) in R Software for each pairs of

**Illustration II:**

The data set is taken from Lawless [48]. The observations involves the number of million revolutions between failures for each of 23 ball bearings, the individual bearings were inspected periodically to determine whether “failure” had occurred. Treating the failure times as continuous, the 23 failure times are:

(17.88, 28.92, 33.00, 41.52, 42.12, 45.60, 48.40, 51.84, 51.96, 54.02, 55.56, 67.80, 68.64, 68.64, 68.88, 84.12, 93.12, 98.64, 105.12, 105.84, 127.92, 128.04, 173.40)

## 10. Results and discussion

We primarily studied the maximum likelihood (MLH) estimation and Bayesian estimation to estimate the rate parameter of Erlang distribution. In Bayesian method, we use Jeffreys’ prior and Quasi prior under three different loss functions. These methods are compared through simulation technique and the results are presented in the Tables 1 and 2 respectively.

N | k | |||||||
---|---|---|---|---|---|---|---|---|

c = −1 | c = 1 | a = 0.5 | a = 1.0 | |||||

15 | 1 | 0.5 | 0.15350 | 0.16179 | 0.10530 | 0.18428 | 0.13350 | 0.12597 |

2 | 1.0 | 0.51869 | 0.54673 | 0.43100 | 0.58896 | 0.47443 | 0.44430 | |

30 | 1 | 0.5 | 0.03480 | 0.03238 | 0.03238 | 0.03266 | 0.04191 | 0.03259 |

2 | 1.0 | 0.06850 | 0.04258 | 0.07049 | 0.02085 | 0.03077 | 0.02171 | |

60 | 1 | 0.5 | 0.03910 | 0.03320 | 0.02856 | 0.03488 | 0.03110 | 0.03060 |

2 | 1.0 | 0.10349 | 0.09918 | 0.08981 | 0.10244 | 0.09398 | 0.09200 |

N | k | d | |||||||
---|---|---|---|---|---|---|---|---|---|

c = −1 | c = 1 | a = 0.5 | a = 1.0 | ||||||

15 | 1 | 0.5 | 0.5 | 0.15350 | 0.14710 | 0.11081 | 0.18979 | 0.13901 | 0.13148 |

1.0 | 0.15350 | 0.16179 | 0.10530 | 0.18428 | 0.13350 | 0.12597 | |||

2 | 1.0 | 0.5 | 0.51869 | 0.51528 | 0.43951 | 0.59747 | 0.48294 | 0.45281 | |

1.0 | 0.51869 | 0.54673 | 0.43100 | 0.58896 | 0.47443 | 0.44430 | |||

30 | 1 | 0.5 | 0.5 | 0.03480 | 0.03286 | 0.02179 | 0.02206 | 0.21630 | 0.02200 |

1.0 | 0.03480 | 0.03238 | 0.03238 | 0.03266 | 0.04191 | 0.03259 | |||

2 | 1.0 | 0.5 | 0.06850 | 0.05828 | 0.06361 | 0.01397 | 0.02389 | 0.01483 | |

1.0 | 0.06850 | 0.04258 | 0.07049 | 0.02085 | 0.03077 | 0.02171 | |||

60 | 1 | 0.5 | 0.5 | 0.03910 | 0.03206 | 0.03206 | 0.03534 | 0.03156 | 0.03106 |

1.0 | 0.03910 | 0.03320 | 0.02856 | 0.03488 | 0.03110 | 0.03060 | |||

2 | 1.0 | 0.5 | 0.10349 | 0.09641 | 0.09021 | 0.10284 | 0.09439 | 0.09241 | |

1.0 | 0.10349 | 0.09918 | 0.08981 | 0.10244 | 0.09398 | 0.09200 |

From the results obtained in Tables 1 and 2, we observe that in most of the cases, Bayesian estimator under Al-Bayyati’s loss function has the smallest mean squared error (MSE) values for Jeffrey’s prior and Quasi prior as compared to other loss functions and the maximum likelihood estimator. Thus we can conclude that Bayes estimator under Al-Bayyati’s loss function is efficient when the loss parameter **C** is −1.

Also we estimated the unknown parameters of sub-models of Erlang distribution. The Akaike information criterion (AIC), Bayesian information criterion (BIC), and the corrected Akaike information criterion (AICC) are used to compare the candidate distributions. The best distribution corresponds to lower −logL, AIC, BIC, AICC statistics value.

From the results obtained in Tables 3 and 4, we observe that the Erlang distribution is a competitive distribution as compared to its sub-models (i.e., exponential distribution and Chi-square distribution). In fact, based on the values of the AIC, BIC and AICC criteria, it shows clear picture that the Erlang distribution provides the best fit for these data among all the models considered.

Model | Parameter estimate | Standard error | Measures | |||
---|---|---|---|---|---|---|

AIC | BIC | AICC | ||||

Exponential | 0.04203 | 315.6853 | 633.3705 | 636.6689 | 581.9245 | |

Chi-square | k = 1.090 | 0.0586 | 312.3096 | 625.6346 | 628.9329 | 633.3902 |

Erlang | k = 2 | 289.9327 | 581.8654 | 585.1637 | 625.6543 | |

0.0594 |

Model | Parameter estimate | Standard error | Measures | |||
---|---|---|---|---|---|---|

AIC | BIC | AICC | ||||

Exponential | 0.002877 | 121.4324 | 244.8648 | 246.0003 | 245.0248 | |

Chi-square | k = 2.9420 | 1.1745 | 177.5835 | 357.1670 | 358.3025 | 357.327 |

Erlang | k = 1 | 113.0298 | 230.0597 | 232.3307 | 230.5212 | |

0.01683 |

## 11. Conclusions

In this paper we have generated three types of data sets with varying sample sizes for Erlang distribution. These data sets were simulated and behavior of the data was checked in case of parameter estimation for Erlang distribution in R Software. By the virtue of the data analysis we are able predict the estimate of rate parameter for Erlang distribution under three different functions by using two different prior distributions. With the help of these results we can also do comparison between loss functions and the priors.

Also the comparison of Erlang distribution with its sub-models was carried out. The results acquired in Tables 3 and 4, it shows the clear picture that Erlang distribution performs better as compared to its sub-models. Thus we can say that Erlang distribution is efficient as compared to its sub-models (i.e., exponential distribution and Chi-square distribution) on the basis of the above procedures.

## Acknowledgments

The authors acknowledge editor of this journal for encouragement to finalize the chapter. Further, the authors acknowledge profound thanks to anonymous referee for giving critical comments which have immensely improved the presentation of the chapter. Also I extend my sincere thanks to all the authors whose papers I have consulted for this work.

## Conflict of interest

The authors declare that there is no conflict of interests regarding the publication of this chapter.