Open access peer-reviewed chapter - ONLINE FIRST

A Comparative Study of Maximum Likelihood Estimation and Bayesian Estimation for Erlang Distribution and Its Applications

By Kaisar Ahmad and Sheikh Parvaiz Ahmad

Submitted: September 13th 2018Reviewed: March 4th 2019Published: September 27th 2019

DOI: 10.5772/intechopen.85627

Downloaded: 37

Abstract

In this chapter, Erlang distribution is considered. For parameter estimation, maximum likelihood method of estimation, method of moments and Bayesian method of estimation are applied. In Bayesian methodology, different prior distributions are employed under various loss functions to estimate the rate parameter of Erlang distribution. At the end the simulation study is conducted in R-Software to compare these methods by using mean square error with varying sample sizes. Also the real life applications are examined in order to compare the behavior of the data sets in the parametric estimation. The comparison is also done among the different loss functions.

Keywords

  • Erlang distribution
  • prior distributions
  • loss functions
  • simulation study
  • applications

1. Introduction

Erlang distribution is a continuous probability distribution with wide applicability, primarily due to its relation to the exponential and gamma distributions. The Erlang distribution was developed by Erlang [1] to examine the number of telephone calls that could be made at the same time to switching station operators. This distribution can be expressed as waiting time and message length in telephone traffic. If the duration of individual calls are exponentially distributed then the duration of succession of calls is the Erlang distribution. The Erlang variate becomes gamma variate when its shape parameter is an integer (for details see Evans et al. [2]). Bhattacharyya and Singh [3] obtained Bayes estimator for the Erlangian queue under two prior densities. Haq and Dey [4] addressed the problem of Bayesian estimation of parameters for the Erlang distribution assuming different independent informative priors. Suri et al. [5] used Erlang distribution to design a simulator for time estimation of project management process. Damodaran et al. [6] obtained the expected time between failure measures. Further, they showed that the predicted failure times are closer to the actual failure times. Jodra [7] showed the procedure of computing the asymptotic expansion of the median of Erlang distribution.

The probability density function of an Erlang variate is given by

fxλk=λkk1!xk1eλxforx>0,kNandλ>0.E1

where λand kare the rate and the shape parameters, respectively, such that is k an integer number.

1.1 Graphical representation of pdf for Erlang distribution

In this chapter, Erlang distribution is considered. Some structural properties of Erlang distribution have been obtained. The parameter estimation of Erlang distribution is obtained by employing the maximum likelihood method of estimation, method of moments and Bayesian method of estimation in different sections of this chapter. In Bayesian approach, the parameters are estimated by using Jeffrey’s and Quasi priors under different loss functions (Figure 1).

Figure 1.

Pdf’s of Erlang distribution for different values of lambda and k.

1.2 Relationship of Erlang distribution with other distributions

  1. The gamma distribution is a generalized form of the Erlang distribution.

  2. If the shape parameter k is 1, then Erlang distribution reduces to exponential distribution.

  3. If the scale parameter is 2, then Erlang distribution reduces to Chi-square distribution with 2 degrees of freedom.

Thus from the above descriptions, we can say that exponential distribution and Chi-square distribution are the sub-models of Erlang distribution.

2. Methods used for parameter estimation

In this chapter, we have used different approaches for parameter estimation. The first two methods come under the classical approach which was founded by Fisher in a series of fundamental papers round about 1930.

The alternative approach is the Bayesian approach which was first discovered by Reverend Thomas Bayes. In this chapter, we have used two different priors for the parameter estimation. Also three loss functions are used which are discussed in their respective sections. A number of symmetric and asymmetric loss functions used by various researchers; see Zellner [8], Ahmad and Ahmad [9], Ahmad et al. [10], etc.

These methods of estimations are elaborated in their respective sections accordingly.

2.1 Maximum likelihood (MLH) estimation

The most general method of estimation is known as maximum likelihood (MLH) estimators, which was initially formulated by Gauss. Fisher in the early 1920 firstly introduced MLH as general method of estimation and later on developed by him in a series of papers. He revealed the advantages of this method by showing that it yields sufficient estimators, which are asymptotically MVUES. Thus the important feature of this method is that we look at the value of the random sample and then select our estimate of the unknown population parameter, the value of which the probability of getting the observed data is maximum.

Suppose the observed data sample values are x1x2xn. When X is a discrete random variable, we can write PX1=x1X2=x2Xn=xn=fx1x2xn, which is the value of joint probability distribution at the sample point x1x2xn. Since the sample values has been observed and are therefore fixed numbers, we consider fx1x2,xnλas the value of a function of the parameter λ, referred to as the likelihood function.

Similarly the definition applies when the random sample comes from a continuous population but in that case fx1x2,xnλis the value of joint pdf at the sample point x1x2xn. That is, the likelihood function at the sample value x1x2xnwhich is given by

Lx1x2xnλ=i=1mfxiλ.

Since the principle of maximum likelihood consists in finding an estimator of the parameter which maximizes the likelihood function for variation in the parameter. Thus if there exists a function λ̂=λ̂x1x2xnof the sample values which maximizes Lxλfor variation in λ, then λ̂is to be taken as the estimator of λ. Usually we call λ̂as ML estimators. Thus λ̂is the solution

iffLxλλ=0and2Lxλλ2<0.

Since Lxλ>0, so logLxλwhich shows that Lxλand logLxλattains their extreme values at λ̂. Therefore, the equation becomes

1LxλLxλλ=0logLxλλ=0

a form which is more convenient from practical point of view.

The MLH estimation of the rate parameter of Erlang distribution is obtained in the following theorem:

Theorem 2.1: Let x1x2xnbe a random sample of size n from Erlang density function Eq. (1), then the maximum likelihood estimator of λis given by

λ̂=nki=1nxi.

Proof: The likelihood function of random sample of size n having Erlang density function Eq. (1) is given by

Lxλk=λkk1!ni=1nxik1eλi=1nxi.E2

The log likelihood function is given by

logLxλβ=nklogλ+k1i=1nlogxiλi=1nxinlogk1!.E3

Differentiating Eq. (3) w.r.t. λand equating to zero, we get

λnklogλ+k1i=1nlogxiλi=1nxinlogk1!.=0
λ̂=nki=1nxi.E4

2.2 Method of moments (MM)

One of the simplest and oldest methods of estimation is the method of moments. The method of moments was discovered by Karl Pearson in 1894. It is a method of estimation of population parameters such as mean, variance, etc. (which need not be moments), by equating sample moments with unobservable population moments and then solving those equations for the quantities to be estimated. The method of moments is special case when we need to estimate some known function of finite number of unknown moments.

Suppose fxλ1λ2λpbe the density function of the parent population with p parameters λ1λ2λp. Let μsbe the sth moment of a random variable about origin and is given by

μs=xsfxλ1λ2λp;r=1,2,,p.

In general μ1μ2μpwill be the functions of parameters λ1λ2λp. Let xii=12nbe a random sample of size n from the given population. The method of moments consists in solving the p-equations (i) for λ1λ2λpin terms of μ1μ2μp. Then replacing these moments μss=123pby the sample moments

e.g., λ̂i=λ̂μ1μ2μp=λim1m2..mp;i=1,2,,p.

where miis the ith moment about origin in the sample.

Then by the method of moments λ̂1λ̂2λ̂pare the estimators of respectively.

The MM estimation of the rate parameter of Erlang distribution is obtained in the following theorem:

Theorem 2.2: Let x1x2xnbe a random sample of size n from Erlang density function Eq. (1), then the moment estimator of λis given by

λ̂=kx¯.

Proof: If the numbers x1x2xnrepresents a set of data, then an unbiased estimator for the rth moment about origin is

m̂r=1ni=1nxiE5

where m̂rstands for the estimate of mr.

The rth moment of two parameter Erlang distribution about origin is given by

μr=0xrfxλkdxE6

Using Eq. (5) in Eq. (6), we have

μr=Γr+kλrk1!.E7

If r = 1 in Eq. (7), we get

μ1=kλ.

If r = 2, then Eq. (7) becomes

μ2=kk+1λ2.

Thus the variance is given by σ2=kλ2.

When we divide σ2by μ12, we get an expression which is a function of k only and is given by

σ2μ12=kλ2kλ2
σ2μ12=1k.E8

On taking the square roots of Eq. (8), we have the coefficient of variation

σμ1=1k.E9

The rate parameter λcan be then estimated using the following equation

m1=μ1.

If r = 1, in Eq. (5), then

m1=x¯.

Also if r = 1 in Eq. (7), then

μ1=kλ.

Thus,

m1=μ1
x¯=kλ, where x¯is the mean of the data and

λ̂=kx¯.

3. Bayesian method of estimation

Nowadays, the Bayesian school of thought is garnering more attention and at an increasing rate. This thought of statistics was given by Reverend Thomas Bayes. He first discovered the theorem that now bears his name. It was written up in a paper “An Essay Towards Solving a Problem in the Doctrine of Chances.” This paper was found after his death by his friend Richard Price, who had it published posthumously in the Philosophical Transactions of the Royal Society in 1763. Bayes showed how inverse probability could be used to calculate probability of antecedent events from the occurrence of the consequent event. His methods were adopted by Laplace and other scientists in the nineteenth century. By mid twentieth century interest in Bayesian methods was renewed by De Finetti, Jeffreys and Lindley, among others. They developed a complete method of statistical inference based on Bayes’ theorem. Bayesian analysis is to be used by practitioners for situations where scientists have a priori information about the values of the parameters to be estimated. In everyday life, uncertainty often permeates our choices, and when choices need to be made, past experience frequently proves a helpful aid. Bayesian theory provides a general and consistent framework for dealing with uncertainty.

Within Bayesian inference, there are also different interpretations of probability, and different approaches based on those interpretations. Early efforts to make Bayesian methods accessible for data analysis were made by Raiffa and Schlaifer [11], DeGroot [12], Zellner [13], and Box and Tiao [14]. The most popular interpretations and approaches are objective Bayesian inference and subjective Bayesian inference. Excellent expositions of these approaches are with Bayes and Price [15], Laplace [16], Jeffrey’s [17], Anscombe and Aumann [18], Berger [19, 20], Gelman et al. [21], Leonard and Hsu [22], De-Finetti [23]. Modern Bayesian data analysis and methods based on Markov chain Monte Carlo methods are presented in Bernardo and Smith [24], Robert [25], Gelman et al. [26], Marin and Robert [27], Carlin and Louis [28]. Good elementary introductions to the subject are Ibrahim et al. [29], Ghosh [30], Bansal [31], Koch [32], Hoff [33].

In Bayesian statistics probability is not defined as a frequency of occurrence but as the plausibility that a proposition is true, given the available information. The parameters are treated as random variables. The rules of probability are used directly to make inferences about the parameters. Probability statements about parameters must be interpreted as “degree of belief.” We revise our beliefs about parameters after getting the data by using Bayes’ theorem. This gives our posterior distribution which gives the relative weights we give to each parameter value after analyzing the data. The posterior distribution comes from two sources: the prior distribution and the observed data. This means that the inference is based on the actual occurring data, not all possible data sets that might have occurred.

In this section, posterior distribution of Erlang distribution is obtained by using Jeffrey’s prior and Quasi prior. The rate parameter of Erlang distribution is estimated with the help of different loss functions. For parameter estimation we have used the approach as is used by Ahmad et al. [34], Ahmad et al. [35], etc. Some important prior distributions and loss functions which we have used in this article are given as below:

3.1 Prior distributions used

Prior distribution is the basic part of Bayesian analysis which represents all that is known or assumed about the parameter. Usually the prior information is subjective and is based on a person’s own experience and judgment, a statement of one’s degree of belief regarding the parameter.

Another important feature of the Bayesian analysis is the choice of the prior distribution. If the data have sufficient signal, even a bad prior will still not greatly influence the posterior. We can examine the impact of prior by observing the stability of posterior distribution related to different choices of priors. If the posterior distribution is highly dependent on the prior, then the data (the likelihood function) may not contain sufficient information. However, if the posterior is relatively stable over a choice of priors, then the data indeed contains significant information.

Prior distribution may be categorical in different ways. One common classification is a dichotomy that separated “proper” and “improper” priors.

A prior distribution is proper if it does not depend on the data and the value of integral gλor summation gλis one. If the prior does not depend on the data and the distribution does not integrate or sum to one then we say that the prior is improper.

In this chapter, we have used two different priors Jeffrey’s prior and Quasi prior which are given below:

3.1.1 Jeffrey’s prior

An invariant form for the prior probability in estimation problems is given by Jeffery’s [36].

The general formula of the Jeffreys prior, which is defined by

gλIλE2logLλxλ212.

where Iλis the Fisher information for the parameter λ.When there are multiple parameters I is the Fisher information matrix, the matrix of the expected second partials

Iλ=E2logLλxλiλj.

In this situation, the Jeffreys prior is given by

gλdetIλ.

Jeffrey suggested a thumb rule for specifying non-informative prior for parameter λas

Rule 1: if λtake gλto be constant, i.e., λto be uniformly distributed.

Rule 2: if λ0take gλ1λ, i.e., logλto be uniformly distributed.

Under linear transformation, rule 1 is invariant and under any power transformation of λ, rule 2 is invariant.

3.1.2 Quasi prior

When there is no more information about the distribution parameter, one may use the Quasi density as given by gλ=1λd;λ>0andd>0.

3.2 Loss functions used

The concept of loss function is old as Laplace and was reintroduced in statistics by Abraham Wald [37]. In statistics, typically a loss function is used for parameter estimation, and the event in question is some function of the difference between estimated and true values for an occurrence of data. In the context of economics, loss function is usually economic cost. In optimal control, the loss is the penalty for failing to achieve a desired value.

The word “loss” is used in place of “error” and the loss function is used as a measure of the error or loss. Loss function is a measure of the error and presumably would be greater for large error than for small error. We would want the loss to be small or we want the estimate to be close to what it is estimating. Loss depends on sample and we cannot hope to make the loss small for every possible sample but can try to make the loss small on the average. Our objective is to select an estimator that makes this error or loss small, also which makes the average loss (risk) small and ideally select an estimator that has the small risk.

In this chapter, we have used three different Loss Functions which are as under:

3.2.1 Precautionary loss function (PLF)

The concept of precautionary loss function (PLF) was introduced by Norstrom [38]. He introduced an alternative asymmetric loss function and also presented a general class of precautionary loss functions as a special case. These loss functions approach infinitely near the origin to prevent underestimation, thus giving conservative estimators, especially when low failure rates are being estimated. These estimators are very useful when underestimation may lead to serious consequences. A very useful and simple asymmetric precautionary loss function (PLF) is

Lλ̂λ=λ̂λ2λ.

3.2.2 Al-Bayyati’s loss function (ALF)

The loss function proposed by Al-Bayyati [39] is an asymmetric loss function and is given by

lAλ̂λ=λcλ̂λ2R.

where λand λ̂represents the true and estimated values of the parameter. This loss function is frequently used because of its analytical tractability in Bayesian analysis.

3.2.3 LINEX loss function (LLF)

The idea of LINEX loss function (LLF) was founded by Klebanov [40] and used by Varian [41] in the context of real estate assessment. The formula of LLF is given by

Lλ̂λ=expaλ̂λaλ̂λ1

where λand λ̂represents the true and estimated values of the parameter and the constant c determines the shape of the loss function.

3.3 Posterior density under Jeffrey’s prior

Let x1x2xnbe a random sample of size n having the Erlang density function Eq. (1) which is given by

fxλk=λkk1!xk1eλxforx>0,kNandλ>0

and the likelihood function Eq. (2) given as below

Lxλk=λkk1!ni=1nxik1eλi=1nxi.

Jeffreys’ non-informative prior for λis given by

gλdetIλ.

where Iλ=nE2logfxλkλ2is the Fisher’s information matrix for the probability density function Eq. (1).

On solving the above expression, we have

gλ=1λ.E10

By using the Bayes theorem, we have

π1λx¯Lxλgλ.E11

Using Eqs. (2) and (10) in Eq. (11), we get

π1λx¯λnk1k1!i=1nxik1eλi=1nxi
π1λx¯=ρλnk1eλi=1nxiE12

where ρ is independent of λ and

ρ1=0λnk1eλi=1nxi
ρ=i=1nxinkΓnk.

Using the value of ρ in Eq. (12)

π1λx¯=λnk1eλi=1nxii=1nxinkΓnk.E13

3.4 Posterior density under Quasi prior

Let x1x2xnbe a random sample of size n having the Erlang density function Eq. (2) and the likelihood function Eq. (2).

The Quasi for λis given by

gλ=1λd.E14

By using the Bayes theorem, we have

π2λx¯Lxλgλ.E15

Using Eqs. (2) and (14) in Eq. (15), we have

π2λx¯λnkdk1!i=1nxik1eλi=1nxi
π2λx¯=ρλnkdeλi=1nxiE16

where ρis independent of λand

ρ1=0λnk2ceλi=1nxi
ρ=i=1nxinkd+1Γnkd+1.

By using the value of ρin Eq. (16), we have

π2λx¯=λnkdeλi=1nxii=1nxinkd+1Γnkd+1.E17

4. Estimation of parameters under Jeffrey’s prior

In this section, parameter estimation of Erlang distribution is done by using Jeffreys’ prior under different loss functions. The procedure of calculating the Bayesian estimate is already defined in Section 3. The estimates are obtained in the following theorems:

Theorem 4.1: Assuming the loss function Lpλ̂λ, the Bayesian estimator of the rate parameter λ, if the shape parameter kis known, is of the form

λ̂p=nknk+1i=1nxi.

Proof: The risk function of the estimator λunder the precautionary loss function Lpλ̂λis given by the formula

Rλ̂=0λ̂λλ̂2π1λx¯.E18

Using Eq. (13) in Eq. (18), we get

R(λ̂)=0(λ̂λ)λ̂2(λnk1(i=1nxi)nkeλi=1nxiΓnk)dλR(λ̂)=(i=1nxi)nkΓnk[λ̂0λnk1eλi=1nxidλ+1λ̂0λnk+1eλi=1nxidλ20λnkeλi=1nxidλ].

On solving the above expression, we get

Rλ̂=λ̂+1λ̂nknk+1i=1nxi22nki=1nxi.

Minimization of the risk with respect to λ̂gives us the optimal estimator i.e.,

λ̂Rλ̂=0
λ̂p=nknk+1i=1nxi.E19

Theorem 4.2: Assuming the loss function lAλ̂λ, the Bayesian estimator of the rate parameter λ, if the shape parameter kis known, is of the form

λ̂A=nk+ci=1nxi.

Proof: The risk function of the estimator λunder the Al-Bayyati’s loss function LAλ̂λis given by the formula

Rλ̂=0λcλ̂λ2π1λx¯.E20

On substituting Eq. (13) in Eq. (20), we have.

R(λ̂)=0λc(λ̂λ)2(λnk1(i=1nxi)nkeλi=1nxiΓnk)dλR(λ̂)=(i=1nxi)nkΓnk[λ̂20λnk+c1eλi=1nxidλ+0λnk+c+1eλi=1nxidλ2λ̂0λnk+ceλi=1nxidλ].

On solving the above expression, we get

Rλ̂=λ̂2Γnk+cΓnki=1nxic+Γnk+c+2Γnki=1nxic+22λ̂Γnk+c+1Γnki=1nxic+1.

Minimization of the risk with respect to λ̂gives us the optimal estimator i.e.,

λ̂Rλ̂=0
λ̂A=nk+ci=1nxi.E21

Theorem 4.3: Assuming the loss function Llλ̂λ, the Bayesian estimator of the rate parameter λ, if the shape parameter kis known, is of the form

λ̂l=nklog1+ai=1nyia.

Proof: The risk function of the estimator λunder the LINEX loss function Llλ̂λis given by the formula

Rλ̂=0expaλ̂λaλ̂λ1π1λx¯.E22

Using Eq. (13) in Eq. (22), we have

Rλ̂=0expaλ̂λaλ̂λ1λnk1i=1nxinkeλi=1nxiΓnk
Rλ̂=i=1nxinkΓnkexpaλ̂0expa+i=1nxiλλnk1aλ̂0expi=1nxiλλnk1+a0expi=1nxiλλnk0expi=1nxiλλnk1.

On solving the above expression, we get

Rλ̂=i=1nxinkexpaλ̂a+i=1nxinkaλ̂+aΓnk+1Γnki=1nxi1.

Minimization of the risk with respect to λ̂gives us the optimal estimator i.e.,

λ̂Rλ̂=0
λ̂l=nklog1+ai=1nxia.E23

5. Estimation of parameters under Quasi prior

In this section, parameter estimation of Erlang distribution is done by using QUASI prior under different loss functions. The procedure for obtaining the Bayesian estimate is available in Section 3. The estimates of parameter are obtained in the following theorems.

Theorem 5.1: Assuming the loss function Lpλ̂λ, the Bayesian estimator of the rate parameter λ, if the shape parameter kis known, is of the form

λ̂p=nkd+1nkd+2i=1nxi.

Proof: The risk function of the estimator λunder the precautionary loss function Lpλ̂λis given by the formula

Rλ̂=0λ̂λλ̂2π2λx¯E24

Using Eq. (17) in Eq. (24), we have

Rλ̂=0λ̂λλ̂2λnkdi=1nxinkd+1eλi=1nxiΓnkd+1
Rλ̂=i=1nxinkd+1Γnkd+1λ̂0λnkdeλi=1nxi+1λ̂0λnkd+2eλi=1nxi20λnkd+1eλi=1nxi.

On solving the above expression, we get

Rλ̂=λ̂+1λ̂Γnkd+3Γnkd+1i=1nxi22Γnkd+2Γnkd+1i=1nxi.

Minimization of the risk with respect to λ̂gives us the optimal estimator i.e.,

λ̂Rλ̂=0
λ̂p=nkd+1nkd+2i=1nxi.E25

Remark: Replacing d = 1 in Eq. (25), the same Bayes estimator is obtained as in Eq. (19).

Theorem 5.2: Assuming the loss function lAλ̂λ, the Bayesian estimator of the rate parameter λ, if the shape parameter kis known, is of the form

λ̂A=nkd+c+1i=1nxi.

Proof: The risk function of the estimator λunder the Al-Bayyati’s loss function LAλ̂λis given by the formula

Rλ̂=0λcλ̂λ2π2λx¯.E26

By using Eq. (17) in Eq. (26), we have.

R(λ̂)=0λc(λ̂λ)2(λnkd(i=1nxi)nkd+1eλi=1nxiΓ(nkd+1))dλR(λ̂)=(i=1nxi)nkd+1Γ(nkd+1)[λ̂20λnkd+ceλi=1nxidλ+0λnkd+c+2eλi=1nxidλ2λ̂0λnkd+c+1eλi=1nxidλ].

On solving the above expression, we get

Rλ̂=λ̂2Γnkd+c+1Γnkd+1i=1nxic+Γnkd+c+3Γnkd+1i=1nxic+22λ̂Γnkd+c+2Γnkd+1i=1nxic+1

Minimization of the risk with respect to λ̂gives us the optimal estimator, i.e., λ̂Rλ̂=0

λ̂A=nkd+c+1i=1nxi.E27

Remark: Replacing d = 1 in Eq. (27), the same Bayes estimator is obtained as in Eq. (21).

Theorem 5.3: Assuming the loss function Llλ̂λ, the Bayesian estimator of the rate parameter λ, if the shape parameter kis known, is of the form

λ̂l=nkd+1log1+ai=1nxia.

Proof: The risk function of the estimator λunder the LINEX loss function Llλ̂λis given by the formula

Rλ̂=0expaλ̂λaλ̂λ1π2λx¯.E28

Using Eq. (17) in Eq. (28), we have.

R(λ̂)=0(exp(a(λ̂λ))a(λ̂λ)1)(λnkd(i=1nxi)nkd+1eλi=1nxiΓ(nkd+1))dλR(λ̂)=(i=1nxi)nkd+1Γ(nk2c+1)[exp(aλ̂)0(exp((a+i=1nxi)λ))λnkddλaλ̂0(exp(i=1nxi)λ)λnkddλ+ a0(exp(i=1nxi)λ)λnkd+1dλ0(exp(i=1nxi)λ)λnkddλ].

On solving the above expression, we get

Rλ̂=expaλ̂i=1nxinkd+1a+i=1nxinkd+1aλ̂+aΓnkd+2Γnkd+1i=1nxi1.

Minimization of the risk with respect to λ̂gives us the optimal estimator i.e.,

λ̂Rλ̂=0
λ̂l=nkd+1log1+ai=1nxia.E29

Remark: Replacing d = 1 in Eq. (29), the same Bayes estimator is obtained as in Eq. (23).

6. Entropy estimation of Erlang distribution

The concept of entropy was introduced by Claude. Shannon [42] in the paper “A Mathematical theory of Communication.” This concept of Shannon’s entropy is the central role of information theory, sometimes referred as measure of uncertainty. Shannon entropy provides an absolute limit on the best possible lossless encoding or compression of any communication, assuming that the communication may be represented as a sequence of independent and identical distributed random variables. Entropy is typically measured in bits, when the log is to the base 2, and nats, when the log is to the base n.

Shannon’s definition of entropy, when applied to an information source, can determine the minimum channel capacity required to reliably transmit the source as encoded binary digit. The entropy of a random variable is defined in terms of its probability distribution and can be shown to be a good measure of randomness or uncertainty. For deriving the entropy of probability distributions, we need the following two definitions that are more discussed in Cover et al. [43].

Definition (i): The entropy of the discrete random variable defined on the probability space is given by

HPf=i=1npf=alogpf=a.

It is obvious that HPf0.

Definition (ii): The entropy of the continuous random variable defined on the real line is given by

Hf=Elogx=fxlogfxdx.

In this section, entropy estimation of two parameter Erlang distribution is discussed which given as below.

Theorem 6.1: Let x1x2xnbe n positive independent and identically distributed random samples drawn from a population having Erlang density function Eq. (2), then the Shannon’s entropy of two parameter Erlang distribution is given by

Hfxαβ=logλkk1!k1ψklogλ+k.

Proof: Shannon’s entropy for a continuous random variable is defined as

Hfxλk=E(logfx=fxlogfxdxE30

Using Eq. (1) in Eq. (30), we have

Hfxαβ=Elogλkk1!xk1eλx
Hfxαβ=logλkk1!k1Elogx+λEx.E31

Now

Elogx=0logxfxdx
Elogx=λkk1!0logxxk1eλxdx
Putλx=tx=tλ;asx0,t0andasx,t
Elogx=λk1k1!0logtλtλk1etdt
Elogx=1k1!0logttk1etdtlogλ0tk1etdt
Elogx=ΓkΓklogλ
Elogx=ψklogλ.E32

Also

Ex=0xfxλkdx
Ex=λkk1!0xkeλxdx
Ex=λkk1!Γk+1λk+1
Ex=kλ.E33

Substitute the value of Eqs. (32) and (33) in Eq. (31), we have.

Hfxαβ=logλkk1!k1ψklogλ+k.E34

7. AIC and BIC criterion for Erlang distribution

For model selection the approach of Akaike information criterion (AIC) and Bayesian information criterion (BIC) based on entropy estimation are used. The Akaike information criterion (AIC) was introduced by Hirotsugu Akaike [44] and proposed it as a measure of goodness of fit of an estimated statistical model. It is a measure of the relative quality of a statistical model for a given set of data. It has been found in information theory that it offers a relative estimate of the information lost when a given model is used to represent the process that generates the data.

The AIC is not a test of the model in the sense of hypothesis testing; rather it is a test between models—a tool for model selection. Given a data set, several competing models may be ranked according to their AIC, with the one having the lowest AIC being the best.

The formula for AIC is given by

AIC=2K2logLλ̂.

where K is the number of parameters and Lλ̂is the maximized value of the likelihood function for the estimated model.

AICC was first introduced by Hurvich and Tsai [45] and its different derivations were proposed by Burnham and Anderson [46]. AICC is AIC with a correction for finite sample sizes and is given by

AICC=AIC+2KK+1nK1.

Burnham and Anderson [47] strongly recommended that we should use AICC instead of AIC when the sample size is small or if K is large. Since AICC converges to AIC as the sample size is getting large. Using AIC, instead of AICC, when the sample size is not many times larger than K2, increases the probability of selecting models that have too many parameters, i.e., of over fitting. The probability of AIC over fitting can be substantial, in some cases.

The Bayesian information criterion (BIC) also known as Schwarz Criterion is used as a substitute for full calculation of the Bayes’ factor since it can be calculated without specifying prior distribution. In BIC, the penalty for additional parameters is stronger than that of the AIC.

The formula for the BIC is given by

BIC=Klogn2logLλ̂.

where K is the number of parameters, n is the sample size and Lλ̂is the maximized value of the likelihood function.

The AIC and BIC of two parameter Erlang distribution are obtained in this section, which are given below.

The Shannon’s entropy of two parameter Erlang distribution is given by

SHED=logk1!logλkk1Elogx+λEx
ĤED=logk1!logλkk1logx¯+λx¯.E35

Also

ll(x;λ,k)=nlogλknlog(k1)!+(k1)i=1nlogxiλi=1nxill(x;λ̂,k̂)=n[log(k1)!logλk(k1)logx¯+λx¯].E36

Comparing Eqs. (35) and (36), we have

llxλ̂k̂=nĤED.

The AIC and BIC methodology attempts to find the model that best explains the data with a minimum of their values, we have.

llxλ̂k̂=nĤED, then for Erlang family we have

AIC=2K+2nĤEDE37

or

AIC=2K2ll

and

BIC=Klogn+2nĤEDE38

or

BIC=Klogn2ll.

8. Simulation study of Erlang distribution

We have generated the data for Erlang distribution of different sample sizes (15, 30 and 60) in R Software for each pairs of λk, where k=12and λ=0.51.0. The value for the loss parameter (C1 = −1, 1) and (a = 0.5, 1.0). The values of extension are (C = 0.5, 1.0). The estimates of rate parameter for each method are calculated. The results are presented in the following tables.

9. Comparison of Erlang distribution (ED) with its sub-models

The flexibility and potentiality of the Erlang distribution is compared with its sub models, which is examined by using different criterions like AIC, BIC and AICC with the help of the following illustration.

Illustration I:

We provide the compatibility of the Erlang distribution (ED) with their sub-models; Chi square and exponential distributions. For this purpose, we generated the data set for Erlang distribution of large sample size (i.e., 200) in R Software for each pairs of λk, where k=2and λ=2.5. The data analysis is given in the following table:

Illustration II:

The data set is taken from Lawless [48]. The observations involves the number of million revolutions between failures for each of 23 ball bearings, the individual bearings were inspected periodically to determine whether “failure” had occurred. Treating the failure times as continuous, the 23 failure times are:

(17.88, 28.92, 33.00, 41.52, 42.12, 45.60, 48.40, 51.84, 51.96, 54.02, 55.56, 67.80, 68.64, 68.64, 68.88, 84.12, 93.12, 98.64, 105.12, 105.84, 127.92, 128.04, 173.40)

10. Results and discussion

We primarily studied the maximum likelihood (MLH) estimation and Bayesian estimation to estimate the rate parameter of Erlang distribution. In Bayesian method, we use Jeffreys’ prior and Quasi prior under three different loss functions. These methods are compared through simulation technique and the results are presented in the Tables 1 and 2 respectively.

Nkλλ̂MLλ̂pλ̂Aλ̂l
c = −1c = 1a = 0.5a = 1.0
1510.50.153500.161790.105300.184280.133500.12597
21.00.518690.546730.431000.588960.474430.44430
3010.50.034800.032380.032380.032660.041910.03259
21.00.068500.042580.070490.020850.030770.02171
6010.50.039100.033200.028560.034880.031100.03060
21.00.103490.099180.089810.102440.093980.09200

Table 1.

Mean squared error for λ̂under Jeffrey’s prior.

ML = maximum likelihood estimate, p = precautionary LF, A = Al-Bayyati’s LF, l = LINEX LF.

Nkλdλ̂MLλ̂pλ̂Aλ̂l
c = −1c = 1a = 0.5a = 1.0
1510.50.50.153500.147100.110810.189790.139010.13148
1.00.153500.161790.105300.184280.133500.12597
21.00.50.518690.515280.439510.597470.482940.45281
1.00.518690.546730.431000.588960.474430.44430
3010.50.50.034800.032860.021790.022060.216300.02200
1.00.034800.032380.032380.032660.041910.03259
21.00.50.068500.058280.063610.013970.023890.01483
1.00.068500.042580.070490.020850.030770.02171
6010.50.50.039100.032060.032060.035340.031560.03106
1.00.039100.033200.028560.034880.031100.03060
21.00.50.103490.096410.090210.102840.094390.09241
1.00.103490.099180.089810.102440.093980.09200

Table 2.

Mean squared error for λ̂under Quasi prior.

ML = maximum likelihood estimate, p = precautionary LF, A = Al-Bayyati’s LF, l = LINEX LF.

From the results obtained in Tables 1 and 2, we observe that in most of the cases, Bayesian estimator under Al-Bayyati’s loss function has the smallest mean squared error (MSE) values for Jeffrey’s prior and Quasi prior as compared to other loss functions and the maximum likelihood estimator. Thus we can conclude that Bayes estimator under Al-Bayyati’s loss function is efficient when the loss parameter C is −1.

Also we estimated the unknown parameters of sub-models of Erlang distribution. The Akaike information criterion (AIC), Bayesian information criterion (BIC), and the corrected Akaike information criterion (AICC) are used to compare the candidate distributions. The best distribution corresponds to lower −logL, AIC, BIC, AICC statistics value.

From the results obtained in Tables 3 and 4, we observe that the Erlang distribution is a competitive distribution as compared to its sub-models (i.e., exponential distribution and Chi-square distribution). In fact, based on the values of the AIC, BIC and AICC criteria, it shows clear picture that the Erlang distribution provides the best fit for these data among all the models considered.

ModelParameter estimateStandard errorMeasures
loglAICBICAICC
Exponentialλ̂ = 0.59450.04203315.6853633.3705636.6689581.9245
Chi-squarek = 1.0900.0586312.3096625.6346628.9329633.3902
Erlangk = 2289.9327581.8654585.1637625.6543
λ̂ = 1.18900.0594

Table 3.

AIC, BIC and AICC criterion for different sub-models of ED.

ModelParameter estimateStandard errorMeasures
loglAICBICAICC
Exponentialλ̂ = 0.013870.002877121.4324244.8648246.0003245.0248
Chi-squarek = 2.94201.1745177.5835357.1670358.3025357.327
Erlangk = 1113.0298230.0597232.3307230.5212
λ̂ = 0.055770.01683

Table 4.

AIC, BIC and AICC criterion for different sub-models of ED.

11. Conclusions

In this paper we have generated three types of data sets with varying sample sizes for Erlang distribution. These data sets were simulated and behavior of the data was checked in case of parameter estimation for Erlang distribution in R Software. By the virtue of the data analysis we are able predict the estimate of rate parameter for Erlang distribution under three different functions by using two different prior distributions. With the help of these results we can also do comparison between loss functions and the priors.

Also the comparison of Erlang distribution with its sub-models was carried out. The results acquired in Tables 3 and 4, it shows the clear picture that Erlang distribution performs better as compared to its sub-models. Thus we can say that Erlang distribution is efficient as compared to its sub-models (i.e., exponential distribution and Chi-square distribution) on the basis of the above procedures.

Acknowledgments

The authors acknowledge editor of this journal for encouragement to finalize the chapter. Further, the authors acknowledge profound thanks to anonymous referee for giving critical comments which have immensely improved the presentation of the chapter. Also I extend my sincere thanks to all the authors whose papers I have consulted for this work.

Conflict of interest

The authors declare that there is no conflict of interests regarding the publication of this chapter.

Download

chapter PDF

© 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution 3.0 License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite and reference

Link to this chapter Copy to clipboard

Cite this chapter Copy to clipboard

Kaisar Ahmad and Sheikh Parvaiz Ahmad (September 27th 2019). A Comparative Study of Maximum Likelihood Estimation and Bayesian Estimation for Erlang Distribution and Its Applications [Online First], IntechOpen, DOI: 10.5772/intechopen.85627. Available from:

chapter statistics

37total chapter downloads

More statistics for editors and authors

Login to your personal dashboard for more detailed statistics on your publications.

Access personal reporting

We are IntechOpen, the world's leading publisher of Open Access books. Built by scientists, for scientists. Our readership spans scientists, professors, researchers, librarians, and students, as well as business professionals. We share our knowledge and peer-reveiwed research papers with libraries, scientific and engineering societies, and also work with corporate R&D departments and government entities.

More About Us