The Bayesian Posterior Estimators under Six Loss Functions for Unrestricted and Restricted Parameter Spaces

Abstract

In this chapter, we have investigated six loss functions. In particular, the squared error loss function and the weighted squared error loss function that penalize overestimation and underestimation equally are recommended for the unrestricted parameter space −∞∞; Stein’s loss function and the power-power loss function, which penalize gross overestimation and gross underestimation equally, are recommended for the positive restricted parameter space 0∞; the power-log loss function and Zhang’s loss function, which penalize gross overestimation and gross underestimation equally, are recommended for 01. Among the six Bayesian estimators that minimize the corresponding posterior expected losses (PELs), there exist three strings of inequalities. However, a string of inequalities among the six smallest PELs does not exist. Moreover, we summarize three hierarchical models where the unknown parameter of interest belongs to 0∞, that is, the hierarchical normal and inverse gamma model, the hierarchical Poisson and gamma model, and the hierarchical normal and normal-inverse-gamma model. In addition, we summarize two hierarchical models where the unknown parameter of interest belongs to 01, that is, the beta-binomial model and the beta-negative binomial model. For empirical Bayesian analysis of the unknown parameter of interest of the hierarchical models, we use two common methods to obtain the estimators of the hyperparameters, that is, the moment method and the maximum likelihood estimator (MLE) method.

1. Introduction

In Bayesian analysis, there are four basic elements: the data, the model, the prior, and the loss function. A Bayesian estimator minimizes some posterior expected loss (PEL) function. We confine our interests to six loss functions in this chapter: the squared error loss function (well known), the weighted squared error loss function ([1], p. 78), Stein’s loss function [2, 3, 4, 5, 6, 7, 8, 9, 10], the power-power loss function [11], the power-log loss function [12], and Zhang’s loss function [13]. It is worthy to note that among the six loss functions, the first and second loss functions are defined on $\mathrm{\Theta }=\left(-\infty ,\infty \right)$, and they penalize overestimation and underestimation equally. The third and fourth loss functions are defined on $\mathrm{\Theta }=\left(0,\infty \right)$, and they penalize gross overestimation and gross underestimation equally, that is, an action $a$ will suffer an infinite loss when it tends to $0$ or $\infty$. The fifth and sixth loss functions are defined on $\mathrm{\Theta }=\left(0,1\right)$, and they penalize gross overestimation and gross underestimation equally, that is, an action $a$ will suffer an infinite loss when it tends to $0$ or $1$.

The squared error loss function and the weighted squared error loss function have been used by many authors for the problem of estimating the variance, ${\sigma }^2$, based on a random sample from a normal distribution with mean $\mu$ unknown (see, for instance, [14, 15]). As pointed out by [16], the two loss functions penalize equally for overestimation and underestimation, which is fine for the unrestricted parameter space $\mathrm{\Theta }=\left(-\infty ,\infty \right)$.

For $\mathrm{\Theta }=\left(0,\infty \right)$, the positive restricted parameter space, where $0$ is a natural lower bound and the estimation problem is not symmetric, we should not choose the squared error loss function and the weighted squared error loss function but choose a loss function which can penalize gross overestimation and gross underestimation equally, that is, an action $a$ will suffer an infinite loss when it tends to $0$ or $\infty$. Stein’s loss function owns this property, and thus it is recommended for $\mathrm{\Theta }=\left(0,\infty \right)$ by many researchers (e.g., see [2, 3, 4, 5, 6, 7, 8, 9, 10]). Moreover, [11] proposes the power-power loss function which not only penalizes gross overestimation and gross underestimation equally but also has balanced convergence rates or penalties for its argument too large and too small. Therefore, Stein’s loss function and the power-power loss function are recommended for $\mathrm{\Theta }=\left(0,\infty \right)$.

Analogously, for a restricted parameter space $\mathrm{\Theta }=\left(0,1\right)$, where $0$ and $1$ are two natural bounds and the estimation problem is not symmetric, we should not select the squared error loss function and the weighted squared error loss function but select a loss function which can penalize gross overestimation and gross underestimation equally, that is, an action $a$ will suffer an infinite loss when it tends to $0$ or $1$. It is worthy to note that Stein’s loss function and the power-power loss function are also not appropriate in this case. The power-log loss function proposed by [12] has this property. Moreover, they propose six properties for a good loss function on $\mathrm{\Theta }=\left(0,1\right)$. Specifically, the power-log loss function is convex in its argument, attains its global minimum at the true unknown parameter, and penalizes gross overestimation and gross underestimation equally. Apart from the six properties, [13] proposes the seventh property, that is, balanced convergence rates or penalties for the argument too large and too small, for a good loss function on $\mathrm{\Theta }=\left(0,1\right)$. Therefore, the power-log loss function and Zhang’s loss function are recommended for $\mathrm{\Theta }=\left(0,1\right)$.

The rest of the chapter is organized as follows. In Section 2, we obtain two Bayesian estimators for $\theta \in \mathrm{\Theta }=\left(-\infty ,\infty \right)$ under the squared error loss function and the weighted squared error loss function. In Section 3, we obtain two Bayesian estimators for $\theta \in \mathrm{\Theta }=\left(0,\infty \right)$ under Stein’s loss function and the power-power loss function. In Section 4, we obtain two Bayesian estimators for $\theta \in \mathrm{\Theta }=\left(0,1\right)$ under the power-log loss function and Zhang’s loss function. In Section 5, we summarize three strings of inequalities in a theorem. Some conclusions and discussions are provided in Section 6.

2. Bayesian estimation for θ ∈(−∞,∞)

There are two loss functions which are defined on $\mathrm{\Theta }=\left(-\infty ,\infty \right)$ and penalize overestimation and underestimation equally, that is, the squared error loss function (well known) and the weighted squared error loss function (see [1], p. 78).

2.1 Squared error loss function

The Bayesian estimator under the squared error loss function (well known), ${\delta }_2^{\pi }\left(\mathit{x}\right)$, minimizes the posterior expected squared error loss (PESEL), $\mathrm{E}\left[L_2\left(\theta ,a\right)|\mathit{x}\right]$, that is,

$${\delta }_2^{\pi }\left(\mathit{x}\right)=arg\underset{a\in \mathcal{A}}{min}\mathrm{E}\left[L_2\left(\theta ,a\right)|\mathit{x}\right],$$E1

where $\mathcal{A}\left\{a\left(\mathit{x}\right):a\left(\mathit{x}\right)\in \left(-\infty ,\infty \right)\right\}$ is the action space, $a=a\left(\mathit{x}\right)\in \left(-\infty ,\infty \right)$ is an action (estimator),

$$L_2\left(\theta ,a\right)={\left(\theta -a\right)}^2$$E2

is the squared error loss function, and $\theta \in \left(-\infty ,\infty \right)$ is the unknown parameter of interest. The PESEL is easy to obtain (see [16]):

$$\mathit{\text{PESEL}}\left(\pi ,a|\mathit{x}\right)=\mathrm{E}\left[L_2\left(\theta ,a\right)|\mathit{x}\right]=a^2-2a\mathrm{E}\left(\theta |\mathbf{x}\right)+\mathrm{E}\left({\theta }^2|\mathit{x}\right).$$E3

It is found in [16] that

$${\delta }_2^{\pi }\left(\mathit{x}\right)=\mathrm{E}\left(\theta |\mathit{x}\right)$$E4

by taking partial derivative of the PESEL with respect to $a$ and setting it to $0$.

2.2 Weighted squared error loss function

The Bayesian estimator under the weighted squared error loss function, ${\delta }_{w2}^{\pi }\left(\mathit{x}\right)$, minimizes the posterior expected weighted squared error loss (PEWSEL) (see [1]), $\mathrm{E}\left[L_{w2}\left(\theta ,a\right)|\mathit{x}\right]$, that is,

$${\delta }_{w2}^{\pi }\left(\mathit{x}\right)=arg\underset{a\in \mathcal{A}}{min}\mathrm{E}\left[L_{w2}\left(\theta ,a\right)|\mathit{x}\right],$$E5

where $\mathcal{A}\left\{a\left(\mathit{x}\right):a\left(\mathit{x}\right)\in \left(-\infty ,\infty \right)\right\}$ is the action space, $a=a\left(\mathit{x}\right)\in \left(-\infty ,\infty \right)$ is an action (estimator),

$$L_{w2}\left(\theta ,a\right)=\frac{1}{{\theta }^2}{\left(\theta -a\right)}^2$$E6

is the weighted squared error loss function, and $\theta \in \left(-\infty ,\infty \right)$ is the unknown parameter of interest. The PEWSEL is easy to obtain (see [1]):

$$\mathit{\text{PEWSEL}}\left(\pi ,a|\mathit{x}\right)=\mathrm{E}\left[L_{w2}\left(\theta ,a\right)|\mathit{x}\right]=a^2\mathrm{E}\left(\frac{1}{{\theta }^2}|\mathit{x}\right)-2a\mathrm{E}\left(\frac{1}{\theta }|\mathit{x}\right)+1.$$E7

It is found in [1] that

$${\delta }_{w2}^{\pi }\left(\mathit{x}\right)=\frac{\mathrm{E}\left(\frac{1}{\theta }|\mathit{x}\right)}{\mathrm{E}\left(\frac{1}{{\theta }^2}|\mathit{x}\right)}$$E8

by taking partial derivative of the PEWSEL with respect to $a$ and setting it to $0$.

3. Bayesian estimation for θ ∈ (0,∞)

There are many hierarchical models where the parameter of interest is $\theta \in \mathrm{\Theta }=\left(0,\infty \right)$. As pointed out in the introduction, we should calculate and use the Bayesian estimator of the parameter $\theta$ under Stein’s loss function or the power-power loss function because they penalize gross overestimation and gross underestimation equally. We list several such hierarchical models as follows.

Model (a) (hierarchical normal and inverse gamma model). This hierarchical model has been investigated by [10, 16, 17]. Suppose that we observe $X_1,X_2,\dots ,X_n$ from the hierarchical normal and inverse gamma model:

$$\left\{\begin{array}{l}{X}_i\mid \theta {\displaystyle \stackrel{\mathrm{iid}}{\sim }}N\left(\mu ,\theta \right),i=1,2,\dots ,n,\\ \theta \sim \mathit{IG}\left(\alpha ,\beta \right),\end{array}\right.$$E9

where $-\infty <\mu <\infty$, $\alpha >0$, and $\beta >0$ are known constants, $\theta$ is the unknown parameter of interest, $N\left(\mu ,\theta \right)$ is the normal distribution, and $\mathit{IG}\left(\alpha ,\beta \right)$ is the inverse gamma distribution. It is worthy to note that the problem of finding the Bayesian rule under a conjugate prior is a standard problem and the problem is treated in almost every text on mathematical statistics. The idea of selecting an appropriate prior from the conjugate family was put forward by [18]. Specifically, Bayesian estimation of $\theta$ under the prior $\mathit{IG}\left(\alpha ,\beta \right)$ is studied in Example 4.2.5 (p. 236) of [17] and in Exercise 7.23 (p. 359) of [16]. However, they only calculate the Bayesian estimator with respect to $\mathit{IG}\left(\alpha ,\beta \right)$ prior under the squared error loss, ${\delta }_2^{\pi }\left(\mathit{x}\right)=\mathrm{E}\left(\theta |\mathit{x}\right)$.

Model (b) (hierarchical Poisson and gamma model). This hierarchical model has been investigated by [1, 16, 19, 20]. Suppose that $X_1,X_2,\dots ,X_n$ are observed from the hierarchical Poisson and gamma model:

$$\left\{\begin{array}{l}{X}_i\mid \theta {\displaystyle \stackrel{\mathrm{iid}}{\sim }}P\left(\theta \right),i=1,2,\dots ,n,\\ \theta \sim G\left(\alpha ,\beta \right),\end{array}\right.$$E10

where $\alpha >0$ and $\beta >0$ are hyperparameters to be determined, $P\left(\theta \right)$ is the Poisson distribution with an unknown mean $\theta >0$, and $G\left(\alpha ,\beta \right)$ is the gamma distribution with an unknown shape parameter $\alpha$ and an unknown rate parameter $\beta$. The gamma prior $G\left(\alpha ,\beta \right)$ is a conjugate prior for the Poisson model, so that the posterior distribution of $\theta$ is also a gamma distribution. The hierarchical Poisson and gamma model (10) has been considered in Exercise 4.32 (p. 196) of [4]. It has been shown that the marginal distribution of $X$ is a negative binomial distribution if $\alpha$ is a positive integer. The Bayesian estimation of $\theta$ under the gamma prior is studied in [19] and in Tables 3.3.1 (p. 121) and 4.2.1 (p. 176) of [1]. However, they only calculated the Bayesian posterior estimator of $\theta$ under the squared error loss function.

Model (c) (hierarchical normal and normal-inverse-gamma model). This hierarchical model has been investigated by [2, 21, 22]. Let the observations $X_1,X_2,\dots ,X_n$ be from the hierarchical normal and normal-inverse-gamma model:

$$\left\{\begin{array}{l}{X}_i\mid \left(\mu ,\theta \right){\displaystyle \stackrel{\mathrm{iid}}{\sim }}N\left(\mu ,\theta \right),i=1,2,\dots ,n,\\ \mu \mid \theta \sim N\left({\mu }_0,\theta /{\kappa }_0\right),\theta \sim \mathit{IG}\left(v_0/2,v_0{\sigma }_0^2/2\right),\end{array}\right.$$E11

where $-\infty <{\mu }_0<\infty$, ${\kappa }_0>0$, $v_0>0$, and ${\sigma }_0>0$ are known hyperparameters, $N\left(\mu ,\theta \right)$ is a normal distribution with an unknown mean $\mu$ and an unknown variance $\theta$, $\mu \mid \theta$ is $N({\mu }_0,$$\theta /{\kappa }_0)$ which is a normal distribution, and $\theta$ is $\mathit{IG}\left(v_0/2,v_0{\sigma }_0^2/2\right)$ which is an inverse gamma distribution. More specifically, with a joint conjugate prior $\pi \left(\mu ,\theta \right)\sim N-\mathit{IG}\left({\mu }_0,{\kappa }_0,v_0,{\sigma }_0^2\right)$, which is the normal-inverse-gamma distribution, the posterior distribution of $\theta$ was studied in Example 1.5.1 (p. 20) of [21] and Part I (pp. 69–70) of [22]. However, they did not provide any Bayesian posterior estimator of $\theta$. Moreover, the normal distribution with a normal-inverse-gamma prior which assumes that $\mu$ is unknown is more realistic than the normal distribution with an inverse gamma prior investigated by [10] which assumes that $\mu$ is known.

3.1 Stein’s loss function

3.1.1 One-dimensional case

The Bayesian estimator under Stein’s loss function, ${\delta }_s^{\pi }\left(\mathit{x}\right)$, minimizes the posterior expected Stein’s loss (PESL) (see [1, 10, 16]), $\mathrm{E}\left[L_s\left(\theta ,a\right)|\mathit{x}\right]$, that is,

$${\delta }_s^{\pi }\left(\mathit{x}\right)=arg\underset{a\in \mathcal{A}}{min}\mathrm{E}\left[L_s\left(\theta ,a\right)|\mathit{x}\right],$$E12

where $\mathcal{A}\left\{a\left(\mathit{x}\right):a\left(\mathit{x}\right)>0\right\}$ is the action space, $a=a\left(\mathit{x}\right)>0$ is an action (estimator),

$$L_s\left(\theta ,a\right)=\frac{a}{\theta }-1-log\frac{a}{\theta }$$E13

is Stein’s loss function, and $\theta >0$ is the unknown parameter of interest. The PESL is easy to obtain (see [10]):

$$\mathit{\text{PESL}}\left(\pi ,a|\mathit{x}\right)=\mathrm{E}\left[L_s\left(\theta ,a\right)|\mathit{x}\right]=a\mathrm{E}\left(\frac{1}{\theta }|\mathit{x}\right)-1-loga+\mathrm{E}\left(log\theta |\mathit{x}\right).$$E14

It is found in [10] that

$${\delta }_s^{\pi }\left(\mathit{x}\right)=\frac{1}{\mathrm{E}\left(\frac{1}{\theta }|\mathit{x}\right)}$$E15

by taking partial derivative of the PESL with respect to $a$ and setting it to $0$. The PESLs evaluated at the Bayesian estimators are (see [10])

$$\begin{array}{l}{\mathit{\text{PESL}}}_s\left(\pi ,\mathit{x}\right)={\left.\mathrm{E}\left[L_s\left(\theta ,a\right)|\mathit{x}\right]\right|}_{a={\delta }_s^{\pi }\left(\mathit{x}\right)},\\ {\mathit{\text{PESL}}}_2\left(\pi ,\mathit{x}\right)={\left.\mathrm{E}\left[L_s\left(\theta ,a\right)|\mathit{x}\right]\right|}_{a={\delta }_2^{\pi }\left(\mathit{x}\right)},\end{array}$$E16

where ${\delta }_2^{\pi }\left(\mathit{x}\right)=\mathrm{E}\left(\theta |\mathit{x}\right)$ is the Bayesian estimator under the squared error loss function.

For the variance parameter $\theta$ of the hierarchical normal and inverse gamma model (9), [10] recommends and analytically calculates the Bayesian estimator:

$${\delta }_s^{\pi }\left(\mathit{x}\right)=\frac{1}{{\alpha }^{\ast }{\beta }^{\ast }},$$E17

where

$${\alpha }^{\ast }=\alpha +\frac{n}{2}\text{and}{\beta }^{\ast }={\left[\frac{1}{\beta }+\frac{1}{2}\sum _{i=1}^n{\left(x_i-\mu \right)}^2\right]}^{-1},$$E18

with respect to $\mathit{IG}\left(\alpha ,\beta \right)$ prior under Stein’s loss function. This estimator minimizes the PESL. [10] also analytically calculates the Bayesian estimator,

$${\delta }_2^{\pi }\left(\mathit{x}\right)=\mathrm{E}\left(\theta |\mathit{x}\right)=\frac{1}{\left({\alpha }^{\ast }-1\right){\beta }^{\ast }},$$E19

with respect to $\mathit{IG}\left(\alpha ,\beta \right)$ prior under the squared error loss, and the corresponding PESL. [10] notes that

$$\mathrm{E}\left(log\theta |\mathit{x}\right)=-log{\beta }^{\ast }-\psi \left({\alpha }^{\ast }\right),$$E20

which is essential for the calculation of

$${\mathit{\text{PESL}}}_s\left(\pi ,\mathit{x}\right)=log{\alpha }^{\ast }-\psi \left({\alpha }^{\ast }\right)$$E21

and

$${\mathit{\text{PESL}}}_2\left(\pi ,\mathit{x}\right)=\frac{1}{{\alpha }^{\ast }-1}+log\left({\alpha }^{\ast }-1\right)-\psi \left({\alpha }^{\ast }\right),$$E22

depends on the digamma function $\psi \left(\cdot \right)$. Finally, the numerical simulations exemplify that ${\mathit{\text{PESL}}}_s\left(\pi ,\mathit{x}\right)$ and ${\mathit{\text{PESL}}}_2\left(\pi ,\mathit{x}\right)$ depend only on $\alpha$ and $n$ and do not depend on $\mu$, $\beta$, and $\mathit{x}$; the estimators ${\delta }_s^{\pi }\left(\mathit{x}\right)$ are unanimously smaller than the estimators ${\delta }_2^{\pi }\left(\mathit{x}\right)$; and ${\mathit{\text{PESL}}}_s\left(\pi ,\mathit{x}\right)$ are unanimously smaller than ${\mathit{\text{PESL}}}_2\left(\pi ,\mathit{x}\right)$.

For the hierarchical Poisson and gamma model (43), [20] first calculates the posterior distribution of $\theta$, $\pi \left(\theta |\mathit{x}\right)$, and the marginal pmf of $x$, $\pi \left(x\right)$, in Theorem 1 of their paper. [20] then calculates the Bayesian posterior estimators ${\delta }_s^{\pi }\left(\mathit{x}\right)$ and ${\delta }_2^{\pi }\left(\mathit{x}\right)$, and the PESLs ${\mathit{\text{PESL}}}_s\left(\pi ,\mathit{x}\right)$ and ${\mathit{\text{PESL}}}_2\left(\pi ,\mathit{x}\right)$, and they satisfy two inequalities. After that, the estimators of the hyperparameters of the model (10) by the moment method ${\alpha }_1\left(n\right)$ and ${\beta }_1\left(n\right)$ are summarized in Theorem 2 of their paper. Moreover, the estimators of the hyperparameters of the model (10) by the maximum likelihood estimator (MLE) method ${\alpha }_2\left(n\right)$ and ${\beta }_2\left(n\right)$ are summarized in Theorem 3 of their paper. Finally, the empirical Bayesian estimators of the parameter of the model (10) under Stein’s loss function by the moment method and the MLE method are summarized in Theorem 4 of their paper. In numerical simulations of [20], they have illustrated the two inequalities of the Bayesian posterior estimators and the PESLs, the moment estimators and the MLEs are consistent estimators of the hyperparameters, and the goodness of fit of the model to the simulated data. The numerical results indicate that the MLEs are better than the moment estimators when estimating the hyperparameters. Finally, [20] exploits the attendance data on 314 high school juniors from two urban high schools to illustrate their theoretical studies.

For the variance parameter $\theta$ of the normal distribution with a normal-inverse-gamma prior (11), [23] recommends and analytically calculates the Bayesian posterior estimator, ${\delta }_s^{\pi }\left(\mathit{x}\right)$, with respect to a conjugate prior $\mu \mid \theta \sim N({\mu }_0,$$\theta /{\kappa }_0)$, and $\theta \sim \mathit{IG}\left(v_0/2,v_0{\sigma }_0^2/2\right)$ under Stein’s loss function which penalizes gross overestimation and gross underestimation equally. This estimator minimizes the PESL. As comparisons, the Bayesian posterior estimator, ${\delta }_2^{\pi }\left(\mathit{x}\right)=\mathrm{E}\left(\theta |\mathit{x}\right)$, with respect to the same conjugate prior under the squared error loss function, and the PESL at ${\delta }_2^{\pi }\left(\mathbf{x}\right)$, are calculated. The calculations of ${\delta }_s^{\pi }\left(\mathit{x}\right)$, ${\delta }_2^{\pi }\left(\mathit{x}\right)$, ${\mathit{\text{PESL}}}_s\left(\pi ,\mathit{x}\right)$, and ${\mathit{\text{PESL}}}_2\left(\pi ,\mathit{x}\right)$ depend only on $\mathrm{E}\left(\theta |\mathit{x}\right)$, $\mathrm{E}\left({\theta }^{-1}|\mathit{x}\right)$, and $\mathrm{E}\left(log\theta |\mathit{x}\right)$. The numerical simulations exemplify their theoretical studies that the PESLs depend only on $v_0$ and $n$, but do not depend on ${\mu }_0$, ${\kappa }_0$, ${\sigma }_0$, and especially $\mathbf{x}$. The estimators ${\delta }_2^{\pi }\left(\mathbf{x}\right)$ are unanimously larger than the estimators ${\delta }_s^{\pi }\left(\mathit{x}\right)$, and ${\mathit{\text{PESL}}}_2\left(\pi ,\mathit{x}\right)$ are unanimously larger than ${\mathit{\text{PESL}}}_s\left(\pi ,\mathit{x}\right)$. Finally, [23] calculates the Bayesian posterior estimators and the PESLs of the monthly simple returns of the Shanghai Stock Exchange (SSE) Composite Index, which also exemplify the theoretical studies of the two inequalities of the Bayesian posterior estimators and the PESLs.

3.1.2 Multidimensional case

For estimating a covariance matrix which is assumed to be positive definite, many researchers exploit the multidimensional Stein’s loss function (e.g., see [2, 8, 24, 25, 26, 27, 28, 29, 30, 31]). The multidimensional Stein’s loss function (see [2]) is originally defined to estimate the $p\times p$ unknown covariance matrix $\mathrm{\Sigma }$ by $\widehat{\mathrm{\Sigma }}$ with the loss function:

$$L\left(\mathrm{\Sigma },\widehat{\mathrm{\Sigma }}\right)=\mathrm{tr}{\mathrm{\Sigma }}^{-1}\widehat{\mathrm{\Sigma }}-logdet{\mathrm{\Sigma }}^{-1}\widehat{\mathrm{\Sigma }}-p.$$E23

When $p=1$, the multidimensional Stein’s loss function reduces to

$$L_s\left({\sigma }^2,a\right)=\frac{a}{{\sigma }^2}-log\frac{a}{{\sigma }^2}-1,$$E24

which is in the form of (13), the one-dimensional Stein’s loss function.

3.2 Power-power loss function

The Bayesian estimator under the power-power loss function, ${\delta }_p^{\pi }\left(\mathit{x}\right)$, minimizes the posterior expected power-power loss (PEPL) (see [11]), $\mathrm{E}\left[L_p\left(\theta ,a\right)|\mathit{x}\right]$, that is,

$${\delta }_p^{\pi }\left(\mathit{x}\right)=arg\underset{a\in \mathcal{A}}{min}\mathrm{E}\left[L_p\left(\theta ,a\right)|\mathit{x}\right],$$E25

where $\mathcal{A}\left\{a\left(\mathit{x}\right):a\left(\mathit{x}\right)>0\right\}$ is the action space, $a=a\left(\mathit{x}\right)>0$ is an action (estimator),

$$L_p\left(\theta ,a\right)=\frac{a}{\theta }+\frac{\theta }{a}-2$$E26

is the power-power loss function, and $\theta >0$ is the unknown parameter of interest. The PEPL is easy to obtain (see [11]):

$$\mathit{\text{PEPL}}\left(\pi ,a|\mathit{x}\right)=\mathrm{E}\left[L_p\left(\theta ,a\right)|\mathit{x}\right]=a\mathrm{E}\left(\frac{1}{\theta }|\mathit{x}\right)+\frac{1}{a}\mathrm{E}\left(\theta |\mathit{x}\right)-2.$$E27

It is found in [11] that

$${\delta }_p^{\pi }\left(\mathit{x}\right)=\sqrt{\frac{\mathrm{E}\left(\theta |\mathit{x}\right)}{\mathrm{E}\left(\frac{1}{\theta }|\mathit{x}\right)}}$$E28

by taking partial derivative of the PEPL with respect to $a$ and setting it to $0$. The PEPLs evaluated at the Bayesian estimators are (see [11])

$$\begin{array}{l}{\mathit{\text{PEPL}}}_p\left(\pi ,\mathit{x}\right)={\left.\mathrm{E}\left[L_p\left(\theta ,a\right)|\mathit{x}\right]\right|}_{a={\delta }_p^{\pi }\left(\mathit{x}\right)},\\ {\mathit{\text{PEPL}}}_2\left(\pi ,\mathit{x}\right)={\left.\mathrm{E}\left[L_p\left(\theta ,a\right)|\mathit{x}\right]\right|}_{a={\delta }_2^{\pi }\left(\mathit{x}\right)}.\end{array}$$E29

The power-power loss function is proposed in [11], and it has all the seven properties proposed in his paper. More specifically, it penalizes gross overestimation and gross underestimation equally, is convex in its argument, and has balanced convergence rates or penalties for its argument too large and too small. Therefore, it is recommended for the positive restricted parameter space $\mathrm{\Theta }=\left(0,\infty \right)$.

4. Bayesian estimation for $\mathit{\theta }\mathbf{\in }\left(\mathbf{0},\mathbf{1}\right)$

There are some hierarchical models where the unknown parameter of interest is $\theta \in \mathrm{\Theta }=\left(0,1\right)$. As pointed out in the introduction, we should calculate and use the Bayesian estimator of the parameter $\theta$ under the power-log loss function or Zhang’s loss function because they penalize gross overestimation and gross underestimation equally. We list two such hierarchical models as follows.

Model (d) (beta-binomial model). This hierarchical model has been investigated by [1, 12, 13, 16, 32, 33]. Suppose that $X_1,X_2,\dots ,X_n$ are from the beta-binomial model:

$$\left\{\begin{array}{l}{X}_i\mid \theta {\displaystyle \stackrel{\mathrm{iid}}{\sim }}\mathit{Bin}\left(m,\theta \right),i=1,2,\dots ,n,\\ \theta \sim \mathit{Be}\left(\alpha ,\beta \right),\end{array}\right.$$E30

where $\alpha >0$ and $\beta >0$ are known constants, $m$ is a known positive integer, $\theta \in \left(0,1\right)$ is the unknown parameter of interest, $\mathit{Be}\left(\alpha ,\beta \right)$ is the beta distribution, and $\mathit{Bin}\left(m,\theta \right)$ is the binomial distribution. Specifically, Bayesian estimation of $\theta$ under the prior $\mathit{Be}\left(\alpha ,\beta \right)$ is studied in Example 7.2.14 (p. 324) of [16] and in Tables 3.3.1 (p. 121) and 4.2.1 (p. 176) of [1]. However, they only calculate the Bayesian estimator with respect to $\mathit{Be}\left(\alpha ,\beta \right)$ prior under the squared error loss, ${\delta }_2^{\pi }\left(\mathit{x}\right)=\mathrm{E}\left(\theta |\mathit{x}\right)$. Moreover, they only consider one observation. The beta-binomial model has been investigated recently. For instance, [32] uses the beta-binomial to draw the random removals in progressive censoring; [12, 13] use the beta-binomial to model some magazine exposure data for the monthly magazine Signature; [33] develops estimation procedure for the parameters of a zero-inflated overdispersed binomial model in the presence of missing responses.

Model (e) (beta-negative binomial model). This hierarchical model has been investigated by [1, 34]. Suppose that $X_1,X_2,\dots ,X_n$ are from the beta-negative binomial model:

$$\left\{\begin{array}{l}{X}_i\mid \theta {\displaystyle \stackrel{\mathrm{iid}}{\sim }}\mathit{NB}\left(m,\theta \right),i=1,2,\dots ,n,\\ \theta \sim \mathit{Be}\left(\alpha ,\beta \right),\end{array}\right.$$E31

where $\alpha >0$ and $\beta >0$ are known constants, $m$ is a known positive integer, $\theta \in \left(0,1\right)$ is the unknown parameter of interest, $\mathit{Be}\left(\alpha ,\beta \right)$ is the beta distribution, and $\mathit{NB}\left(m,\theta \right)$ is the negative binomial distribution. Specifically, Bayesian estimation of $\theta$ under the prior $\mathit{Be}\left(\alpha ,\beta \right)$ is studied in Tables 3.3.1 (p. 121) and 4.2.1 (p. 176) of [1]. However, he only calculates the Bayesian estimator with respect to $\mathit{Be}\left(\alpha ,\beta \right)$ prior under the squared error loss function, ${\delta }_2^{\pi }\left(\mathit{x}\right)=\mathrm{E}\left(\theta |\mathit{x}\right)$. Moreover, he only considers one observation.

4.1 Power-log loss function

A good loss function $L\left(\theta ,a\right)=L\left(a|\theta \right)={\left.L\left(x\right)\right|}_{x=a/\theta }$ for $\mathrm{\Theta }=\left(0,1\right)$ should have the six properties summarized in Table 1 (see Table 1 in [12]).

Properties	$L\left(x\right)$	$L\left(a|\mathit{\theta }\right)$
(a)	$L\left(x\right)\ge \mathit{0}$ for all $\mathit{0}<x<\frac{\mathit{1}}{\theta }$	$L\left(a|\theta \right)\ge \mathit{0}$ for all $\mathit{0}<a<\mathit{1}$
(b)	$L\left(\mathit{1}\right)=\mathit{0}$	$L\left(\theta |\theta \right)={\left.L\left(a|\theta \right)\right|}_{a=\theta }=\mathit{0}$
(c)	$L\left({\left(\frac{\mathit{1}}{\theta }\right)}^{-}\right)=\underset{x\to {\left(\frac{\mathit{1}}{\theta }\right)}^{-}}{lim}L\left(x\right)=\infty$	$L\left({\mathit{1}}^{-}|\theta \right)=\underset{a\to {\mathit{1}}^{-}}{lim}L\left(a|\theta \right)=\infty$
(d)	$L\left({\mathit{0}}^{+}\right)=\underset{x\to {\mathit{0}}^{+}}{lim}L\left(x\right)=\infty$	$L\left({\mathit{0}}^{+}|\theta \right)=\underset{a\to {\mathit{0}}^{+}}{lim}L\left(a|\theta \right)=\infty$
(e)	Convex in $x$ for all $\mathit{0}<x<\frac{\mathit{1}}{\theta }$	Convex in $a$ for all $\mathit{0}<a<\mathit{1}$
(f)	$L^{\text{'}}\left(\mathit{1}\right)={\left.\frac{\mathit{dL}\left(x\right)}{\mathit{dx}}\right|}_{x=\mathit{1}}=\mathit{0}$	${\left.\left[\frac{\partial }{\partial a}L\left(a|\theta \right)\right]\right|}_{a=\theta }=\mathit{0}$

Table 1.

(Table 1 in [12]) The six properties of a good loss function for $\mathit{\Theta }=\left(\mathit{0},\mathit{1}\right)$. $\mathit{0}<\theta <\mathit{1}$ is fixed.

In Table 1, property (a) means that any action $a$ of the parameter $\theta$ should incur a nonnegative loss. Property (b) means that when $x=a/\theta =1$, or $a=\theta$, that is, $a$ correctly estimates $\theta$, the loss is $0$. Property (c) means that when $x=a/\theta \to {\left(1/\theta \right)}^{-}$, that is, $a$ is moving away from $\theta$ and tends to $1^{-}$, it will incur an infinite loss. Property (d) means that when $x=a/\theta \to 0^{+}$, that is, $a$ is moving away from $\theta$ and tends to $0^{+}$, it will also incur an infinite loss. Properties (c) and (d) mean that the loss function will penalize gross overestimation and gross underestimation equally. Property (e) is useful in the proofs of some propositions of the minimaxity and the admissibility of the Bayesian estimator (see [1]). Property (f) means that $1$ and $\theta$ are the local extrema of $L\left(x\right)$ and $L\left(a|\theta \right)$, respectively. Property (f) also implies that $L\left(\theta +\mathit{\Delta a}|\theta \right)=o\left(\mathit{\Delta a}\right)$, that is, the loss incurred by an action $a=\theta +\mathit{\Delta a}$ near $\theta$ ($\mathit{\Delta a}\approx 0$), is very small compared to $\mathit{\Delta a}$.

Let

$$g_{\mathit{pl}}\left(x\right)=\frac{{\left(\frac{1}{\theta }-1\right)}^2}{\frac{1}{\theta }-x}-logx\text{and}{g}_{\mathit{pl}}\left(1\right)=\frac{1}{\theta }-1.$$E32

Define

$$L_{\mathit{pl}}\left(x\right)=g_{\mathit{pl}}\left(x\right)-g_{\mathit{pl}}\left(1\right)=\frac{{\left(\frac{1}{\theta }-1\right)}^2}{\frac{1}{\theta }-x}-logx-\left(\frac{1}{\theta }-1\right).$$E33

Thus

$$\begin{array}{cc}{L}_{\mathit{pl}}\left(\theta ,a\right)& =L_{\mathit{pl}}\left(a|\theta \right)={\left.L_{\mathit{pl}}\left(x\right)\right|}_{x=a/\theta }=\frac{{\left(\frac{1}{\theta }-1\right)}^2}{\frac{1}{\theta }-\frac{a}{\theta }}-log\frac{a}{\theta }-\left(\frac{1}{\theta }-1\right)\\ & =\frac{\theta {\left(\frac{1}{\theta }-1\right)}^2}{1-a}-loga+log\theta -\left(\frac{1}{\theta }-1\right).\end{array}$$E34

It is easy to check (see the supplement of [12]) that $L_{\mathit{pl}}\left(\theta ,a\right)=L_{\mathit{pl}}\left(a|\theta \right)={\left.L_{\mathit{pl}}\left(x\right)\right|}_{x=a/\theta }$, which is called the power-log loss function, satisfies all the six properties listed in Table 1. Consequently, the power-log loss function is a good loss function for $\mathrm{\Theta }=\left(0,1\right)$, and thus it is recommended for $\mathrm{\Theta }=\left(0,1\right)$.

We remark that the power-log loss function on $\mathrm{\Theta }=\left(0,1\right)$ is an analog of the power-log loss function on $\mathrm{\Theta }=\left(0,\infty \right)$, which is the popular Stein’s loss function.

The Bayesian estimator under the power-log loss function, ${\delta }_{\mathit{pl}}^{\pi }\left(\mathit{x}\right)$, minimizes the posterior expected power-log loss (PEPLL) (see [12]), $\mathrm{E}\left[L_{\mathit{pl}}\left(\theta ,a\right)|\mathit{x}\right]$, that is,

$${\delta }_{\mathit{pl}}^{\pi }\left(\mathit{x}\right)=arg\underset{a\in \mathcal{A}}{min}\mathrm{E}\left[L_{\mathit{pl}}\left(\theta ,a\right)|\mathit{x}\right],$$E35

where $\mathcal{A}\left\{a\left(\mathit{x}\right):a\left(\mathit{x}\right)\in \left(0,1\right)\right\}$ is the action space, $a=a\left(\mathit{x}\right)\in \left(0,1\right)$ is an action (estimator), $L_{\mathit{pl}}\left(\theta ,a\right)$ given by (34) is the power-log loss function, and $\theta \in \left(0,1\right)$ is the unknown parameter of interest. The PEPLL is easy to obtain (see [12]):

$$\mathit{\text{PEPLL}}\left(\pi ,a|\mathit{x}\right)=\mathrm{E}\left[L_{\mathit{pl}}\left(\theta ,a\right)|\mathit{x}\right]=\frac{E_1\left(\mathit{x}\right)}{1-a}-loga+E_2\left(\mathit{x}\right)-E_3\left(\mathit{x}\right)+1,$$E36

where

$$\begin{array}{l}{E}_1\left(\mathit{x}\right)=\mathrm{E}\left[{\theta }^{-1}{\left(1-\theta \right)}^2|\mathit{x}\right]>0,\\ E_2\left(\mathit{x}\right)=\mathrm{E}\left[log\theta |\mathit{x}\right]<0,\\ E_3\left(\mathit{x}\right)=\mathrm{E}\left[{\theta }^{-1}|\mathit{x}\right]>0.\end{array}$$E37

It is found in [12] that

$${\delta }_{\mathit{pl}}^{\pi }\left(\mathit{x}\right)=\frac{2+E_1\left(\mathit{x}\right)-\sqrt{E_1\left(\mathit{x}\right)\left(E_1\left(\mathit{x}\right)+4\right)}}{2}$$E38

by taking partial derivative of the PEPLL with respect to $a$ and setting it to $0$. The PEPLLs evaluated at the Bayesian estimators are (see [12])

$$\begin{array}{c}{\mathit{\text{PEPLL}}}_{\mathit{pl}}\left(\pi ,\mathit{x}\right)={\left.\mathrm{E}\left[L_{\mathit{pl}}\left(\theta ,a\right)|\mathit{x}\right]\right|}_{a={\delta }_{\mathit{pl}}^{\pi }\left(\mathit{x}\right)},\\ {\mathit{\text{PEPLL}}}_2\left(\pi ,\mathit{x}\right)={\left.\mathrm{E}\left[L_{\mathit{pl}}\left(\theta ,a\right)|\mathit{x}\right]\right|}_{a={\delta }_2^{\pi }\left(\mathit{x}\right)}.\end{array}$$E39

Finally, the numerical simulations and a real data example of some monthly magazine exposure data (see [35]) exemplify the theoretical studies of two size relationships about the Bayesian estimators and the PEPLLs in [12].

4.2 Zhang’s loss function

Zhang et al. [12] proposed six properties for a good loss function $L\left(\theta ,a\right)=L\left(a|\theta \right)={\left.L\left(x\right)\right|}_{x=a/\theta }$ on $\mathrm{\Theta }=\left(0,1\right)$. Apart from the six properties, [13] proposes the seventh property (balanced convergence rates or penalties for the argument too large and too small) for a good loss function on $\mathrm{\Theta }=\left(0,1\right)$. Moreover, the seven properties for a good loss function on $\mathrm{\Theta }=\left(0,1\right)$ are summarized in Table 1 of [13]. The explanations of the first six properties in Table 1 of [13] can be found in the previous subsection (see also [12]). In Table 1 of [13], property (g) (the seventh property) means that $L\left(k_1\left(\theta \right)\frac{1}{n}\right)$ and $L\left(\frac{1}{\theta }\left(1-\frac{1}{n}\right)\right)$ tend to $\infty$ at the same rate and $L\left(k_2\left(\theta \right)\frac{1}{n}|\theta \right)$ and $L\left(1-\frac{1}{n}|\theta \right)$ tend to $\infty$ at the same rate. In other words,

$$\underset{n\to \infty }{lim}\frac{L\left(k_1\left(\theta \right)\frac{1}{n}\right)}{L\left(\frac{1}{\theta }\left(1-\frac{1}{n}\right)\right)}=1\text{and}\underset{n\to \infty }{lim}\frac{L\left(k_2\left(\theta \right)\frac{1}{n}|\theta \right)}{L\left(1-\frac{1}{n}|\theta \right)}=1.$$E40

And they say that $L\left(k_1\left(\theta \right)\frac{1}{n}\right)$ and $L\left(\frac{1}{\theta }\left(1-\frac{1}{n}\right)\right)$ are asymptotically equivalent. Similarly, $L\left(k_2\left(\theta \right)\frac{1}{n}|\theta \right)$ and $L\left(1-\frac{1}{n}|\theta \right)$ are said to be asymptotically equivalent. They also say that $L\left(x\right)$ ($L\left(a|\theta \right)$) has balanced convergence rates or penalties for $x$ ($a$) too large and too small. It is worthy to note that $k_1\left(\theta \right)\frac{1}{n}\to 0$ and $\frac{1}{\theta }\left(1-\frac{1}{n}\right)\to \frac{1}{\theta }$ at the same order $O\left(\frac{1}{n}\right)$. Analogously, $k_2\left(\theta \right)\frac{1}{n}\to 0$ and $1-\frac{1}{n}\to 1$ at the same order $O\left(\frac{1}{n}\right)$. Finally, only when properties (c) and (d) hold, property (g) may hold.

Let

$$g_z\left(x\right)=\frac{1}{{\left(\frac{1}{\theta }-1\right)}^{2}x}+\frac{1}{\frac{1}{\theta }-x}\text{and}{g}_z\left(1\right)=\frac{1}{\theta {\left(\frac{1}{\theta }-1\right)}^2}.$$E41

Let

$$L_z\left(x\right)=g_z\left(x\right)-g_z\left(1\right)=\frac{1}{{\left(\frac{1}{\theta }-1\right)}^{2}x}+\frac{1}{\frac{1}{\theta }-x}-\frac{1}{\theta {\left(\frac{1}{\theta }-1\right)}^2}.$$E42

Thus

$$\begin{array}{c}{L}_z\left(\theta ,a\right)=L_z\left(a|\theta \right)={\left.L_z\left(x\right)\right|}_{x=a/\theta }=\frac{1}{{\left(\frac{1}{\theta }-1\right)}^2\frac{a}{\theta }}+\frac{1}{\frac{1}{\theta }-\frac{a}{\theta }}-\frac{1}{\theta {\left(\frac{1}{\theta }-1\right)}^2}\\ =\frac{\theta }{{\left(\frac{1}{\theta }-1\right)}^{2}a}+\frac{\theta }{1-a}-\frac{1}{\theta {\left(\frac{1}{\theta }-1\right)}^2}.\end{array}$$E43

It is easy to check (see the supplement of [13]) that $L_z\left(\theta ,a\right)=L_z\left(a|\theta \right)={\left.L_z\left(x\right)\right|}_{x=a/\theta }$, which is called Zhang’s loss function, satisfies all the seven properties listed in Table 1 of [13]. Consequently, Zhang’s loss function is a good loss function, and thus it is recommended for $\mathrm{\Theta }=\left(0,1\right)$.

The Bayesian estimator under Zhang’s loss function, ${\delta }_z^{\pi }\left(\mathit{x}\right)$, minimizes the posterior expected Zhang’s loss (PEZL) (see [13]), $\mathrm{E}\left[L_z\left(\theta ,a\right)|\mathit{x}\right]$, that is,

$${\delta }_z^{\pi }\left(\mathit{x}\right)=arg\underset{a\in \mathcal{A}}{min}\mathrm{E}\left[L_z\left(\theta ,a\right)|\mathit{x}\right],$$E44

where $\mathcal{A}\left\{a\left(\mathit{x}\right):a\left(\mathit{x}\right)\in \left(0,1\right)\right\}$ is the action space, $a=a\left(\mathit{x}\right)\in \left(0,1\right)$ is an action (estimator), $L_z\left(\theta ,a\right)$ given by (43) is Zhang’s loss function, and $\theta \in \left(0,1\right)$ is the unknown parameter of interest. The PEZL is easy to obtain (see [13]):

$$\mathit{\text{PEZL}}\left(\pi ,a|\mathit{x}\right)=\mathrm{E}\left[L_z\left(\theta ,a\right)|\mathit{x}\right]=\frac{E_1\left(\mathit{x}\right)}{a}+\frac{E_2\left(\mathit{x}\right)}{1-a}-E_3\left(\mathit{x}\right),$$E45

where

$$\begin{array}{cc}{E}_1\left(\mathit{x}\right)& =\mathrm{E}\left[\frac{{\theta }^3}{{\left(1-\theta \right)}^2}|\mathit{x}\right],\\ E_2\left(\mathit{x}\right)& =\mathrm{E}\left(\theta |\mathit{x}\right),\\ E_3\left(\mathit{x}\right)& =\mathrm{E}\left[\frac{\theta }{{\left(1-\theta \right)}^2}|\mathit{x}\right].\end{array}$$E46

It is found in [13] that

$${\delta }_z^{\pi }\left(\mathit{x}\right)=\frac{\sqrt{E_1\left(\mathit{x}\right)}}{\sqrt{E_1\left(\mathit{x}\right)}+\sqrt{E_2\left(\mathit{x}\right)}}$$E47

by taking partial derivative of the PEZL with respect to $a$ and setting it to $0$. The PEZLs evaluated at the Bayesian estimators are (see [13])

$$\begin{array}{cc}{\mathit{\text{PEZL}}}_z\left(\pi ,\mathit{x}\right)& ={\left.\mathrm{E}\left[L_z\left(\theta ,a\right)|\mathit{x}\right]\right|}_{a={\delta }_z^{\pi }\left(\mathit{x}\right)},\\ {\mathit{\text{PEZL}}}_2\left(\pi ,\mathit{x}\right)& ={\left.\mathrm{E}\left[L_z\left(\theta ,a\right)|\mathit{x}\right]\right|}_{a={\delta }_2^{\pi }\left(\mathit{x}\right)}.\end{array}$$E48

Zhang et al. [13] considers an example of some magazine exposure data for the monthly magazine Signature (see [12, 35]) and compares the numerical results with those of [12].

For the probability parameter $\theta$ of the beta-negative binomial model (31), [34] recommends and analytically calculates the Bayesian estimator ${\delta }_z^{\pi }\left(\mathit{x}\right)$, with respect to $\mathit{Be}\left(\alpha ,\beta \right)$ prior under Zhang’s loss function which penalizes gross overestimation and gross underestimation equally. This estimator minimizes the PEZL. They also calculate the usual Bayesian estimator ${\delta }_2^{\pi }\left(\mathit{x}\right)=\mathrm{E}\left(\theta |\mathit{x}\right)$ which minimizes the PESEL. Moreover, they also obtain the PEZLs evaluated at the two Bayesian estimators, ${\mathit{\text{PEZL}}}_z\left(\pi ,\mathit{x}\right)$ and ${\mathit{\text{PEZL}}}_2\left(\pi ,\mathit{x}\right)$. After that, they show two theorems about the estimators of the hyperparameters of the beta-negative binomial model (31) when m is known or unknown by the moment method (Theorem 1 in [34]) and the MLE method (Theorem 2 in [34]). Finally, the empirical Bayesian estimator of the probability parameter $\theta$ under Zhang’s loss function is obtained with the hyperparameters estimated by the moment method or the MLE method from the two theorems.

In the numerical simulations of [34], they have illustrated three things: the two inequalities of the Bayesian posterior estimators and the PEZLs, the moment estimators and the MLEs, which are consistent estimators of the hyperparameters, and the goodness of fit of the beta-negative binomial model to the simulated data. Numerical simulations show that the MLEs are better than the moment estimators when estimating the hyperparameters in terms of the goodness of fit of the model to the simulated data. However, the MLEs are very sensitive to the initial estimators, and the moment estimators are usually proved to be good initial estimators.

In the real data section of [34], they consider an example of some insurance claim data, which are assumed from the beta-negative binomial model (31). They consider four cases to fit the real data. In the first case, they assume that $m=6$ is known for illustrating purpose (of course, one can assume another known $m$ value). In the other three cases, they assume that $m$ is unknown, and they provide three approaches to handle this scenario. The first two approaches consider a range of m values, for instance, $m=1,2,\dots ,20$. The first approach is to maximize the log-likelihood function. The second approach is to maximize the p-value of the goodness of fit of the model (31) to the real data. The third approach is to determine the hyperparameters $\alpha$, $\beta$, and $m$ from Theorems 1 and 2 in [34] by the moment method and the MLE method, respectively, when $m$ is unknown. Four tables which show the number of claims, the observed frequencies, the expected probabilities, and the expected frequencies of the insurance claims data are provided to illustrate the four cases.

5. Inequalities among Bayesian posterior estimators

For the six loss functions, we have the corresponding six Bayesian estimators ${\delta }_{w2}^{\pi }\left(\mathit{x}\right)$, ${\delta }_{\mathit{pl}}^{\pi }\left(\mathit{x}\right)$, ${\delta }_s^{\pi }\left(\mathit{x}\right)$, ${\delta }_p^{\pi }\left(\mathit{x}\right)$, ${\delta }_2^{\pi }\left(\mathit{x}\right)$, and ${\delta }_z^{\pi }\left(\mathit{x}\right)$. Interestingly, for the six Bayesian estimators, we discover three strings of inequalities which are summarized in Theorem 1 (see Theorem 1 in [36]). To our surprise, an order between the two Bayesian estimators ${\delta }_{w2}^{\pi }\left(\mathit{x}\right)$ and ${\delta }_{\mathit{pl}}^{\pi }\left(\mathit{x}\right)$ on $\mathrm{\Theta }=\left(0,1\right)$ does not exist. It is worthy to note that the three strings of inequalities only depend on the loss functions. Moreover, the inequalities are independent of the chosen models, and the used priors provided the Bayesian estimators exist, and thus they exist in a general setting which makes them quite interesting.

In this section, we compare the six Bayesian estimators ${\delta }_{w2}^{\pi }\left(\mathit{x}\right)$, ${\delta }_{\mathit{pl}}^{\pi }\left(\mathit{x}\right)$, ${\delta }_s^{\pi }\left(\mathit{x}\right)$, ${\delta }_p^{\pi }\left(\mathit{x}\right)$, ${\delta }_2^{\pi }\left(\mathit{x}\right)$, and ${\delta }_z^{\pi }\left(\mathit{x}\right)$. The domains of the loss functions, the six Bayesian estimators, the PELs, and the smallest PELs are summarized in Table 2 (see Table 1 in [36]). The six PELs are PEWSEL, PEPLL, PESL, PEPL, PESEL, and PEZL. In Table 2, each Bayesian estimator minimizes some corresponding PEL. Furthermore, the smallest PEL is the PEL evaluated at the corresponding Bayesian estimator.

Domain	Bayesian estimators	PELs	Smallest PELs
$\mathrm{\Theta }=\left(-\infty ,\infty \right)$	${\delta }_{w2}^{\pi }\left(\mathit{x}\right)=\frac{E\left(\frac{1}{\theta }|\mathit{x}\right)}{E\left(\frac{1}{{\theta }^2}|\mathit{x}\right)}$	$\begin{array}{cc}\mathit{\text{PEWSEL}}\left(\pi ,a|\mathit{x}\right)& =E\left[L_{w2}\left(\theta ,a\right)|\mathit{x}\right]\\ & =E\left[\frac{1}{{\theta }^2}{\left(\theta -a\right)}^2|\mathit{x}\right]\end{array}$	$\begin{array}{c}{\mathit{\text{PEWSEL}}}_{w2}\left(\pi ,\mathit{x}\right)={\left.\mathit{\text{PEWSEL}}\left(\pi ,a|\mathit{x}\right)\right|}_{a={\delta }_{w2}^{\pi }\left(\mathit{x}\right)}\end{array}$
$\mathrm{\Theta }=\left(0,1\right)$	$\begin{array}{c}{\delta }_{\mathit{pl}}^{\pi }\left(\mathit{x}\right)\\ =\frac{2+E_1^{\mathit{pl}}\left(\mathit{x}\right)-\sqrt{E_1^{\mathit{pl}}\left(\mathit{x}\right)\left(E_1^{\mathit{pl}}\left(\mathit{x}\right)+4\right)}}{2}\\ \text{with} E_1^{\mathit{pl}}\left(\mathit{x}\right)=E\left[\frac{{\left(1-\theta \right)}^2}{\theta }|\mathit{x}\right]>0\end{array}$	$\begin{array}{cc}\mathit{\text{PEPLL}}\left(\pi ,a|\mathit{x}\right)& =E\left[L_{\mathit{pl}}\left(\theta ,a\right)|\mathit{x}\right]\\ & =E\left[\frac{\theta {\left(\frac{1}{\theta }-1\right)}^2}{1-a}-loga+log\theta -\frac{1}{\theta }+1|\mathit{x}\right]\end{array}$	$\begin{array}{c}{\mathit{\text{PEPLL}}}_{\mathit{pl}}\left(\pi ,\mathit{x}\right)={\left.\mathit{\text{PEPLL}}\left(\pi ,a|\mathit{x}\right)\right|}_{a={\delta }_{\mathit{pl}}^{\pi }\left(\mathit{x}\right)}\end{array}$
$\mathrm{\Theta }=\left(0,\infty \right)$	${\delta }_s^{\pi }\left(\mathit{x}\right)=\frac{1}{E\left(\frac{1}{\theta }|\mathit{x}\right)}$	$\begin{array}{cc}\mathit{\text{PESL}}\left(\pi ,a|\mathit{x}\right)& =E\left[L_s\left(\theta ,a\right)|\mathit{x}\right]\\ & =E\left[\frac{a}{\theta }-log\frac{a}{\theta }-1|\mathit{x}\right]\end{array}$	$\begin{array}{c}{\mathit{\text{PESL}}}_s\left(\pi ,\mathit{x}\right)={\left.\mathit{\text{PESL}}\left(\pi ,a|\mathit{x}\right)\right|}_{a={\delta }_s^{\pi }\left(\mathit{x}\right)}\end{array}$
$\mathrm{\Theta }=\left(0,\infty \right)$	${\delta }_p^{\pi }\left(\mathit{x}\right)=\sqrt{\frac{E\left(\theta |\mathit{x}\right)}{E\left(\frac{1}{\theta }|\mathit{x}\right)}}$	$\begin{array}{cc}\mathit{\text{PEPL}}\left(\pi ,a|\mathit{x}\right)& =E\left[L_p\left(\theta ,a\right)|\mathit{x}\right]\\ & =E\left[\frac{a}{\theta }+\frac{\theta }{a}-2|\mathit{x}\right]\end{array}$	$\begin{array}{c}{\mathit{\text{PEPL}}}_p\left(\pi ,\mathit{x}\right)={\left.\mathit{\text{PEPL}}\left(\pi ,a|\mathit{x}\right)\right|}_{a={\delta }_p^{\pi }\left(\mathit{x}\right)}\end{array}$
$\mathrm{\Theta }=\left(-\infty ,\infty \right)$	${\delta }_2^{\pi }\left(\mathit{x}\right)=E\left(\theta |\mathit{x}\right)$	$\begin{array}{cc}\mathit{\text{PESEL}}\left(\pi ,a|\mathit{x}\right)& =E\left[L_2\left(\theta ,a\right)|\mathit{x}\right]\\ & =E\left[{\left(\theta -a\right)}^2|\mathit{x}\right]\end{array}$	$\begin{array}{c}{\mathit{\text{PESEL}}}_2\left(\pi ,\mathit{x}\right)={\left.\mathit{\text{PESEL}}\left(\pi ,a|\mathit{x}\right)\right|}_{a={\delta }_2^{\pi }\left(\mathit{x}\right)}\end{array}$
$\mathrm{\Theta }=\left(0,1\right)$	$\begin{array}{cc}{\delta }_z^{\pi }\left(\mathit{x}\right)& =\frac{\sqrt{E_1^z\left(\mathit{x}\right)}}{\sqrt{E_1^z\left(\mathit{x}\right)}+\sqrt{E_2^z\left(\mathit{x}\right)}}\\ \text{with}{E}_1^z\left(\mathit{x}\right)& =E\left[\frac{{\theta }^3}{{\left(1-\theta \right)}^2}|\mathit{x}\right]\\ \text{and}{E}_2^z\left(\mathit{x}\right)& =E\left[\theta |\mathit{x}\right]\end{array}$	$\begin{array}{cc}\mathit{\text{PEZL}}\left(\pi ,a|\mathit{x}\right)& =E\left[L_z\left(\theta ,a\right)|\mathit{x}\right]\\ & =E\left[\frac{\theta }{{\left(\frac{1}{\theta }-1\right)}^{2}a}+\frac{\theta }{1-a}-\frac{1}{\theta {\left(\frac{1}{\theta }-1\right)}^2}|\mathit{x}\right]\end{array}$	$\begin{array}{c}{\mathit{\text{PEZL}}}_z\left(\pi ,\mathit{x}\right)={\left.\mathit{\text{PEZL}}\left(\pi ,a|\mathit{x}\right)\right|}_{a={\delta }_z^{\pi }\left(\mathit{x}\right)}\end{array}$

Table 2.

(Table 1 in [36]) The six Bayesian estimators, the PELs, and the smallest PELs.

It is easy to see that all the six loss functions are well defined on $\mathrm{\Theta }=\left(0,1\right)$, and thus all the six Bayesian estimators are well defined on $\mathrm{\Theta }=\left(0,1\right)$. There are only four loss functions defined on $\mathrm{\Theta }=\left(0,\infty \right)$, since the power-log loss function and Zhang’s loss function are only defined on $\mathrm{\Theta }=\left(0,1\right)$. Hence, only four Bayesian estimators are well defined on $\mathrm{\Theta }=\left(0,\infty \right)$. Moreover, only the weighted squared error loss function and the squared error loss function are defined on $\mathrm{\Theta }=\left(-\infty ,\infty \right)$, and therefore only two Bayesian estimators are well defined on $\mathrm{\Theta }=\left(-\infty ,\infty \right)$. Among the six Bayesian estimators, there exist three strings of inequalities which are summarized in the following theorem.

Theorem 1 (Theorem 1 in [36]). Assume the prior satisfies some regularity conditions so that the posterior expectations involved in the definitions of the six Bayesian estimators exist. Then for$\mathrm{\Theta }=\left(0,1\right)$, there exists a string of inequalities among the six Bayesian estimators:

$$max\left({\delta }_{w2}^{\pi }\left(\mathit{x}\right),{\delta }_{\mathit{pl}}^{\pi }\left(\mathit{x}\right)\right)\le {\delta }_s^{\pi }\left(\mathit{x}\right)\le {\delta }_p^{\pi }\left(\mathit{x}\right)\le {\delta }_2^{\pi }\left(\mathit{x}\right)\le {\delta }_z^{\pi }\left(\mathit{x}\right).$$E49

Moreover, for$\mathrm{\Theta }=\left(0,\infty \right)$, there exists a string of inequalities among the four Bayesian estimators:

$${\delta }_{w2}^{\pi }\left(\mathit{x}\right)\le {\delta }_s^{\pi }\left(\mathit{x}\right)\le {\delta }_p^{\pi }\left(\mathit{x}\right)\le {\delta }_2^{\pi }\left(\mathit{x}\right).$$E50

Finally, for$\mathrm{\Theta }=\left(-\infty ,\infty \right)$, there exists an inequality between the two Bayesian estimators:

$${\delta }_{w2}^{\pi }\left(\mathit{x}\right)\le {\delta }_2^{\pi }\left(\mathit{x}\right).$$E51

The proof of Theorem 1 exploits a key, important, and unified tool, the covariance inequality (see Theorem 4.7.9 (p. 192) in [16]), and the proof can be found in the supplement of [36].

It is worthy to note that the six Bayesian estimators and the six smallest PELs are all functions of $\pi$, $\mathit{x}$, and the loss function. Because there exists three strings of inequalities among the six Bayesian estimators, we would wonder whether there exists a string of inequalities among the six smallest PELs, in other words, ${\mathit{\text{PEWSEL}}}_{w2}\left(\pi ,\mathit{x}\right)$, ${\mathit{\text{PEPLL}}}_{\mathit{pl}}\left(\pi ,\mathit{x}\right)$, ${\mathit{\text{PESL}}}_s\left(\pi ,\mathit{x}\right)$, ${\mathit{\text{PEPL}}}_p\left(\pi ,\mathit{x}\right)$, ${\mathit{\text{PESEL}}}_2\left(\pi ,\mathit{x}\right)$, and ${\mathit{\text{PEZL}}}_z\left(\pi ,\mathit{x}\right)$. The answer to this question is no! The numerical simulations of the smallest PELs exemplify this fact (see [36]).

6. Conclusions and discussions

In this chapter, we have investigated six loss functions: the squared error loss function, the weighted squared error loss function, Stein’s loss function, the power-power loss function, the power-log loss function, and Zhang’s loss function. Now we give some suggestions on the conditions for using each of the six loss functions. It is worthy to note that among the six loss functions, the first two loss functions are defined on $\mathrm{\Theta }=\left(-\infty ,\infty \right)$ and they penalize overestimation and underestimation equally on $\left(-\infty ,\infty \right)$, and thus we recommend to use them when the parameter space is $\left(-\infty ,\infty \right)$. Moreover, the middle two loss functions are defined on $\mathrm{\Theta }=\left(0,\infty \right)$, and they penalize gross overestimation and gross underestimation equally on $\left(0,\infty \right)$, and thus we recommend to use them when the parameter space is $\left(0,\infty \right)$. In particular, if one prefers the loss function to have balanced convergence rates or penalties for its argument too large and too small, then we recommend to use the power-power loss function on $\left(0,\infty \right)$. Furthermore, the last two loss functions are defined on $\mathrm{\Theta }=\left(0,1\right)$, and they penalize gross overestimation and gross underestimation equally on $\left(0,1\right)$, and thus we recommend to use them when the parameter space is $\left(0,1\right)$. In particular, if one prefers the loss function to have balanced convergence rates or penalties for its argument too large and too small, then we recommend to use Zhang’s loss function on $\left(0,1\right)$.

For each one of the six loss functions, we can find a corresponding Bayesian estimator, which minimizes the corresponding posterior expected loss. Among the six Bayesian estimators, there exist three strings of inequalities summarized in Theorem 1 (see also Theorem 1 in [36]). However, a string of inequalities among the six smallest PELs does not exist.

We summarize three hierarchical models where the unknown parameter of interest is $\theta \in \mathrm{\Theta }=\left(0,\infty \right)$, that is, the hierarchical normal and inverse gamma model (9), the hierarchical Poisson and gamma model (10), and the hierarchical normal and normal-inverse-gamma model (11). In addition, we summarize two hierarchical models where the unknown parameter of interest is $\theta \in \mathrm{\Theta }=\left(0,1\right)$, that is, the beta-binomial model (30) and the beta-negative binomial model (31).

Now we give some suggestions on the selection of the hyperparameters. One way to select the hyperparameters is through the empirical Bayesian analysis, which relies on a conjugate prior modeling, where the hyperparameters are estimated from the observations and the “estimated prior” is then used as a regular prior in the later inference. The marginal distribution can then be used to recover the prior distribution from the observations. For empirical Bayesian analysis, two common methods are used to obtain the estimators of the hyperparameters, that is, the moment method and the MLE method. Numerical simulations show that the MLEs are better than the moment estimators when estimating the hyperparameters in terms of the goodness of fit of the model to the simulated data. However, the MLEs are very sensitive to the initial estimators, and the moment estimators are usually proved to be good initial estimators.

Acknowledgments

The research was supported by the Fundamental Research Funds for the Central Universities (2019CDXYST0016; 2018CDXYST0024), China Scholarship Council (201606055028), National Natural Science Foundation of China (11671060), and MOE project of Humanities and Social Sciences on the west and the border area (14XJC910001).

Conflict of interest

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.