Quantitative Structure-Activity Relationship Modeling and Bayesian Networks: Optimality of Naive Bayes Model

Abstract

Previously, computational drag design was usually based on simplified laws of molecular physics, used for calculation of ligand’s interaction with an active site of a protein-enzyme. However, currently, this interaction is widely estimated using some statistical properties of known ligand-protein complex properties. Such statistical properties are described by quantitative structure-activity relationships (QSAR). Bayesian networks can help us to evaluate stability of a ligand-protein complex using found statistics. Moreover, we are possible to prove optimality of Naive Bayes model that makes these evaluations simple and easy for practical realization. We prove here optimality of Naive Bayes model using as an illustration ligand-protein interaction.

1. Introduction

The determination within the chapter is based on a paper [1]. Bayes classifiers are broadly utilized right now for recognition, identification, and knowledge discovery. The fields of application are, for case, image processing, personalized medicine [2], chemistry (QSAR (quantitative structure-activity relationship) [3, 4]; see Figure 1). The especial importance Bayes Classifiers have in Medical Diagnostics and Bioinformatics. Cogent illustrations of this can be found in the work of Raymer and colleagues [5].

Figure 1.

Quantitative structure-activity relationship.

Let us give some example of using QSAR from papers [3, 4]:

“Molecular recognition and binding performed by proteins are the background of all biochemical processes in a living cell. In particular, the usual mechanism of drug function is effective binding and inhibition of activity of a target protein. Direct modeling of molecular interactions in protein-inhibitor complexes is the basis of modern computational drug design but is an extremely complicated problem. In the current paradigm, site similarity is recognized by the existence of chemically and spatially analogous regions from binding sites. We present a novel notion of binding site local similarity based on the analysis of complete protein environments of ligand fragments. Comparison of a query protein binding site (target) against the 3D structure of another protein (analog) in complex with a ligand enables ligand fragments from the analog complex to be transferred to positions in the target site, so that the complete protein environments of the fragment and its image are similar. The revealed environments are similarity regions and the fragments transferred to the target site are considered as binding patterns. The set of such binding patterns derived from a database of analog complexes forms a cloudlike structure (fragment cloud), which is a powerful tool for computational drug design.”

However, these Bayes classifiers have momentous property—by strange way the Naive Bayes classifier more often than not gives a decent and great description of recognition. More complex models of Bayes classifier cannot progress it significantly [1]. In the paper [6] creators clarify this exceptional property. In any case, they utilize a few suspicions (zero–one misfortune) which diminish all-inclusiveness and simplification of this proof. We allow in this chapter a common verification of Naive Bayes classifier optimality. The induction within the current chapter is comparative to [1]. The consequent attractive consideration of Naive Bayes classifier optimality problem was made in [7, 8]. Be that as it may, shockingly these papers do not incorporate any investigation of the past one [1].

We would like to prove Naive Bayes classifier optimality using QSAR terminology. Indeed, we use QSAR only for clearness; the proof is correct for any field of use of Naive Bayes classifier.

Let us define the essential issue that we attempt to unravel within the chapter. Assume that we have a set of states for a complex of ligand-active site of protein and a set of factors that characterize these states. For each state, we know the likelihood dispersion for each factor. In any case, we have no data of the approximate relationships of the factors. Presently, assume that we know factor values for some test of the state. What is the probability that this test corresponds to some state? It could be a commonplace issue of recognition over a condition of incomplete data.

In the simplest case, we can define two states for “ligand-active site of protein” complex. It is 0 (ligand is not bound to active site of protein) or 1 (ligand is not bound to active site of protein).

The next step is definition of factors (reliabilities below) that characterize strength of a bond for “ligand-active site of protein” complex. Let us grant an illustration of factors (reliabilities below) from experience of QSAR in papers [3, 4]:

“First, consider the protein 5 A°-environment A = {a₁, a₂,…a_N} of one ligand atom X in the analog protein, that is, all atoms from the binding site that are in the 5 A°-neighborhood of X. Suppose that the complete target binding site T consists of N′ atoms: T = {t₁, t₂,…t_N’} and there exists a subset T₀ $\subseteq$ T of size n (N′ ≥ n ≥ 4) such that n atoms from T₀ are similar to n atoms A₀ = {a_i1, a_i2,…a_in} $\subseteq$ A in their chemical types and spatial arrangement. The search for A₀ and T₀ is performed using a standard clique detection technique in the graph whose nodes represent pairs (a_i, t_i) of chemically equivalent atoms and edges reflect similarity of corresponding pairwise distances. If the search is successful, the optimal rigid motion superimposing matched protein atoms is applied both to the initial ligand atom X and its complete environment A (Figure 2(a) in [3]). The atoms are thus transferred to the target binding site. Then we extend the matching between A₀ and T₀ by such atom pairs (a_i,t_i) that a_i and t_i have the same chemical atom type in the coarser 10-type typification mentioned above, and the distance between t_i and the image a′_i of atom a_i is below a threshold. Next, a reliability value R, with 0 ≤ R ≤ 1, is assigned to the image X′ of X in the target site and reflects the similarity between the environments of X and its image X′. If the environments are highly similar (R ≈ 1) we expect that the position of X′ is the place where an atom with chemical type identical to X can be bound by the target, since the environment of X′ contains only atoms required for binding with no “alien” atoms. However, as illustrated in Figure 2(a) in [3], the analog site may contain extra binding atoms (shown on the lower side) that decrease the reliability value. In a simple form, the reliability R can be defined as the sum of the number of matched atoms divided by the total number of analog and target atoms in the 5 A°-environments of X and X′, respectively (Figure 2(b) in [3]):

Figure 2.

Function ${\Gamma }_{\mathit{opt}}\left(\alpha ,\beta ,\theta \right)$ : ${\left[0,1\right]}^3\to \left[0,1\right]$ .

R = 2n/(N + N′), using the notation presented above. In fact, we use a somewhat more complicated definition that accounts for the quality of spatial superposition of matched atoms and their distance from X′.”

We do not want to discuss here these definitions for these factors and states. Our purpose is not the demonstration of effectiveness of these definitions or effectiveness of QSAR. The interested reader can learn it from papers [3, 4] and references inside of these papers. As we said above, we use QSAR only for clearness; the proof is correct for any field of use of Naive Bayes classifier.

Let us consider the case when no relationships exist between reliabilities. In this case, the Naive Bayes model is a correct arrangement of the issue. We demonstrate in this chapter that for the case that we don’t know relationships between reliabilities even approximately—the Naive Bayes model is not correct, but ideal arrangement in a few senses. More point by point, we demonstrate that the Naive Bayes model gives minimal mean error over all conceivable models of relationship. We assume in this confirmation that all relationship models have the same likelihood. We think that this result can clarify the depicted over secretive optimality of Naive Bayes model.

The Chapter is built as described in the following statements. We grant correct numerical description of the issue for two states and two reliabilities in Section 2. We characterize our notations in Section 3. We define general form of conditional likelihood for all conceivable relationships of our reliabilities in Section 4. We characterize the limitations of the functions depicting the relationships in Section 5. We find the formula for an interval between two models of probability (correlation) in Section 6. We discover constraints for our fundamental functions in Section 7. We illuminate our primary issue; we demonstrate Naive Bayes model’s optimality for uniform distribution of all conceivable relationships in Section 8. We discover mean error between the Naive Bayes model and a genuine model for uniform distribution of all conceivable relationships in Section 9. We consider the case of more than two states and reliabilities in Section 10. We make conclusions in Section 11.

2. Definition of the task

Suppose that A is a state for “ligand-active site of protein” complex. It is 0 (ligand is not bound to active site of protein) or 1 (ligand is not bound to active site of protein). Accept that the $\mathit{\text{apriori}}$ likelihood $P\left(A\right)=P\left(A=1\right)$ is known, and indicate it by $\theta$ . Let $X_1,X_2$ be two reliability values (defined above), with values in a set $\left[0;1\right]$ . However, for generality, we will define $X_1,X_2$ in a set [−∞;+∞], but probability density to find $X_1,X_2$ in [−∞; 0] or [1;+∞] is equal to zero. We have the taking after data: $X_1=x_1$ and $X_2=x_2$ (gotten through estimation). Moreover, we have two functions, “classifiers,” which for given $x_1$ and $x_2$ give us

$$P\left(A=1/X_1=x_1\right)=P\left(A/x_1\right)\equiv \alpha ,$$

$$P\left(A=1/X_2=x_2\right)=P\left(A/x_2\right)\equiv \beta .$$

We want to find the likelihood

$$P\left(A=1/X_1=x_1,X_2=x_2\right)=P\left(A/x_1,x_2\right)$$

in terms of $\alpha ,\beta$ and $\theta$ . More particularly we wish to discover a function ${\Gamma }_{\mathit{opt}}\left(\alpha ,\beta ,\theta \right)$ which on the average is the most excellent estimation for $P\left(A/x_1,x_2\right)$ in a sense to be characterized expressly within the following consideration (see Figure 2).

3. Notation and preliminaries

${\rho }_{X_1,X_2}\left(x_1,x_2\right)$ —joint PDF (probability density function) for $X_1$ and $X_2$ .

${\rho }_{X_1,X_2/A}\left(x_1,x_2\right)=h\left(x_1,x_2\right)$ —joint PDF for $X_1$ and $X_2$ , known $A=1$ . In terms of $h\left(x_1,x_2\right)$ and $\theta$ , we can find $P\left(A/x_1,x_2\right)$ as follows:

$$P\left(A/x_1,x_2\right)=\frac{\mathit{\theta h}\left(x_1,x_2\right)}{\mathit{\theta h}\left(x_1,x_2\right)+\left(1-\theta \right)\overline{h}\left(x_1,x_2\right)},$$ E1

here

$\overline{h}\left(x_1,x_2\right)\equiv {\rho }_{X_1,X_2/\overline{A}}\left(x_1,x_2\right)$ —joint PDF for $X_1$ and $X_2$ , known $A=0$ .

We can find

$${\rho }_{X_1}\left(x_1\right)=\underset{-\infty }{\overset{+\infty }{\int }}{\rho }_{X_1,X_2}\left(x_1,x_2\right)d{x}_2,$$

$${\rho }_{X_2}\left(x_2\right)=\underset{-\infty }{\overset{+\infty }{\int }}{\rho }_{X_1,X_2}\left(x_1,x_2\right)d{x}_1,$$

$$h_1\left(x_1\right)\equiv {\rho }_{X_1/A}\left(x_1\right)=\underset{-\infty }{\overset{+\infty }{\int }}h\left(x_1,x_2\right)d{x}_2,$$

$$h_2\left(x_2\right)\equiv {\rho }_{X_2/A}\left(x_2\right)=\underset{-\infty }{\overset{+\infty }{\int }}h\left(x_1,x_2\right)d{x}_1,$$

$${\overline{h}}_1\left(x_1\right)\equiv {\rho }_{X_1/\overline{A}}\left(x_1\right)=\underset{-\infty }{\overset{+\infty }{\int }}\overline{h}\left(x_1,x_2\right)d{x}_2,$$

$${\overline{h}}_2\left(x_2\right)\equiv {\rho }_{X_2/\overline{A}}\left(x_2\right)=\underset{-\infty }{\overset{+\infty }{\int }}\overline{h}\left(x_1,x_2\right)d{x}_1.$$

4. Generic form of $\mathit{P}\left(\mathit{A}/{\mathit{x}}_{\mathbf{1}},{\mathit{x}}_{\mathbf{2}}\right)$

Let us define the function $g\left(x_1,x_2\right)$ and $\overline{g}\left(x_1,x_2\right)$

$$g\left(x_1,x_2\right)\equiv \frac{h\left(x_1,x_2\right)}{h_1\left(x_1\right)h_2\left(x_2\right)},$$

$$\overline{g}\left(x_1,x_2\right)\equiv \frac{\overline{h}\left(x_1,x_2\right)}{{\overline{h}}_1\left(x_1\right){\overline{h}}_2\left(x_2\right)}.$$

Let us say that if $X_1$ and $X_2$ are conditionally independent, i.e.,

$$\begin{array}{cc}h\left(x_1,x_2\right)& ={\rho }_{X_1{X}_2/A}\left(x_1,x_2\right)={\rho }_{X_1/A}\left(x_1\right){\rho }_{X_2/A}\left(x_2\right)\\ & =h_1\left(x_1\right)h_2\left(x_2\right),\end{array}$$

then

$$g\left(x_1,x_2\right)=\overline{g}\left(x_1,x_2\right)=1.$$

Let us define the following monotonously nondecreasing probability distribution functions:

$$H_1\left(x_1\right)\equiv \underset{-\infty }{\overset{x_1}{\int }}{h}_1\left(z\right)\mathit{dz},$$

$$H_2\left(x_2\right)\equiv \underset{-\infty }{\overset{x_2}{\int }}{h}_2\left(z\right)\mathit{dz},$$

$${\overline{H}}_1\left(x_1\right)\equiv \underset{-\infty }{\overset{x_1}{\int }}{\overline{h}}_1\left(z\right)\mathit{dz},$$

$${\overline{H}}_2\left(x_2\right)\equiv \underset{-\infty }{\overset{x_2}{\int }}{\overline{h}}_2\left(z\right)\mathit{dz}.$$

Take attention that since $H_1\left(x_1\right),H_2\left(x_2\right),{\overline{H}}_1\left(x_1\right)$ and ${\overline{H}}_2\left(x_2\right)$ are monotonous (At this point we can assume that $h_1\left(x_1\right),h_2\left(x_2\right),{\overline{h}}_1\left(x_1\right),{\overline{h}}_2\left(x_2\right)>0$ so that $H_1\left(x_1\right),H_2\left(x_2\right),{\overline{H}}_1\left(x_1\right)$ and ${\overline{H}}_2\left(x_2\right)$ are monotonously increasing. However, such limitation will be unnecessary as we will see within the following conclusion.), the inverse functions $H_1^{-1}\left(x_1\right),H_2^{-1}\left(x_2\right),{\overline{H}}_1^{-1}\left(x_1\right)$ and ${\overline{H}}_2^{-1}\left(x_2\right)$ must exist. As a result, we can characterize

$$J\left(a,b\right)\equiv g\left(H_1^{-1}\left(a\right),H_2^{-1}\left(b\right)\right),$$

$$\overline{J}\left(a,b\right)\equiv \overline{g}\left({\overline{H}}_1^{-1}\left(a\right),{\overline{H}}_2^{-1}\left(b\right)\right).$$

To be brief, let us use the following concise designation:

$$J\equiv J\left(H_1\left(x_1\right),H_2\left(x_2\right)\right)=g\left(H_1^{-1}\left(H_1\left(x_1\right)\right)\right),H_2^{-1}\left(H_2\left(x_2\right)\right)=g\left(x_1,x_2\right),$$

$$\overline{J}\equiv \overline{J}\left({\overline{H}}_1\left(x_1\right),{\overline{H}}_2\left(x_2\right)\right)=\overline{g}\left({\overline{H}}_1^{-1}\left({\overline{H}}_1\left(x_1\right)\right),{\overline{H}}_2^{-1}\left({\overline{H}}_2\left(x_2\right)\right)\right)=\overline{g}\left(x_1,x_2\right).$$

By the definition

$$h\left(x_1,x_2\right)=J{h}_1\left(x_1\right)h_2\left(x_2\right),$$ E2

$$\overline{h}\left(x_1,x_2\right)=\overline{J}{\overline{h}}_1\left(x_1\right){\overline{h}}_2\left(x_2\right).$$ E3

We currently obtain

$$h_1\left(x_1\right)\equiv {\rho }_{X_1/A}\left(x_1\right)=\frac{{\rho }_{X_1}\left(x_1\right)P\left(A/x_1\right)}{P\left(A\right)}=\frac{\alpha {\rho }_{X_1}\left(x_1\right)}{\theta },$$ E4

$$h_2\left(x_2\right)\equiv {\rho }_{X_2/A}\left(x_2\right)=\frac{{\rho }_{X_2}\left(x_2\right)P\left(A/x_2\right)}{P\left(A\right)}=\frac{\beta {\rho }_{X_2}\left(x_2\right)}{\theta },$$ E5

$${\overline{h}}_1\left(x_1\right)\equiv {\rho }_{X_1/\overline{A}}\left(x_1\right)=\frac{{\rho }_{X_1}\left(x_1\right)P\left(\overline{A}/x_1\right)}{P\left(\overline{A}\right)}=\frac{\left(1-\alpha \right){\rho }_{X_1}\left(x_1\right)}{1-\theta },$$ E6

$${\overline{h}}_2\left(x_2\right)\equiv {\rho }_{X_2/\overline{A}}\left(x_2\right)=\frac{{\rho }_{X_2}\left(x_2\right)P\left(\overline{A}/x_2\right)}{P\left(\overline{A}\right)}=\frac{\left(1-\alpha \right){\rho }_{X_2}\left(x_2\right)}{1-\theta }.$$ E7

As a result from Eqs. (2) and (3)

$$h\left(x_1,x_2\right)=J\frac{\mathit{\alpha \beta }{\rho }_{X_1}\left(x_1\right){\rho }_{X_2}\left(x_2\right)}{{\theta }^2},$$

$$\overline{h}\left(x_1,x_2\right)=\overline{J}\frac{\left(1-\alpha \right)\left(1-\beta \right){\rho }_{X_1}\left(x_1\right){\rho }_{X_2}\left(x_2\right)}{{\left(1-\theta \right)}^2}.$$

Now from Eq. (1)

$$\begin{array}{cc}P\left(A/x_1,x_2\right)& =\frac{\frac{J}{\theta }\mathit{\alpha \beta }{\rho }_{X_1}\left(x_1\right){\rho }_{X_2}\left(x_2\right)}{\frac{J}{\theta }\mathit{\alpha \beta }{\rho }_{X_1}\left(x_1\right){\rho }_{X_2}\left(x_2\right)+\frac{\overline{J}}{\left(1-\theta \right)}\left(1-\alpha \right)\left(1-\beta \right){\rho }_{X_1}\left(x_1\right){\rho }_{X_2}\left(x_2\right)}\\ & =\frac{\mathit{\alpha \beta }}{\mathit{\alpha \beta }+\frac{\overline{J}}{J}\frac{\theta }{1-\theta }\left(1-\alpha \right)\left(1-\beta \right)}.\end{array}$$ E8

Note, that for values of $J=\overline{J}=1$ (conditional independence of x₁ and x₂) equation (8) becomes the exact solution for the optimal model:

$$\Gamma \left(\alpha ,\beta ,\theta \right)=P\left(A/x_1,x_2\right).$$

5. Limitations for the functions $\mathit{J}\left(\mathit{a},\mathit{b}\right)$ and $\overline{\mathit{J}}\left(\mathit{a},\mathit{b}\right)$

We can write

$$h_1\left(x_1\right)=\underset{-\infty }{\overset{+\infty }{\int }}J\left(H_1\left(x_1\right),H_2\left(x_2\right)\right)h_1\left(x_1\right)h_2\left(x_2\right)d{x}_2.$$ E9

As a result

$$\begin{array}{c}1=\underset{-\infty }{\overset{+\infty }{\int }}J\left(H_1\left(x_1\right),H_2\left(x_2\right)\right)h_2\left(x_2\right)d{x}_2=\underset{0}{\overset{1}{\int }}J\left(H_1\left(x_1\right),H_2\left(x_2\right)\right)d{H}_2\left(x_2\right).\end{array}$$ E10

Thus, we obtain the following condition:

$$\underset{0}{\overset{1}{\int }}J\left(a,b\right)\mathit{db}=1,$$ E11

and similarly

$$\underset{0}{\overset{1}{\int }}J\left(a,b\right)\mathit{da}=1.$$ E12

Similarly, we can get

$$\begin{array}{c}\underset{0}{\overset{1}{\int }}\overline{J}\left(a,b\right)\mathit{da}=1\\ \underset{0}{\overset{1}{\int }}\overline{J}\left(a,b\right)\mathit{db}=1.\end{array}$$ E13

Obviously

$$J\left(a,b\right),\overline{J}\left(a,b\right)\ge 0,$$ E14

$$\underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}J\left(a,b\right)\mathit{\text{dadb}}=\underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}\overline{J}\left(a,b\right)\mathit{\text{dadb}}=1.$$ E15

All the solutions of Eqs. (11)–(15) together with (8) can define the set of all possible realizations of $P\left(A/x_1,x_2\right)$ .

Let us give some example of a solution of (11), (12) and (14), (15):

Let $\rho \left(x\right)$ be a function such that $\rho \left(x\right)\ge 0$ and $\underset{0}{\overset{1}{\int }}\rho \left(x\right)\mathit{dx}=1$

$$J\left(a,b\right)=\left\{\begin{array}{ll}\rho \left(a-b\right)& ,a\ge b\\ \rho \left(a-b+1\right)& ,a<b\end{array}\right),$$

satisfy Eqs. (11), (12), (14), and (15).

6. Definition of distance

We define the distance between the proposed approximation of $P\left(A/x_1,x_2\right)$ - $\Gamma \left(\alpha ,\beta ,\theta \right)$ and the actual function $P\left(A/x_1,x_2\right)$ as follows:

$$\begin{array}{c}‖\Gamma \left(\alpha ,\beta ,\theta \right)-P\left(A/x_1,x_2\right)‖=\int \underset{-\infty }{\overset{+\infty }{\int }}{\rho }_{X_1{X}_2}\left(x_1,x_2\right){\left[\Gamma \left(\alpha ,\beta ,\theta \right)-P\left(A/x_1,x_2\right)\right]}^{2}d{x}_{1}d{x}_2.\end{array}$$

Now we have from Eqs. (2) and (3) and Eqs. (4)–(7)

$$\begin{array}{cc}{\rho }_{X_1{X}_2}\left(x_1,x_2\right)& =\mathit{\theta h}\left(x_1,x_2\right)+\left(1-\theta \right)\overline{h}\left(x_1,x_2\right)\\ & =\mathit{\theta J}{h}_1\left(x_1\right)h_2\left(x_2\right)+\left(1-\theta \right)\overline{J}{\overline{h}}_1\left(x_1\right){\overline{h}}_2\left(x_2\right)\\ & =\left[J\frac{\mathit{\alpha \beta }}{\theta }+\overline{J}\frac{\left(1-\alpha \right)\left(1-\beta \right)}{\left(1-\theta \right)}\right]{\rho }_{X_1}\left(x_1\right){\rho }_{X_2}\left(x_2\right),\end{array}$$

$$\begin{array}{cc}‖\Gamma \left(\alpha ,\beta ,\theta \right)-P\left(A/x_1,x_2\right)‖& =\int \underset{-\infty }{\overset{+\infty }{\int }}{\rho }_{X_1}\left(x_1\right){\rho }_{X_2}\left(x_2\right)\left[J\frac{\mathit{\alpha \beta }}{\theta }+\overline{J}\frac{\left(1-\alpha \right)\left(1-\beta \right)}{\left(1-\theta \right)}\right]\times {\left(\Gamma \left(\alpha ,\beta ,\theta \right)-P\left(A/x_1,x_2\right)\right)}^{2}d{x}_{1}d{x}_2\\ & =\underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}\left[\frac{\mathit{J\alpha \beta }}{\theta }+\frac{\overline{J}\left(1-\alpha \right)\left(1-\beta \right)}{\left(1-\theta \right)}\right]\times {\left(\Gamma \left(\alpha ,\beta ,\theta \right)-P\left(A/x_1,x_2\right)\right)}^{2}d{F}_1\left(x_1\right)d{F}_2\left(x_2\right).\end{array}$$ E16

Here

$$F_1\left(x_1\right)={\int }_{-\infty }^{x_1}{\rho }_{X_1}\left(z\right)\mathit{dz},$$

$$F_2\left(x_2\right)={\int }_{-\infty }^{x_2}{\rho }_{X_2}\left(z\right)\mathit{dz}.$$

7. Constraints for basic functions

We will consider further all functions with arguments $1\ge F_1,F_2\ge 0$ , but not $x_1,x_2$ . We have six functions of $F_1,F_2$ , which define Eq. (16): $J,\overline{J},H_1,H_2,\alpha ,\beta$ . Let us to write the functions by help these functions (F₁, F₂) and find restrictions for these functions:

$$\alpha =P\left(A/x_1\right)=\theta h_1\left(x_1\right)/{\rho }_{X_1}\left(x_1\right)=\theta (d{H}_1/d{x}_1)/(d{F}_1/d{x}_1)=\theta \frac{d{H}_1}{d{F}_1}.$$

By the same way

$$\beta =\theta \frac{d{H}_2}{d{F}_2}.$$

We know that functions $H_1,F_1,H_2,F_2$ are cumulative distribution functions of $x_1$ , $x_2$ , correspondingly. These functions are monotonously nondecreasing and change from 0 to 1 from the definition of cumulative distribution functions. Therefore, we can conclude the following restraints for functions $H_1,H_2$ as functions of $F_1,F_2$ exist:

$$\begin{array}{c}{H}_1\left(1\right)=H_2\left(1\right)=1,\\ H_1\left(0\right)=H_2\left(0\right)=0,\end{array}$$

$$0\le \alpha =\theta \frac{d{H}_1}{d{F}_1},\beta =\theta \frac{d{H}_2}{d{F}_2}\le 1,$$

$$0\le \theta \le 1,$$

$$\begin{array}{cc}{\overline{H}}_1\left(x_1\right)& ={\int }_{-\infty }^{x_1}{\overline{h}}_1\left(x_1\right)d{x}_1\\ & ={\int }_{-\infty }^{x_1}\frac{\left(1-\alpha \right){\rho }_{X_1}\left(x_1\right)}{1-\theta }d{x}_1\\ & =\frac{1}{1-\theta }{\int }_{-\infty }^{x_1}{\rho }_{X_1}\left(x_1\right)d{x}_1-\frac{\theta }{1-\theta }{\int }_{-\infty }^{x_1}\frac{\alpha {\rho }_{X_1}\left(x_1\right)}{\theta }d{x}_1\\ =\frac{F_1}{1-\theta }-\frac{\theta }{1-\theta }{H}_1\left(x_1\right).\end{array}$$

By the same way

$${\overline{H}}_2\left(x_2\right)=\frac{F_2}{1-\theta }-\frac{\theta }{1-\theta }{H}_2\left(x_2\right),$$

$$\begin{array}{c}J\left(H_1\left(F_1\right),H_2\left(F_2\right)\right):J\left(H_1\left(F_1\right),H_2\left(F_2\right)\right)\ge 0\\ {\int }_0^{1}J\left(a,b\right)\mathit{db}=1\\ {\int }_0^{1}J\left(a,b\right)\mathit{da}=1,\end{array}$$

$$\begin{array}{c}\overline{J}\left({\overline{H}}_1\left(F_1\right),{\overline{H}}_2\left(F_2\right)\right):\overline{J}\left({\overline{H}}_1\left(F_1\right),{\overline{H}}_2\left(F_2\right)\right)\ge 0\\ {\int }_0^1\overline{J}\left(a,b\right)\mathit{db}=1\\ {\int }_0^1\overline{J}\left(a,b\right)\mathit{da}=1,\end{array}$$

$$P\left(A/x_1,x_2\right)=\frac{\frac{\mathit{J\alpha \beta }}{\theta }}{\frac{\mathit{J\alpha \beta }}{\theta }+\frac{\overline{J}\left(1-\alpha \right)\left(1-\beta \right)}{1-\theta }}.$$ E17

8. Optimization

We shall find the best approximation of $\Gamma \left(\alpha ,\beta ,\theta \right)$ as follows:

$$\mathit{mi}{n}_{\Gamma \left(\alpha ,\beta ,\theta \right)}E\left[‖\Gamma \left(\alpha ,\beta ,\theta \right)-P\left(A/x_1,x_2\right)‖\right]\to \Gamma \left(\alpha ,\beta ,\theta \right),$$

where the expected value (or expectation or mathematical expectation or mean or the first moment) $E\left[\dots \right]$ is taken with respect to the joint PDF of possible realizations of $J,\overline{J},\alpha ,\beta ,H_1,H_2$ for given $F_1$ and $F_2$ .

For the sake of brevity, we denote

$$C\equiv \frac{\mathit{J\alpha \beta }}{\theta }+\frac{\overline{J}\left(1-\alpha \right)\left(1-\beta \right)}{1-\theta }$$

$$D\equiv \frac{\mathit{J\alpha \beta }}{\theta }.$$

Then from Eqs. (17) and (16)

$$\begin{array}{cc}‖\Gamma \left(\alpha ,\beta ,\theta \right)-P\left(A/x_1,x_2\right)‖& =\underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}C{\left(\Gamma \left(\alpha ,\beta ,\theta \right)-D/C\right)}^{2}d{F}_{1}d{F}_2\\ & =\underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}d{F}_{1}d{F}_2\left[D^{2}C+{\Gamma }^2\left(\alpha ,\beta ,\theta \right)C-2\Gamma \left(\alpha ,\beta ,\theta \right)D\right].\end{array}$$ E18

Thus

$$\begin{array}{c}\mathit{mi}{n}_{\Gamma \left(\alpha ,\beta ,\theta \right)}E\left[‖\Gamma \left(\alpha ,\beta ,\theta \right)-P\left(A/x_1,x_2\right)‖\right]\\ =\mathit{mi}{n}_{\Gamma \left(\alpha ,\beta ,\theta \right)}E\left[\underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}d{F}_{1}d{F}_2\left[D^{2}C+{\Gamma }^2\left(\alpha ,\beta ,\theta \right)C-2\Gamma \left(\alpha ,\beta ,\theta \right)D\right]\right]\\ =\mathit{mi}{n}_{\Gamma \left(\alpha ,\beta ,\theta \right)}E\left[\underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}d{F}_{1}d{F}_2\left[D^{2}C\right]\right]\\ +\mathit{mi}{n}_{\Gamma \left(\alpha ,\beta ,\theta \right)}E\left[\underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}d{F}_{1}d{F}_2\left[{\Gamma }^2\left(\alpha ,\beta ,\theta \right)C-2\Gamma \left(\alpha ,\beta ,\theta \right)D\right]\right]\\ =\mathit{\text{Const}}+\mathit{mi}{n}_{\Gamma \left(\alpha ,\beta ,\theta \right)}E\left[\underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}d{F}_{1}d{F}_2\left[{\Gamma }^2\left(\alpha ,\beta ,\theta \right)C-2\Gamma \left(\alpha ,\beta ,\theta \right)D\right]\right].\end{array}$$ E19

It remains to calculate the expected value in Eq. (19).

We have by obvious assumptions

$$\begin{array}{cc}{\rho }_{J,\overline{J},\alpha ,\beta ,H_1{H}_2/F_1,F_2}\left(J,\overline{J},\alpha ,\beta ,H_1,H_2/F_1,F_2\right)& ={\rho }_{J/H_1,H_2}\left(J/H_1,H_2\right){\rho }_{\overline{J}/{\overline{H}}_1,{\overline{H}}_2}\left(\overline{J}/{\overline{H}}_1,{\overline{H}}_2\right){\rho }_{\alpha /F_1}\left(\alpha /F_1\right)\\ & \times {\rho }_{H_1/\alpha F_1}\left(H_1/\alpha ,F_1\right){\rho }_{\beta /F_2}\left(\beta /F_2\right){\rho }_{H_2/\beta ,F_2}\left(H_2/\beta ,F_2\right).\end{array}$$ E20

Lemma 1

$$E\left[J\left(a,b\right)\right]={\int }_0^{+\infty }{\rho }_{J\left(a,b\right)/a,b}\left(J\left(a,b\right)/a,b\right)J\left(a,b\right)\mathit{dJ}=1,$$

$$E\left[\overline{J}\left(a,b\right)\right]={\int }_0^{+\infty }{\rho }_{\overline{J}\left(a,b\right)/a,b}\left(\overline{J}\left(a,b\right)/a,b\right)\overline{J}\left(a,b\right)d\overline{J}=1.$$

Proof: We can take into the consideration the function ${\rho }_{J\left(a,b\right)/a,b}$ . The domain of the function $J\left(a,b\right)$ is square $0\le a,b\le 1$ . By dividing this square into small squares $\left(i,j\right)$ , we can get sampling of the function $J$ . Then, on every square $i,j$ , we can define the value of the function $J_{\mathit{ij}}$ . We can write the following restraints for function $J\left(\ast \ast \ast \right)$ :

$$J_{\mathit{ij}}\ge 0,$$

$$\frac{1}{N}\sum _{i=1}^N{J}_{\mathit{ij}}=1,$$

$$\frac{1}{N}\sum _{j=1}^N{J}_{\mathit{ij}}=1.$$

Here $i=1,\dots ,N$ , $j=1,\dots ,N$ .

All matrixes $\left(J_{\mathit{ij}}\right)$ that satisfy the above conditions have the same probability. So we can define probability density function

$$\rho \left(J_{11},\dots ,J_{\mathit{ij}},\dots ,J_{\mathit{NN}}\right).$$

This density function should be symmetric according to transpositions of columns and rows of the matrix $\left(J_{\mathit{ij}}\right)$ , because the density function has the same probability for all matrixes $\left(J_{\mathit{ij}}\right)$ that satisfy the above conditions. Indeed, these conditions are also symmetric according to transpositions of columns and rows of matrix $\left(J_{\mathit{ij}}\right)$ . From symmetry conditions that define this function $\left(\rho \right)$ according to transpositions of columns and rows of matrix $\left(J_{\mathit{ij}}\right)$ , it is possible to conclude that this function $\left(\rho \right)$ also does not transform according to these transpositions.

We can consider function ${\rho }_{u/\mathit{ij}}\left(u/\mathit{ij}\right)$ that is a discretization of the function ${\rho }_{J\left(a,b\right)/a,b}\left(J\left(a,b\right)/a,b\right)$ :

$${\rho }_{u/\mathit{ij}}\left(u/\mathit{ij}\right)=\int \dots {\int }_0^{+\infty }\rho \left(J_{11},\dots J_{\mathit{nk}},\dots J_{\mathit{ij}}=u,\dots ,J_{\mathit{NN}}\right){\prod }_{\left(\mathit{lm}\right)\ne \left(\mathit{ij}\right)}d{J}_{\mathit{lm}}.$$

We can transpose columns and rows $\left(J_{\mathit{ij}}\right)$ in such a way that element $J_{\mathit{ij}}$ will be replaced by the other element $J_{\mathit{nk}}$ , and after it the function $\rho \left(J_{11},\dots \right)$ will not be transformed. So from the above equation, we can get

$${\rho }_{u/\mathit{ij}}\left(u/\mathit{ij}\right)=\int \dots {\int }_0^{+\infty }\rho \left(J_{11},\dots J_{\mathit{nk}},\dots J_{\mathit{ij}}=u,\dots ,J_{\mathit{NN}}\right){\prod }_{\left(\mathit{lm}\right)\ne \left(\mathit{nk}\right)}d{J}_{\mathit{lm}}={\rho }_{u/\mathit{nk}}\left(u/\mathit{nk}\right).$$

From this equation we can conclude that ${\rho }_{u/\mathit{ij}}\left(u/\mathit{ij}\right)$ does not depend on $\mathit{ij}$ so ${\rho }_{J/a,b}\left(J/a,b\right)$ does not depend on $\mathit{ab}$ and

$${\rho }_{J/a,b}\left(J/a,b\right)={\rho }_J\left(J\right),$$

and

$$E\left[J\left(a,b\right)\right]={\int }_0^{+\infty }{\rho }_J\left(J\right)\mathit{JdJ}=\mathit{\text{Const}},$$ E21

From

$$\underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}J\left(a,b\right)\mathit{\text{dadb}}=1,$$

we can conclude that

$$\underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}E\left[J\left(a,b\right)\right]\mathit{\text{dadb}}=1.$$

So we can obtain that $\mathit{\text{Const}}=1$ in Eq. (21).

Lemma 2: Probability distribution functions $\alpha$ and $\beta$ do not depend on $F_1$ and $F_2$ :

$${\rho }_{\alpha /F_1}\left(\alpha /F_1\right)={\rho }_{\alpha }\left(\alpha \right),$$

$${\rho }_{\beta /F_2}\left(\beta /F_2\right)={\rho }_{\beta }\left(\beta \right).$$

Proof: Let us make sampling of the function $\alpha \left(F_1\right)$ by dividing the domain of this function $F_1,\left[0,1\right]$ on intervals of $1/N,N\gg 1$ . Then restriction conditions for ${\alpha }_k,k=1,\dots ,N$

$$0\le {\alpha }_k\le 1,$$

$$\frac{1}{N}\sum _{k=1}^N{\alpha }_k=\underset{0}{\overset{1}{\int }}\mathit{\theta d}{H}_1\left(F_1\right)d{F}_{1}d{F}_1=\theta .$$

All columns $\left({\alpha }_k\right)$ that are satisfied by these conditions have equal probability. We can consider respective function $\rho \left({\alpha }_1,\dots ,{\alpha }_k,\dots ,{\alpha }_l,\dots ,{\alpha }_N\right)$ . From symmetry conditions that define this function according to transpositions ${\alpha }_k\to {\alpha }_l$ , function $\rho \left({\alpha }_1,\dots ,{\alpha }_k,\dots ,{\alpha }_l,\dots ,{\alpha }_N\right)$ also does not transform according to such transpositions. As a result, it is possible to write

$$\begin{array}{cc}{\rho }_k\left(u\right)& =\underset{0}{\overset{1}{\int }}\rho \left({\alpha }_1,\dots ,{\alpha }_k=u,\dots ,{\alpha }_l,\dots ,{\alpha }_N\right){\prod }_{n\ne k}d{\alpha }_n\\ & =\underset{0}{\overset{1}{\int }}\rho \left({\alpha }_1,\dots ,{\alpha }_k,\dots ,{\alpha }_l=u,\dots ,{\alpha }_N\right){\prod }_{n\ne l}d{\alpha }_n\\ & ={\rho }_l\left(u\right).\end{array}$$

From this equation, we can conclude that function ${\rho }_{\alpha /F_1}\left(\alpha /F_1\right)$ does not depend on $F_1$ :

$${\rho }_{\alpha /F_1}\left(\alpha /F_1\right)={\rho }_{\alpha }\left(\alpha \right).$$

From (20) we obtain

$$\begin{array}{cc}E\left[{\Gamma }^2\left(\alpha ,\beta ,\theta \right)C-2\Gamma \left(\alpha ,\beta ,\theta \right)D\right]& =\underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}{\rho }_{\alpha }\left(\alpha \right){\rho }_{\beta }\left(\beta \right)\mathit{\text{dαdβ}}\\ & \times \underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}{\rho }_{H_1/\alpha ,F_1}\left(H_1/\alpha ,F_1\right){\rho }_{H_2/\beta ,F_2}\left(H_2/\beta ,F_2\right)d{H}_{1}d{H}_2{\int }_0^{\infty }{\int }_0^{\infty }{\rho }_J\left(J\right){\rho }_{\overline{J}}\left(\overline{J}\right)\\ & \times \left[{\Gamma }^2\left(\alpha ,\beta ,\theta \right)\left[\frac{\mathit{J\alpha \beta }}{\theta }+\frac{\overline{J}\left(1-\alpha \right)\left(1-\beta \right)}{1-\theta }\right]-2\Gamma \left(\alpha ,\beta ,\theta \right)\frac{\mathit{J\alpha \beta }}{\theta }\right]\mathit{dJ}d\overline{J}\\ & =\underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}{\rho }_{\alpha }\left(\alpha \right){\rho }_{\beta }\left(\beta \right)\mathit{\text{dαdβ}}\\ & \times \left[{\Gamma }^2\left(\alpha ,\beta ,\theta \right)\left[\frac{E\left[J\right]\mathit{\alpha \beta }}{\theta }+\frac{E\left[\overline{J}\right]\left(1-\alpha \right)\left(1-\beta \right)}{1-\theta }\right]-2\Gamma \left(\alpha ,\beta ,\theta \right)\frac{E\left[J\right]\mathit{\alpha \beta }}{\theta }\right].\end{array}$$

Let us define

$$\overline{C}=\frac{\mathit{\alpha \beta }}{\theta }+\frac{\left(1-\alpha \right)\left(1-\beta \right)}{1-\theta },$$

$$\overline{D}=\frac{\mathit{\alpha \beta }}{\theta }.$$

By Lemma 1, $E\left[J\right]=E\left[\overline{J}\right]=1$ . Hence

$$\begin{array}{c}E\left[{\Gamma }^2\left(\alpha ,\beta ,\theta \right)C-2\Gamma \left(\alpha ,\beta ,\theta \right)D\right]=\underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}\left[{\Gamma }^2\left(\alpha ,\beta ,\theta \right)\overline{C}-2\Gamma \left(\alpha ,\beta ,\theta \right)\overline{D}\right]{\rho }_{\alpha }\left(\alpha \right){\rho }_{\beta }\left(\beta \right)\mathit{\text{dαdβ}}.\end{array}$$

It remains to find

$$\mathit{mi}{n}_{\Gamma \left(\alpha ,\beta ,\theta \right)}\underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}d{F}_{1}d{F}_2\underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}\mathit{\text{dαdβ}}{\rho }_{\alpha }\left(\alpha \right){\rho }_{\beta }\left(\beta \right)\left[{\Gamma }^2\left(\alpha ,\beta ,\theta \right))\overline{C}-2\Gamma \left(\alpha ,\beta ,\theta \right)\overline{D}\right].$$ E22

Since

$${\rho }_{\alpha }\left(\alpha \right){\rho }_{\beta }\left(\beta \right)\ge 0,$$

if the expression in square brackets is minimized at each point, then the whole integral in Eq. (22) is minimized. Thus, we may proceed as follows:

$$\frac{\partial }{\partial \Gamma }\left[{\Gamma }^2\left(\alpha ,\beta ,\theta \right)\overline{C}-2\Gamma \left(\alpha ,\beta ,\theta \right)\overline{D}\right]=2\Gamma \left(\alpha ,\beta ,\theta \right)\overline{C}-2\overline{D}=0.$$

Hence the optimum $\Gamma \left(\alpha ,\beta ,\theta \right)$ is given by

$${\Gamma }_{\mathit{opt}}\left(\alpha ,\beta ,\theta \right)=\frac{\overline{D}}{\overline{C}}=\frac{\frac{\mathit{\alpha \beta }}{\theta }}{\frac{\mathit{\alpha \beta }}{\theta }+\frac{\left(1-\alpha \right)\left(1-\beta \right)}{1-\theta }}.$$

9. Mean distance between the proposed approximation of $\mathit{P}\left(\mathit{A}/{\mathit{x}}_{\mathbf{1}},{\mathit{x}}_{\mathbf{2}}\right)$ − $\mathit{\Gamma }\left(\mathit{\alpha },\mathit{\beta },\mathit{\theta }\right)$ and the actual function $\mathit{P}\left(\mathit{A}/{\mathit{x}}_{\mathbf{1}},{\mathit{x}}_{\mathbf{2}}\right)$

The mean distance from (18) is

$$\begin{array}{cc}\mathit{DIS}& =E\left[‖\Gamma \left(\alpha ,\beta ,\theta \right)-P\left(A/x_1,x_2\right)‖\right]\\ & ={\int }_0^1\underset{0}{\overset{1}{\int }}{\rho }_{\alpha }\left(\alpha \right){\rho }_{\beta }\left(\beta \right)\mathit{\text{dαdβ}}\left[{\Gamma }^2\left(\alpha ,\beta ,\theta \right)\overline{C}-2\Gamma \left(\alpha ,\beta ,\theta \right)\overline{D}\right]+\mathit{\text{Const}},\end{array}$$

where $\mathit{\text{Const}}$ in this equation is defined by

$$\mathit{\text{Const}}=E\left[\underset{-\infty }{\overset{+\infty }{\int }}\underset{-\infty }{\overset{+\infty }{\int }}{\rho }_{X_1{X}_2}\left(x_1,x_2\right){\left[P\left(A/x_1,x_2\right)\right]}^{2}d{x}_{1}d{x}_2\right].$$

From this equation we can find boundaries of the $\mathit{\text{Const}}$ . From $0\le P\left(A/x_1,x_2\right)\le 1$ we can conclude

$$\mathit{\text{Const}}\le E\left[\underset{-\infty }{\overset{+\infty }{\int }}\underset{-\infty }{\overset{+\infty }{\int }}{\rho }_{X_1,X_2,}\left(x_1,x_2\right)P\left(A/x_1,x_2\right)d{x}_{1}d{x}_2\right]=E\left[\theta \right]=\theta .$$

The second condition is

$$\begin{array}{l}0\le E\left[{\int }_{-\infty }^{+\infty }\underset{-\infty }{\overset{+\infty }{\int }}{\rho }_{X_1,X_2}\left(x_1,x_2\right){\left[P\left(A/x_1,x_2\right)-\theta \right]}^{2}d{x}_{1}d{x}_2\right]\\ =E\left[{\int }_{-\infty }^{+\infty }\underset{-\infty }{\overset{+\infty }{\int }}{\rho }_{X_1,X_2}\left(x_1,x_2\right)\left[P{\left(A/x_1,x_2\right)}^2+{\theta }^2-2P\left(A/x_1,x_2\right)\theta \right]d{x}_{1}d{x}_2\right]\\ =E\left[{\int }_{-\infty }^{+\infty }\underset{-\infty }{\overset{+\infty }{\int }}{\rho }_{X_1,X_2}\left(x_1,x_2\right){\left[P\left(A/x_1,x_2\right)\right]}^{2}d{x}_{1}d{x}_2\right]-{\theta }^2.\end{array}$$

So from these two equations, we can conclude

$${\theta }^2\le \mathit{\text{Const}}\le \theta .$$

In the next step, we would like find function ${\rho }_{\alpha }\left(\alpha \right)$ ( ${\rho }_{\beta }\left(\beta \right)$ ) in the equation for $\mathit{DIS}$ .

Restrictions for function $\alpha \left(F_1\right),0\le F_1\le 1$ are the following:

$$\underset{0}{\overset{1}{\int }}\alpha \left(F_1\right)d{F}_1=\theta ,$$

$$0\le \alpha \left(F_1\right)\le 1.$$

In discrete form (for $N\to \infty$ ), we can rewrite ${\alpha }_{\mathit{set}}=\left\{{\alpha }_1,{\alpha }_2,\dots ,{\alpha }_N\right\}$

$$\frac{1}{N}\sum _{i=1}^N{\alpha }_i=\theta ,$$

$$0\le {\alpha }_i\le 1,i=1,2,\dots ,N.$$

Let us define a function $U\left({\alpha }_{\mathit{set}}\right)$ in the following way:

$$U\left({\alpha }_{\mathit{set}}\right)=\left\{\begin{array}{lll}{\sum }_{i=1}^N{\alpha }_i& \text{for} 0\le {\alpha }_i\le 1& ,i=1,2,\dots ,N\\ +\infty & \text{otherwise}& \end{array}\right.,$$

$$U\left({\alpha }_{\mathit{set}}\right)=\sum _{i=1}^N{U}_i\left({\alpha }_i\right),$$

$$U_i\left({\alpha }_i\right)=\left\{\begin{array}{ll}{\alpha }_i& \text{for} 0\le {\alpha }_i\le 1\\ +\infty & \text{otherwise}\end{array}\right..$$

Then the function that satisfies equal probability distribution with considering restrictions (i) and (ii) is the following:

$${\rho }_{{\alpha }_{\mathit{set}}}\left({\alpha }_{\mathit{set}}\right)=\frac{1}{C}\delta \left(U\left({\alpha }_{\mathit{set}}\right)-\mathit{N\theta }\right).$$ E23

here $\delta$ is the Dirac delta function.

We can define the constant $C$ by

$$\underset{-\infty }{\overset{+\infty }{\int }}\dots \underset{-\infty }{\overset{+\infty }{\int }}{\rho }_{{\alpha }_{\mathit{set}}}\left({\alpha }_{\mathit{set}}\right)d{\alpha }_1\dots d{\alpha }_N=1.$$

It can be proved for $N\to \infty$ that distribution (23) is equal to the following distribution (from “statistical mechanics” [9]; transform from microcanonical to canonical distribution):

$${\rho }_{{\alpha }_{\mathit{set}}}\left({\alpha }_{\mathit{set}}\right)=\frac{1}{Z}{e}^{-\mathit{KU}\left({\alpha }_{\mathit{set}}\right)}.$$

Here we can find $Z$ and $K$ from the following equations:

$$\underset{-\infty }{\overset{+\infty }{\int }}\dots \underset{-\infty }{\overset{+\infty }{\int }}{\rho }_{{\alpha }_{\mathit{set}}}\left({\alpha }_{\mathit{set}}\right)d{\alpha }_1\dots d{\alpha }_N=1,$$ E24

$$\underset{-\infty }{\overset{+\infty }{\int }}\dots \underset{-\infty }{\overset{+\infty }{\int }}U\left({\alpha }_{\mathit{set}}\right){\rho }_{{\alpha }_{\mathit{set}}}\left({\alpha }_{\mathit{set}}\right)d{\alpha }_1\dots d{\alpha }_N=\mathit{N\theta }.$$ E25

Quest function ${\rho }_{\alpha }\left(\alpha \right)$ can be found by

$${\rho }_{\alpha }\left(\alpha \right)=\underset{-\infty }{\overset{+\infty }{\int }}\dots \underset{-\infty }{\overset{+\infty }{\int }}{\rho }_{{\alpha }_{\mathit{set}}}\left({\alpha }_1,\dots ,{\alpha }_j=\alpha ,\dots ,{\alpha }_N\right)\prod _{i=1,i\ne j}^{N}d{\alpha }_i=\frac{1}{D}{e}^{-K{U}_j\left({\alpha }_J=\alpha \right)},$$ E26

where

$$D^N=Z.$$ E27

From Eqs. (24) and (25), we can find

$$\frac{1}{Z}={\left(\frac{K}{1-e^{-K}}\right)}^N,$$ E28

$$\theta =\Lambda \left(K\right),$$ E29

where $\Lambda \left(K\right)$ is the decreasing function

$$\Lambda \left(K\right)=\left\{\begin{array}{ll}1& \text{for}K=-\infty \\ 0& \text{for}K=+\infty \\ 1/2& \text{for}K=0\\ \frac{1}{K}-\frac{1}{e^K-1}& \text{otherwise}\end{array}\right..$$

If $K$ is the root of Eq. (29), we can write from Eqs. (26)–(29) for function ${\rho }_{\alpha }\left(\alpha \right)$

$${\rho }_{\alpha }\left(\alpha \right)=\left\{\begin{array}{c}\left\{\begin{array}{cc}\text{For}& K=0\\ 1& \text{for} 0\le \alpha \le 1\\ 0& \text{otherwise}\alpha \end{array}\right.\\ \left\{\begin{array}{cc}\text{For}& K=+\infty \\ 2\delta \left(\alpha \right)& 0\le \alpha \le 1\\ 0& \text{otherwise}\alpha \end{array}\right.\\ \left\{\begin{array}{cc}\text{For}& K=-\infty \\ 2\delta \left(\alpha -1\right)& 0\le \alpha \le 1\\ 0& \text{otherwise}\alpha \end{array}\right.\\ \left\{\begin{array}{cc}\text{For}& \text{otherwise}K\\ \frac{1}{D}{e}^{-\mathit{K\alpha }}& 0\le \alpha \le 1\\ 0& \text{otherwise}\alpha \end{array}\right.\end{array}\right.,$$

where $2{\int }_0^1\delta \left(\alpha -1\right)\mathit{d\alpha }=2\underset{0}{\overset{1}{\int }}\delta \left(\alpha \right)\mathit{d\alpha }=1$ and $\frac{1}{D}=\frac{K}{1-e^{-K}}.$

10. The case of more than two states A and reliabilities X

Let A be a state, with values in set $0,1,\dots ,L$ . This number can characterize strength of a bond. Assume that the $\mathit{\text{apriori}}$ probability $P\left(A=i\right)$ is known, and denote it by ${\theta }_i$ ; here $i=1,\dots ,L$ . Let $X_1,\dots ,X_K$ be random variables, with values in some set, say $]-\infty ;+\infty [$ . We have the following information: $X_1=x_1$ ,…, $X_K=x_K$ (obtained through measurement). Furthermore, we have systems, “classifiers,” which for given $x_1$ ,…, $x_K$ produce

$$P\left(A=i/X_j=x_j\right)\equiv {\alpha }_{\mathit{ij}}.$$

We want to find the probability $P\left(A=i/X_1=x_1,\dots ,X_K=x_K\right)$ in terms of ${\alpha }_{\mathit{ij}}$ and ${\theta }_i$ . In more detail, we want to find a function ${\Gamma }_{\mathit{opt},M}\left({\alpha }_{\mathit{ij}},{\theta }_i\right)$ , which is the best approximation for $P\left(A=M/x_1,\dots ,x_K\right)$ on the average. By the same way, in case of two variables, it is possible to find that the ${\Gamma }_{\mathit{opt},M}\left({\alpha }_{\mathit{ij}},{\theta }_i\right)$ can be defined by the following equation:

$${\Gamma }_{\mathit{opt},M}\left({\alpha }_{\mathit{ij}},{\theta }_i\right)=\frac{\left({\prod }_{j=1}^K{\alpha }_{\mathit{Mj}}\right)/{\theta }_M^{K-1}}{{\sum }_{i=1}^L\left({\prod }_{j=1}^K{\alpha }_{\mathit{ij}}\right)/{\theta }_i^{K-1}}.$$

We have evidential restraints for ${\alpha }_{\mathit{ij}}$ , ${\theta }_i$

$$\begin{array}{c}0\le {\alpha }_{\mathit{ij}}\le 1\\ \sum _{i=1}^L{\alpha }_{\mathit{ij}}=1,\end{array}$$

$$\begin{array}{c}0\le {\theta }_i\le 1\\ \sum _{i=1}^L{\theta }_i=1.\end{array}$$

11. Conclusions

Using as an illustration the QSAR, we demonstrated effectively that the Naive Bayes model gives minimal mean error over uniform dispersion of all conceivable relationships between characteristic reliabilities. This result can clarify the portrayed over secretive optimality of Naive Bayes model. We too found the mean error that the Naive Bayes model gives for uniform distribution of all conceivable relationships of reliabilities.

Medicinal chemistry (quantitative structure-activity relationships, QSAR) prediction increasingly relies on Bayesian network-based methods. Its importance derives partly from the difficulty and inaccuracies of present quantum chemical models (e.g., in SYBYL and other software) and from the impracticality of sufficient characterization of structure of drug molecules and receptor active sites, including vicinal waters in and around hydrophobic pockets in active sites. This is particularly so for biologicals (protein and nucleic acid APIs (nucleic acid active pharmaceutical ingredients)) and target applications that exhibit extensive inter-receptor trafficking, genomic polymorphisms, and other system biology phenomena. The effectiveness and accuracy of Bayesian methods for drug development likewise depend on certain prerequisites, such as an adequate distance metric by which to measure similarity/difference between combinatorial library molecules and known successful ligand molecules targeting a particular receptor and addressing a particular clinical indication. In this connection, the distance metric proposed in Section 6 of the chapter manuscript and the associated Lemmas and Proofs are of substantial value in the future of high-throughput screening (HTS) and medicinal chemistry.

However, our purpose here was not demonstration of effectiveness of these definitions or effectiveness of QSAR. The interested reader can learn it from papers [3, 4] and references inside of these papers. As we said above, we use QSAR only for clearness; the proof is correct for any field of use of Naive Bayes classifier.