Comparison in terms of prediction accuracy of models M2 and M3 under scenarios S1, S2 and S3.
Genomic selection (GS) is playing a major role in plant breeding for the selection of candidate individuals (animal or plants) early in time. However, for improving GS better statistical models are required. For this reason, in this chapter book we provide an improved version of the Bayesian multiple-trait and multiple-environment (BMTME) model of Montesinos-López et al. that takes into account the correlation between traits (genetic and residual) and between environments since allows general covariance’s matrices. This improved version of the BMTME model was derived using the matrix normal distribution that allows a more easy derivation of all full conditional distributions required, allows a more efficient model in terms of time of implementation. We tested the proposed model using simulated and real data sets. According to our results we have elements to conclude that this model improved considerably in terms of time of implementation and it is better than a Bayesian multiple-trait, multiple-environment model that not take into account general covariance structure for covariance’s of the traits and environments.
- genomic selection
- multiple-trait and multiple-environment
- general covariance’s matrices
Genomic selection is revolutionizing plant breeding, since allows the selection of candidate individuals (animal or plants) early in time. However, the success of genomic selection is linked directly to the use of statistical models, since the process of selection of candidate individuals is done using statistical models. However, most of the models currently used in genomic selection are univariate models mostly for continuous phenotypes, which not exploit the existing correlation between traits when the selection of individuals (genotypes or animals) is done with the purpose to improve simultaneously multiple-traits. The advantage of jointly modeling multiple-traits compared to analyzing each trait separately, is that the inference process appropriately accounts for the correlation among the traits, which helps to increase prediction accuracy, statistical power, parameter estimation accuracy, and reduce trait selection bias [1, 2]. For this reason, there is a great interest of plant and animal scientist to develop appropriate genomic selection models for multiple-traits and multiple-environments to take advantage of this correlation and to improve the prediction accuracy in the selection of candidate individuals.
For this reason, in this chapter we propose an improved version of the Bayesian multiple-trait, multiple-environment (BMTME) model proposed by Montesinos-López et al.  that is appropriate for correlated multiple-traits and multiple-environments but instead of building this model using the multivariate normal distribution we propose to build it using the matrix normal distribution which should avoid that the number of rows of the datasets grows proportional to the number of traits under study.
Also, the BMTME model was improved adding a general covariance structure for the genetic covariance of environments in place of assuming a diagonal matrix as the original BMTME model. Additionally, in this chapter we compare the improved model in terms of prediction accuracy and time of implementation with the original BMTME model of Montesinos-López et al.  and with a multiple-trait and multiple-environment model where it is ignored the correlation between traits and between environments. Our hypothesis is that the improved model should be similar in terms of prediction accuracy, but considerably faster in terms of time of implementation with regard to the original BMTME of Montesinos-López et al.  and a little better in terms of prediction accuracy that a multiple-trait and multiple-environment model that ignore the correlation between traits and environments. Also, we propose to implement the proposed model with simulated and real data sets. Our results suggest that the construction and implementation of the proposed model should be of great help for breeding scientist and programs since will help to select candidate genotypes early in time with more accuracy.
2. Material and methods
2.1. Matrix normal distribution
The matrix normal distribution is a probability distribution that is a generalization of the multivariate normal distribution to matrix-valued random variables. According with Rowe  the n × p matrix normal distribution can be derived as a special case of the np-variate Multivariate Normal distribution when the covariance matrix is separable. A np-dimensional vector x is distributed according to multivariate normal distribution with np-dimensional mean μ and np × np covariance matrix Ω if its probability density function is given by
When the covariance matrix Ω is separable, that is, is one of the form Ω = Σ ⊗ Φ, where ⊗ is the Kronecker product which multiplies every entry of its first matrix argument by its entire second matrix argument, Eq. (1) becomes
upon using the following matrix identities
where X and M are matrix of dimension n × p such that x = vec(X) and μ = vec(M), with vec is the vec operator that stacks the columns of its matrix argument from left to right into a single vector, tr(.) is the trace operator which gives the sum of the diagonal elements of a square matrix argument.
where (M, Σ, Φ) parametrize the above distribution with n × p, and Σ and Φ are positive defined matrix of dimension n × n and p × p, respectively. The matrices Σ and Φ are commonly referred to as the within and between covariance matrices. Sometimes they are referred to as the right and left covariance matrices .
Some useful properties of the matrix normal distribution are: the mean and model is equal to E(X| M, Σ, Φ) = M and the variance var(vec(X)| M, Σ, Φ) = Σ ⊗ Φ, which can be found by integration and differentiation. Since X follows a Matrix Normal distribution, the conditional and marginal distributions of any row or column subset are Multivariate Normal distributions .
2.2. Univariate model with genotype by environment interaction (M1)
First, for each trait we considered the following univariate linear mixed model:
were yij represents the normal response from the jth line in the ith environment (i = 1, 2, …, I, j = 1, 2, …, J). For illustration purposes, we will use I = 3. Ei represents the fixed effect of the ith environment and is assumed as a fixed effect, gj represents the random effect of the genomic effect of jth line, with is of order J × J and represents the Genomic Relationship Matrix (GRM) and is calculated using the VanRaden method  as , with W as the matrix of marker of order J × p. gEij is the random interaction term between the genomic effect of the jth line and the ith environment where and eij is a random error term associated with the jth line in the ith environment distributed as N(0, σ2). As previously mentioned, this model was used for each of the l = 1, …, L traits, where L denotes the number of traits under study.
2.3. Multivariate correlated model with multiple-trait and multiple-environment (M2)
To account for the correlation between traits, all of the L traits given in Eq. (6) should be jointly modeled in a whole multiple-trait, multiple-environment mixed model as the following:
where Y is of order n × L, with n = I × J, X is of order n × I, β is of order I × L and contains the beta coefficients of the environment by trait combinations, Z1 is of order n × J, Z2 is of order n × IJ, b1 is of order J × L and follows a normal matrix distribution MNJ × L(0, Gg, Σt), b2 is of order IJ × L with a normal matrix distribution b2 ∼ MNIJ × L(0, ΣE ⊗ Gg, Σt) and e is of order n × L with a normal matrix distribution e ∼ MNn × L(0, In, Re), where Σt is the genetic covariance matrix between traits and it is assumed unstructured (or general), ⊗ denotes a Kronecker product, ΣE is assumed as a general matrix of order I × I, Re is the residual general covariance matrix between traits. It is important to point out that the trait × environment (T × E) interaction term is included in the fixed effects, while the trait × genotype (T × G) interaction term is included in the random effect b1 and the three-way (T × G × E) interaction term is included in b2.
2.4. Joint posterior density and prior specification
In this section, we provide the joint posterior density and prior specification for the improved BMTME model. Assuming independent prior distributions for β, Σt, ΣE, and Re, the joint posterior density of the parameter vector becomes:
where P(β), P(Σt), P(ΣE) and P(Re) denote the density prior distributions of β, Σt, ΣE, and Re, respectively. Specifically, we are assuming an Inverse-Wishart (IW) for Ωv with shape parameter κ and scale matrix parameter B, and is denoted by Ωv ∼ IW(κ, B), with density function given by both are positive definite matrices. For the remaining parameters we are assuming the following prior distributions: ∼ MNn × p(β0, II, IL), b1|Σt ∼ MNJ × L(0, Gg, Σt), Σt ∼ IW(νt + L − 1, St), b2|Σt, ΣE ∼ MNIJ × L(0, ΣE ⊗ Gg, Σt), ΣE ∼ IW(νE + I − 1, SE), and Re ∼ IW(νe + L − 1, Se). Next we combine the joint posterior density of the parameter vector with the priors to obtain the full conditional distribution for parameters β, b1, b2, Σt, Re. All full conditionals, as well as details of their derivations, are given in Appendix A.
2.5. Gibbs sampler
In order to produce posterior means for all relevant model parameters, below we outline the exact Gibbs sampler procedure that we proposed for estimating the parameters of interest. The ordering of draws is somewhat arbitrary; however, we suggest the following order:
Step 1. Simulate β according to the normal distribution given in Appendix A (A.1).
Step 2. Simulate bh for h = 1, 2, according to the normal distribution given in Appendix A (A.2 and A.3).
Step 3. Simulate Σt according to the IW distribution given in Appendix A (A.4).
Step 4. Simulate ΣE according to the IW distribution given in Appendix A (A.5).
Step 5. Simulate Re according to the IW distribution given in Appendix A (A.6).
Step 6. Return to step 1 or terminate when chain length is adequate to meet convergence diagnostics.
2.6. Multivariate uncorrelated model with multiple-trait and multiple-environment (M3)
To compare the model given in Eq. (7) we considered also model M3 (Eq. 6) that consists of using the following multi-trait, multi-environment model that ignore the correlation between traits and between environments:
where yijl represents the normal response from the jth line in the ith environment for trait l (i = 1, 2, …, I, j = 1, 2, …, J, l = 1, …, L). Tl represents the fixed effect of the lth trait, TEil is the fixed interaction term between the lth trait and the ith environment, gTjl represents the random effect of the interaction of genotype j and the lth trait, with , gETijl is the three-way interaction of genotype j, the ith environment and the lth trait, with and eijl is a random error term associated with the jth line in the ith environment distributed as N(0, σ2).
2.7. Experimental data sets
2.7.1. Simulate data sets
For testing the proposed models and methods we simulated multiple-trait and multiple-environment data using model in Eq. (7). We studied six scenarios depending of the parameters used. For the first scenario (S1) we used the following parameters: three environments, three traits, 80 genotypes, 1 replication for environment-trait-genotype combination. We assumed that βT = [15, 12, 7, 14, 10, 9, 13, 11, 8], where the first three beta coefficients belong to traits 1, 2 and 3 in environment 1, the second three values for the three traits in environment 2 and the last three for environment 3. We assumed that the genomic relationship matrix is known and is equal to Gg = 0.3I80 + 0.7J80, where I80 is an identity matrix of order 80 and J80 is a matrix of order 80 × 80 of ones. Therefore, the total number of observations is 3 × 80 × 3 × 1 = 720, that is, 240 for each trait. Since a covariance matrix can be expressed in terms of a correlation matrix (Rr) and a standard deviation matrix (as:, with r = t, E, e, where r = t represent the genetic covariance between traits, r = E represents the genetic covariance matrix between environments and r = e, represents the residual covariance matrix between traits. For the three covariance matrices (r = t, E, e) in this scenario we used Rr = 0.15I3 + 0.85J3, where J3 is a matrix of order 3x3 of ones, and), ) and ). For the second scenario (S2) we used exactly the same set of parameters defined in S1 except that for the correlation matrix now we assumed that the pair of correlations between traits and between environments is equal to 0.5, that is, Rr = 0.5I3 + 0.5J3, while the third scenario (S3) also is exactly as S1 with the exception that Rr = 0.75I3 + 0.25J3, that is, the pair of correlations between traits and between environments is equal to 0.25. These three set of correlation matrices given in S1, S2 and S3 were proposed in order to study the performance of the methods proposed in the context of high correlation (S1), medium (S2) and low correlation (S3) between traits (genetic and residual) and between environments. Other 3 scenarios were studied: scenario 4 (S4) is exactly as scenario S1 but in place of 80 lines were used 100 lines, scenario 5 (S5) was exactly as scenario S2 but with 100 lines and the last scenario (S6) was exactly as scenario S3 but using 100 lines in place of 80.
2.7.2. Real wheat data set
Here, we present the information on the first real data set used for implementing the proposed models. This real data set composed of 250 wheat lines that were extracted from a large set of 39 yield trials grown during the 2013–2014 crop season in Ciudad Obregon, Sonora, Mexico . The trials under study were days to heading (DTHD), grain yield (GRYLD), plant height (PTHT) and the green normalized difference vegetation index (GNDVI), each of these traits were evaluated in three environments (Bed2IR, Bed5IR and Drip). The marker information used after editing was 12,083 markers. This data set was also used by Montesinos-López et al.  for this reason those interested in more details of this data set see this publication.
2.7.3. Real maize data set
The second real data set used for implementing the proposed models is composed of 309 double-haploid maize lines. Traits available in this data set include grain yield (Yield), anthesis-silking interval (ASI), and plant height (PH); each of these traits were evaluated in three optimum rainfed environments (EBU, KAT, and KTI). The marker information used after editing was 12,083 markers. Also, this data set was also used by Montesinos-López et al.  for this reason those interested in more details of this data set see this publication.
2.8. Assessing prediction accuracy
For assessing prediction accuracy for the simulated and real data sets a 20 training (trn)-testing (tst) random partitions were implemented under a cross-validation that mimicked a situation where lines were evaluated in some environments for the traits of interest; however, some lines were missing in all traits in the other environments, this cross-validation scheme is called CV1. Under this cross-validation, we assigned 80% of the lines to the trn set and the remaining 20% to the tst set. We used the Pearson correlation and mean square error of prediction (MSEP) to compare the predictive performance of the proposed models. Models with Pearson correlation closet to one indicated better predictions, while under the MSEP values closed to zero are better in terms of prediction accuracy. It is important to point out that model M2 was implemented with R code done for the authors implementing the Gibbs sampler given above for this model, while model M3 was implemented in the R package BGLR .
The results are presented in two sections. The first section presents the results of the simulated data set, while the second the results with the real data sets.
3.1. Simulated data sets
In Table 1, under scenario S1 we can observe that the proposed model M2 was the best in terms of prediction accuracy (with Pearson correlation and MSEP) since in the 9 trait-environment combinations model M2 (improved BMTME model) was better than model M3 (uncorrelated multiple-trait multiple-environment). In average in terms of Pearson correlation the model M2 was better than the model M3 by 8.72%, while in terms of MSEP model M2 was better than model M3 in average by 6.24%. Under scenario S2, in terms of Pearson correlation model M2 was better in 7 out of 9 trait-environment combinations and in 6 out of 9 trait-environment combination in terms of MSEP. In terms of Pearson correlation model M2 was better than M3 in average by 7.76%, while in terms of MSEP was better by 2.27% in average (Table 1). While under scenario S3 also model M2 was better than model M3, since in 7 out of 9 trait-environment was the best, while under MSEP model M2 was better than M3 in 5 out of 9 trait-environment combination, however, in average model M2 was better than model M3 by 3.98 and 1.028% in terms of Pearson correlation and MSEP, respectively (Table 1).
In Table 2, under scenario S4 model M2 was the best in terms of prediction accuracy (with Pearson correlation and MSEP) since in the 9 trait-environment combinations was better than model M3. In average in terms of Pearson correlation and MSEP model M2 was better than model M3 by 4.4 and 4.1%, respectively. Also, under scenario S5, in terms of Pearson correlation and MSEP, model M2 was better than model M3 in 7 out of 9 and in 6 out of 9 trait-environment combinations, respectively. Model M2 was better than M3 in average by 1.6% in terms of Pearson correlation and by 1.2% in average in terms of MSEP (Table 2). While under scenario S6 also model M2 was better than model M3 in terms of Pearson correlation, since in 7 out of 9 trait-environment was the best, while under MSEP model M2 was better than M3 in 5 out of 9 trait-environment combination, however, in average model M2 was better than model M3 by 1.6 and 1.02% in terms of Pearson correlation and MSEP, respectively (Table 2).
3.2. Real data sets
In Table 3 we can observe that in the wheat data set the best predictions were observed under the proposed improved BMTME model (M2), since in all trait-environment combinations was better model M2 in terms of Pearson correlation and in 10 out of 12 was the better in terms of MSEP than model M3 (that ignore the correlation between traits and between environments). However, in the maize data set the best predictions were observed under model M3, since in 5 out of 9 trait-environment combinations this model was superior to model M2, however there is not a great superiority of the results under model M3 regarded to model M2. This results obtained in the maize data set are in agreement with the correlation study performed since this data set has a very low genetic correlation between traits and between environments.
According to the results observed with the simulated data sets (Tables 1 and 2) and real data sets (Table 3) there is evidence that the larger the correlation between traits (genetic and residual) and environments (genetic) the better the performance of the proposed improved BMTME (M2) model with regard to the uncorrelated multiple-trait and multiple-environment model (M3), which means that when the there is considerable correlation between traits and between environments this help to increase prediction accuracy.
In this paper we proposed an improved version of the Bayesian multiple-trait multiple-environment (BMTME) model of Montesinos-López et al.  that was derived using the matrix normal distribution. The advantage of the proposed model (M2) is that it is more efficient in terms of time of implementation since this improved version works using as rows the genotypes by environment combinations in place of using as rows the combination of traits, genotypes and environments which allows a more practical implementation of the Gibbs sampler in terms of time of implementation. Another, improvement of the BMTME model is that now allows unstructured covariance matrix for modeling environments in place of only a diagonal matrix as the original BMTME model. We compared the extended model (M2) with an uncorrelated multiple-trait and multiple-environment model (M3) that ignores the general correlation between traits (genetic and residual) and between environments and we found that the proposed improved BMTME model (M2) outperforms model (M3) in all the scenarios under study with simulation, however the larger the correlation between traits and between environments the better the performance in terms of prediction accuracy of the improved BMTME model. Additionally, we provided all full conditionals required for the implementation of the improved BMTME model (see Gibbs sampler section and Appendix A). However, we are aware that more empirical evidence with real and simulated data is needed to support our findings, and for this reason, we encourage researcher to implement our proposed improved model and compare with models that ignore the correlation between traits and between environments like the model M3 given in Eq. (8).
Full conditional distribution for vec(β)
where , .
In the simplification of some calculations the following properties were involved: tr(AB) = vec(AT)Tvec(B) = vec(B)Tvec(AT), and vec (AXB) = (BT ⊗ A)vec(X).
Full conditional for vec(b1)
Full conditional for vec(b2)
where and .
Full conditional for Σt
Full conditional for ΣE
Full conditional for Re
where ‖(Y − Xβ − Z1b1 − Z2b2)‖ = (Y − Xβ − Z1b1 − Z2b2)T(Y − Xβ − Z1b1 − Z2b2) .