Open access peer-reviewed chapter

Bayesian Estimation of Multivariate Autoregressive Hidden Markov Model with Application to Breast Cancer Biomarker Modeling

Written By

Hamid El Maroufy, El Houcine Hibbah, Abdelmajid Zyad and Taib Ziad

Submitted: December 12th, 2016Reviewed: June 8th, 2017Published: November 2nd, 2017

DOI: 10.5772/intechopen.70053

Chapter metrics overview

1,524 Chapter Downloads

View Full Metrics


In this work, a first-order autoregressive hidden Markov model (AR(1)HMM) is proposed. It is one of the suitable models to characterize a marker of breast cancer disease progression essentially the progression that follows from a reaction to a treatment or caused by natural developments. The model supposes we have observations that increase or decrease with relation to a hidden phenomenon. We would like to discover if the information about those observations can let us learn about the progression of the phenomenon and permit us to evaluate the transition between its states (supposed discrete here). The hidden states governed by the Markovian process would be the disease stages, and the marker observations would be the depending observations. The parameters of the autoregressive model are selected at the first level according to a Markov process, and at the second level, the next observation is generated from a standard autoregressive model of first order (unlike other models considering the successive observations are independents). A Markov Chain Monte Carlo (MCMC) method is used for the parameter estimation, where we develop the posterior density for each parameter and we use a joint estimation of the hidden states or block update of the states.


  • autoregressive hidden Markov model
  • breast cancer progression marker
  • Gibbs sampler
  • hidden states joint estimation
  • Markov Chain Monte Carlo

1. Introduction

The main motivation behind this work is to characterize progression in breast cancer. In fact, disease progression cannot be assessed correctly without the use of biomarkers, which would effectively monitor the evolution of the patient health state, and this is the case for breast cancer. The major challenge in this matter for researchers and clinicians is to unravel the stage of the disease, so as to tailor the treatment for each patient and to monitor the response of a patient to a treatment.

Currently, studies have shown that there is a correlation between the levels of certain markers such as cancer antigen CA15-3, carcinoembryonic antigen (CEA), and serum HER2 Neu with the stage of the disease [1]. This gives an opportunity of using a hidden Markov model (HMM) to predict the stage of the disease based on biomarker data and to address the effectiveness of the treatments in their influence on the transition of the cancer from one state to another. In HMM, we have two constituents: the Markovian hidden process suitable to represent the breast cancer stage and the observation process given by the biomarker data. By the way, we can learn about the disease transition rates and how it progresses from primary breast cancer to advanced cancer stage, for example.

Indeed, HMM is a useful tool for tackling numerous concrete problems in many fields but some possible applications of HMM are in speech processing [2], in biology [3], in disease progression [4], in economics [5, 6], and in gene expression [7]. For a complete review of HMM, the reader is referred to Zucchini and MacDonald [8], in which properties and definitions of HMM are presented in a plausible way with both classical estimation by maximum likelihood method and expectation maximization (EM) algorithm and the new Bayesian inference is addressed.

The model we consider here is a variation of the regular hidden Markov model, since we will use extensions to incorporate dependence among successive observations, suggesting autoregressive dependence among continuous observations. Consequently, we have relaxed the conditional independence assumption from a standard HMM, because we would like to add some dynamics to the patient disease progression and because in reality the current patient biomarker observation is dependent on the past one. In fact, the autoregressive assumption in HMM has shown its advantage over regular HMM that cannot catch the strong dependence between successive observations (e.g., Ref. [9]). A similar model to ours can be found in Ref. [10]. This kind of models, which were first proposed in Ref. [11] to describe econometrics time series, is generalization of both HMM and autoregressive models, will be effective in representing multiple heterogeneous dynamics such as the disease progression dynamics, and can be even generalized to a regime-switching ARMA models such as in Ref. [12].

Moreover, Our model can also be viewed as an extension of the multivariate double-chain Markov model (DCMM) developed by Ref. [13], where there are two discrete Markov chains of first order: the first Markov chain is observed and the second one is hidden. In contrast to this DCMM, our multivariate first-order autoregressive hidden Markov model (MAR(1)HMM) will lead to continuous observations, where each observation conditional on the hidden process will depend on the previous observation according to an autoregressive process of first order. This dynamic is promising for continuous observed disease biomarkers.

Parameter estimation is very challenging for HMM family models since the likelihood is not available in a closed form most of the time. Thus, we call for a Markov Chain Monte Carlo (MCMC) procedure instead of a maximum likelihood-based approach. This choice rises from the fact that the Bayesian analysis uses prior knowledge about the process being measured, and it allows direct probability statements and an approximation of posterior distributions for the parameters. Instead in the maximum likelihood approach, we cannot have declared prior or have exact distribution for the parameters when the likelihood is untractable or when we have missing data (e.g., Refs. [1416]).

Since the realization of HMM includes two separate entities: the parameters and the hidden states, the Bayesian computation is carried out after augmenting the likelihood by the missing hidden states [17]. The hidden states are sampled using a Gibbs sampler adopting a joint estimation of the hidden states or block update of the states (instead of a single update of each state separately) by means of a forward filtering/backward smoothing algorithm. Given the hidden states, we can compute the autoregressive parameters and the transition probabilities of the Markov chain by Gibbs sampler from their posterior densities after specifying conjugate priors for the parameters. Hence, the MCMC algorithm will alternate between simulating the hidden states and the parameters. Finally, we can obtain posteriors statistics such as the means, standard deviations and confidence intervals after assessing the convergence of the MCMC algorithm.

This chapter is organized as follows: after a preliminary on HMM, a description of the model is given in Section 3. In Section 4, we will give the Bayesian estimation of the parameters and the hidden states and provide the details of the MCMC algorithm, before presenting the results of a simulation studies in Section 5 and we will finish by a conclusion.


2. Preliminary

Since the model suggested is of the HMM type, we will describe HMM in more detail: an HMM is a stochastic process {Xt,Yt}t=0T, where {Xt}t=0Tis a hidden Markov chain (unobservable) and {Yt}t=0Tis a sequence of observable independent random variables such that Ytdepends only on Xtfor the time t= 0,1,…T. Here the process {Xt}t=0Tevolves independently of {Yt}t=0Tand is supposed to be a homogeneous finite Markov chain with probability transition matrix Π of dimension a× a, where aindicates the number of the hidden states and Π0 = (Π01,…,Π0a) is the initial state distribution.

We denote the probability density function of Yt= ytgiven Xt= kfor k∈ {1, …, a} with Pxt(yt,θk), where θkrefers to the parameters of Pwhen Xt= k. We suppose further that the processes Yt|Xtand Yt′|Xt′are independent for tt′. Let Θ = (θ1,…, θa) and θ= (Π0, Π, Θ), and then, the HMM can be described as follows: First, the likelihood of the observations and the hidden states can be decomposed to P(y0,,yT,x0,,xT,θ)=P(y0,,yT|x0,,xT,θ)P(x0,,xT,θ).Since {Xt}t=0Tis a Markov chain, P(x0,,xT,θ)=Π0(x0)t=1TΠ(xt|xt1). Under the conditional independence of the observations given the hidden states, P(y0,,yT|x0,,xT,θ)=Px0(y0|θx0)t=1TPxt(yt|θxt). Consequently, the likelihood function for the hidden states and the observations is given by P(y0,y1,,yT,x0,x1,,xT,θ)=Π(x0)Px0(y0|θx0)t=0TΠ(xt|xt1)Pxt(yt|θxt).


3. Model description and specification

The MAR(1)HMM model we consider in this work is a hidden Markov model, where conditionally on the latent states, the observations are not independent like it is the case for a regular hidden Markov model. Instead, the current observation is allowed to depend on the previous observation according to an autoregressive model of first order. As in an HMM model, the latent states evolve according to a discrete first-order time homogeneous Markov model. We consider data of ncontinuous random variables observed over time, each of potentially different lengths, i.e., for each individual i= 1,2, …, n, we observe a vector yi,.=(yi,ui,,yi,mi)T, with ui< mi.

Define u0=min1in{ui}and M=max1in{mi}and note that the times uiand mimay vary over the entire observation period from u0 to Mwith the restriction that ui– mi≥ 1, for i= 1,2,…,n.

We assume, for i= 1,2,…,nfor integer time t= ui,…,mi, that the random variable Yi,ttaking nonnegative values depends only on the states Xtand the previous observation Yi,t–1, and based on the model developed by Farcomeni and Arima [10], we get the following model:


The choice of the autoregressive part of the model is motivated by the fact that successive biomarker observations are most of the time correlated from many diseases unlike the hypothesis of independence between observations in HMMs.

We interpret xas the vector of the hidden health states of the patients; in the case of breast cancer, those states would be localized or advanced metastatic breast cancer for example, while yis the vector of the biomarkers observed and measured for the patients. The εi, tare normal variables with mean 0 and variance σ2 such that εi, tand εi′, t′are uncorrelated, (i, t) ≠ (i′, t′).

The parameters β(xt)and μ(xt)are parameters taking values in Rfor each hidden state and σ2R+.

Similar to Ref. [13], the transition matrix of the Markov chain Π is time homogeneous with dimension a× awhere ais the number of hidden states, and Π = (Π gh, g= 1,…,a; h= 1,…,a) where Π gh= P(Xt= h|Xt–1 = g), for g= 1,2,…,a; h= 1,2,…, a; and t= u0+1,…, M. We let the first state Xu0to be selected from a discrete distribution with vector of probabilities r= (r1,…,ra). Also we consider the time of initial observation ui, the initial observed state yi,ui, and the number of consecutive time points that were observed miui+ 1. Let μ= (μ(1),…, μ(a)), β= (β(1),…,β(a)), and θ= (μ, β, σ2, r, Π) be the set of all parameters in the model. We suppose that the individuals, i.e., Yi, t, behave independently conditionally on X. Therefore, for i= 1, …, n, P(yi,.|yi,ui,x,θ)=t=ui+1miP(yi,t|yi,t1,xt,Θ)and P(x|θ)=P(xu0)t=u0+1MP(xt|xt1,Π), where P(xt|xt1,Π)=P(Xt=xt|Xt1=xt1,Π)=Πxt1,xt.Then, the likelihood density for the observations of all individuals y= (y1,…,yn) given first time vector of observations y0=(y1,u1,,yn,un), x, and θis P(y|y0,x,θ)=i=1nP(yi,.|yi,ui,x,θ),

This is due to the conditional independence of the yi, given xand θ. The joint mass of each yi,. and xgiven yuiand θcan be written as follows: P(yi,.,x|yi,ui,θ)=P(yi,.|yi,ui,x,θ)×P(x|yui,θ).Using the Markov property of the hidden process, we have after simplification P(x|yi,ui,θ)P(yi,ui|x,θ)P(x|θ)=P(yi,ui|xui,θ)rxu0Πxu0,xu0+1××ΠxM1,xM.In addition, P(yi,.|yi,ui,x,θ)=t=ui+1miP(yi,t|yi,t1,x,θ), and consequently,

P(yi,.,x|yi,ui,θ)rxu0P(yi,ui|xui,θ)t=u0+1MΠxt1,xtt=ui+1miP(yi,t|yi,t1,x,θ).Finally, under the hypothesis of normal error distribution for the autoregressive parameters of the model (Eq. (1)) and the Chapman-Kolmogorov property, the joint distribution of yi,. and xgiven yi,uiand θcan be simplified to:


where ϕdenotes the density of a standard normal distribution N(0,1)and χ{A}(x)is the usual indicator function of a set A. Finally, the joint distribution of yand xhas the following form:


4. Bayesian estimation of the model parameters

We will use a Bayesian approach to estimate the model parameters. Inference in the Bayesian framework is obtained through the posterior density, which is proportional to the prior multiplied by the likelihood. The posterior distribution for our model, as in most cases, cannot be derived analytically, and we will approximate it through MCMC methods specifically designed for working with the augmented likelihood with the hidden states. In fact, MCMC methods start by specifying the prior density Π(θ) for the parameters. Since the data Yare available, the general sampling methods work recursively by alternating between simulating the full conditional distributions Xgiven yand θgiven xand y.

4.1. Prior distributions

Under the assumption of independence between the parameters θ=(μ,β,σ2,r,Π), the prior density could be written as P(θ)=P(r)P(Π)P(μ)P(β)P(σ2). ris the parameters of a multinomial distribution; hence, the natural choice for the prior would be a Dirichlet distribution rD(α01,,α0a). Later on, j=1aΠij=1, and we assume that ΠiD(δi1,,δia)for each row iof the transition matrix. This choice of the Dirichlet prior can be even the default D(1,,1)as recently discussed in Ref. [18]. In fact, a Dirichlet prior is justified because the posterior density of each row of the transition matrix is proportional to the density of a Dirichlet distribution, and hence, choosing a Dirichlet prior would give a posterior Dirichlet. This can be justified as follows for a given set of parameters λ=(λ1,,λa)from a discrete or from a multinomial density:

π(x1,,xa,λ1,,λa)=n!x1!xa!λ1x1λaxafor the nonnegative integers x1,,xa, with i=1axi=n.This probability mass function can be expressed, using the gamma function Γ, as π(x1,,xa,λ1,,λa)=Γ(i=1axi+1)i=1aΓ(xi+1)i=1aλixi.This form shows its resemblance to the Dirichlet distribution, and by starting from supposing the prior λD(α0,,αa), the posterior is P(λ|x)P(λ)P(x|λ)iλixiiλiαi1iλixi+αi1D(x1+α1,,xa+αa).

Furthermore, concerning the priors for parameters of the autoregressive model, we suppose for h=1,,a: μ(h)N(αh,τh), β(h)N(bh,ch), and inverse gamma (IG) prior for σ2IG(ε,ζ). αh,τh,bh,ch,ϵ,ζare hyperparameters to be specified. For more details on Bayesian inference and prior selection in HMM, the reader is referred to Ref. [19]. In our case, prior distributions for the autoregressive parameters were proposed by Ref. [20] for a mixture autoregressive model, who points out that they are conventional prior choices for mixture models.

4.2. Sampling the posterior distribution for the hidden states

Chib [21] developed a method for the simulation of the hidden states from the full joint distribution for the univariate hidden Markov model case. We will describe his full Bayesian algorithm for the univariate hidden Markov model before a generalization to our MAR(1)HMM.

4.2.1. Chib’s algorithm for the univariate hidden Markov model for estimation of the states

Suppose we have an observed process Yn=(y1,,yn)and the hidden states Xn=(x1,,xn), θare the parameters of the model. We adopt for simplicity Xt=(x1,,xt)the history of the states up to time tand Xt+1=(xt+1,,xn)the future from t+ 1 to n. We use the same notation for Ytand Yt+1.

For each state xt{1,2,,a}for t=1,2,,n, the hidden model can be described by a conditional density given the hidden states π(yt|Yt1,xt=k)=π(yt|Yt1,θk), k=1,,a, with xtdepending only on xt–1 and having transition matrix Π and initial distribution Π0, and the parameters for π(.) are θ=(θ1,,θa).

Chib [21] shows that it is preferable to simulate the full latent data Xn=(x1,,xn)from the joint distribution of x1,,xn|Yn,θ, in order to improve the convergence property of the MCMC algorithm because instead of nadditional blocks if each state is simulated separately, only one additional block is required. First, we write the joint conditional density as


For sampling, it is sufficient to consider the sampling of xtfrom P(xt|Yn,Xt+1,θ,Π). Moreover, P(xt|Yn,Xt+1,θ,Π)P(xt|Yt,θ,Π)P(xt+1|xt,Π). This expression has two ingredients: the first is P(xt+1|xt,Π), which is the transition matrix from the Markov chain. The second is P(xt|Yt,θ,Π)that would be obtained by recursively starting at t= 1.

The mass function P(xt1|Yt1,θ,Π)is transformed into P(xt|Yt,θ,Π), which is in turn transformed into P(xt+1|Yt+1,θ,Π)and so on. The update is as follows: for k=1,,a, we could write


These calculations are initialized at t= 0, by setting P(x1|Y0,θ)to be the stationary distribution of the Markov chain. Precisely, the simulation proceeds for k=1,,a,recursively by first simulating P(x1=k|Y0,θ),from the initial distribution Π0(k) and P(x1=k|Y1,θ,Π)P(x1=k|Y0,θ,Π)π(y1|Y0,θk).Then, we get by forward calculation P(xt=k|Yt1,θ)=l=1aΠlkP(xt1=l|Yt1,θ), for each t=2,,n, where Π lkis the transition probability and P(xt=k|Yt,θ)P(xt=k|Yt1,θ,P)Π(yt|Yt1,θk).The last term in the forward computation P(xn=k|Yn,θ)would serve as a start for the backward pass, and we get recursively for each t=n1,,1.; P(xt=k|Yn,Xt+1,θ)P(xt|Yt1,θ,Π)P(xt+1|xt=k,Π),which permits the obtention of Xn=(x1,,xn).

4.2.2. Simulating the hidden states for the MAR(1)HMM

Returning to our model, and adopting notations and algorithm developed by Fitzpatrick and Marchev, fwill denote the observation density for the MAR(1)HMM, and for u0<t<M; xt=(xu0,,xt),xt=(xt,,xM),y(t)=(yi,t,i=1,2,,n), y,t=i:ui<t{yi,ui,,yi,min{t,mi}}, and yt=i:t<mi{yi,max{t+1,ui},,yi,mi}. The posterior distribution of the hidden state could be written as: P(xM|y,M,θ)=P(xM|y,M,θ)××P(xu0|y,M,xu0+1,θ). So we could sample the whole sequence of states by sampling from P(xt|y,M,xt+1,θ).Hence, the estimation of the hidden states is performed recursively by first initializing


We perform a similar calculation for every state at time t, and we conclude by calculating P(xM=k|y,M1,θ)=l=1aΠlkP(xM1=l|Y,M1,θ),and P(xM=k|y,M,θ)P(xM=k|y,M1,θ)f(y(M)|y,M1,θk). Later on, we get P(xM=k|y,M,θ), which permits the simulation of P(xM|y,M,θ).Finally, by backward calculation, we simulate from the probabilities P(xt|y,M,xt+1,θ)P(xt+1|xt,Π)P(xt|y,t,θ)for each time t=M1,,u0. Those backward probabilities would permit the simulation of the latent states.

4.3. Sampling from P(θ|x, y)

4.3.1. Sampling Π

Under the prior assumption of Dirichlet prior for each row of the transition matrix P(Πi)D(δi1,,δia), and the independence assumption between those rows, the posterior distribution for Π ican be developed using Eq. (2) as follows: Let nijdenote the number of single transitions from state ito state j, so


4.3.2. Sampling posterior distribution for initial distribution

Let n0l=χxu0(l), for l=1,,a. Using (2), under Dirichlet prior D(δ01,,δ0a)for the parameter r, we obtain P(r|x,y)P(r)l=1arlχ{xu0}(l)l=1arlδ0l+n0l1D(δ01+n01,,δ0a+n0a).

4.3.3. Sampling posterior distribution for the autoregressive parameters μ,β,σ2

When a complete conditional distribution is known such as the normal distribution or beta distribution, we use the Gibbs sampler to draw the random variable. This is the case for our model. Let us define nui(l)=i=1nχ{xui=l},nl=i=1nt=ui+1miχ{xt=l},N=l=1anl,n0l=χ{xu0}(l).So for l=1,2,,a;by supposing N(αl,τl)as prior distribution and using Eq. (2), the conditional posterior distribution of μ(l) is:


then μ(l)/y,xN(τ˜l,α˜l)with inverse mean τ˜l1=nui(l)+nlσ2+1τland variance


For β(l),l=1,,a, and similar to μ(l), N(bl,cl)was proposed as prior choice to obtain:


and therefore, β(l)/y,xN(c˜l,b˜l)with inverse mean c˜l1=1cl+i=1nt=ui+1,xt=lmiyi,t12σ2and variance


For the posterior distribution of σ2, by supposing IG(ε,ζ)as prior, we deduce from Eq. (2)


consequently σ2/y,xIG(ε˜,ζ˜)withparametersε˜=nui+N2+εand


Finally, the algorithm is ran for d= 1,…,Diterations by alternating between the following steps, where in each step we compute a conditional posterior for the given parameter:

The MCMC algorithm:

  1. For h=1,2,....,a, give reference values for the hyperparameters αh, τh, ah, bh, δ0h, and δihfor i=1,2,....,a.

  2. Initialization (Step d= 1 of the MCMC iterations): Initialize Π(1), r(1), μ(1), β(1), and σ2(1).

  3. Simulation of the hidden states:

    1. Initialization of forward simulation: P(xu0(d)|y,u0,θ)P(y,u0|xu0(d))P(xu0(d)|r(d)), with y,u0={yi,ui,,ui=u0,i=1,…,n}.

    2. Forward simulation: For k=1,….,aand t=u0+1,….,M:P(xt(d)=k|y,t1,θ)=l=1aΠlk(d)P(xt1(d)=l|Y,t1,θ)and


    3. Initialization of backward simulation: For k=1,….,a, given

      P(xM(d)=k|y,M,θ)from forward simulation, we get P(xM(d)|y,M,θ).

    4. Backward simulation: For k=1,….,aand t=M1,…,u0:


  4. Estimation of the initial distribution and the transition distribution

    1. for l=1,….,a, k=1,….,a. Calculate n0l=χ{xu0(d)}(l)and nkl=Σt=u0+1Mχ{xt1(d),xt(d)}(k,l).

    2. Sample (r1(d+1),….,ra(d+1))D(δ01+n01,….,δ0a+n0a).

    3. For i=1,….,a; sample (Πi1(d+1),….,Πia(d+1))D(δi1+ni1,….,δia+nia).

  5. Simulation of μ: For l=1,….,a,

    1. τ˜l1=nui(l)+nlσ(d)2+1τl.

    2. α˜l=τ˜l(i=1,xui=lnyi,ui+i=1nt=ui+1,xt=lmi(yi,tβ(d)(l)yi,t1)σ(d)2+αlτl).

    3. Simulate μ(d+1)(l)/y,xN(α˜l,τ˜l).

  6. Simulation of β: For l=1,….,a,

    1. c˜l1=1cl+i=1nt=ui+1,xt=lmiyi,t12σ(d)2.

    2. b˜l=c˜l(blcl+i=1nt=ui+1,xt=lmi(yi,tμd+1(l))yi,t1σ(d)2).

    3. Simulate β(d+1)(l)/y,xN(b˜l,c˜l).

  7. - Simulation of σ2:

    1. ϵ˜=nui+N2+ϵ.

    2. ζ˜=i=1n(yi,uiμ(d+1)(xui))2+i=1nt=ui+1mi(yi,tμ(d+1)(xt)β(d+1)(xt)yi,t1)22+ζ.

    3. Simulate σ(d+1)2/y,xIG(ϵ,ζ˜).


5. Simulation study

In this section, we apply our results to the breast cancer model discussed earlier. The main reason behind our work is that the progression of breast cancer cannot be seen directly unless we use observations related to the disease that could characterize its progression; those observations here are quantities which could be measured; they are called biomarkers, where the word biomarker is used to designate any objective indication of a biological process or disease condition including during treatment and should be measurable. Furthermore, biomarkers are increasingly used in the management of breast cancer patients. One example is reported in Ref. [22], stating that there is correlation between elevation of CEA and/or CA15-3 and disease progression, in breast cancer patients. Also we use the autoregressive dependence among the observations to add more dynamics to the model unlike conventional HMMs where the successive observations given the Markov process are independent. We used the classification of breast cancer in three states: local where the disease is confined within the breast, the regional phase when the lymph nodes are involved, and the distant stage where the cancer is found in other parts of the body. We restrict ourselves to these three stages unlike other stage classifications that divide the progression in more than three stages such as the TNM (tumor, node, and metastasis) system. By lack of finding data about breast cancer biomarkers, we will confine ourselves to simulate an MAR(1)HMM model for observation time M= 24, and a number of individuals n= 210, a= 3 for Markov states number, with the length observation time for each individual selected uniformly between 2 and M. The simulation process supposes we have for the autoregressive means μ=(μ(1),μ(2),μ(3))=(12,24,36), since markers such as CA15–3 increase as the disease advances toward metastatic breast cancer. In addition, CA15–3 increase rapidly between successive observations, and thus, we take in the simulation the parameters β=(β(1),β(2),β(3))=(0.2,0.4,0.8).

The algorithm of simulation works as follows:

  1. For each individual i=1,,n, choose mithe length of observation for that individual i.

  2. Generate each discrete disease state xtusing transition matrixΠ=(0.7,0.2,0.1;0.1,0.6,0.3;0.2,0.3,0.5)for t=u0+1,,M.

  3. Generate the observations yi,tfor all individuals using our model 1.

We choose a prior σ2IG(0.001,0.001), a D(1,,1)prior for each row of Π, and Gaussian noninformative priors for the μs and the βs. Having the hidden states and the observations, we ran our algorithm for 8000 MCMC iterations. MCMC algorithm convergence was assessed by analyzing MCMC iterations mixing plots that are shown in Figure 1 , autocorrelation sample graphs checking as illustrated in Figure 2 , and inspecting histograms of posterior densities for the parameters of the models in Figure 3 . All parameters show good mixing of chains, autocorrelations that decay immediately after a few lags, and perfect posterior densities fitting. Also the Gelman [23] potential scale reduction factor (PSRF) was plot. The PSRF is measured for more than two MCMC chains (three chains in this works are considered), and it is measured for each parameter of the model; it should show how the chains have forgotten their initial values and that the output from all chains is indistinguishable. It is based on a comparison of within-chain and between-chains variances and is similar to a classical analysis of variance; when the PSRF is high (perhaps greater than 1.1 or 1.2), then we should run our chains out longer to improve convergence to the stationary distribution. Each PSRF declines to 1 as the number of iterations approaches infinity to confirm convergence. All the parameters have shown a PSRF less than 1.1 as the number of iteration increases and by the way a good sign of convergence ( Figure 4 ). Moreover, we should point out that in the family of Markov switching model there is the so-called label switching problem (e.g., Ref. [24]) which arises identifiability problem, and hence, we would not estimate perfectly the parameters. In addition, the posterior densities could show evidence of multimodality. some authors postprocess the output of the MCMC to deal with the issue (e.g., [25]), while other uses a random permutation of the parameters in each iteration of the MCMC algorithm (e.g., [26]) or one can call for an invariant loss function method (e.g., [27]). In our case, no identifiability issue is noticed since we used well-separated prior hyperparameters. Even when we start from different initial values for the parameters, our algorithm converges immediately after a few iterations.

Figure 1.

Markov chain mixing for each parameter through MCMC algorithm simulation.

Figure 2.

Autocorrelation sample plots for parameters of the model.

Figure 3.

Posterior densities for the parameters of the model (after 8000 iterations).

Figure 4.

Potential scale reduction factor convergence to less than 1.02 with more iterations.

Finally and before giving our results, we should report that the simulation of the Dirichlet posterior was carried out following ([28, p. 22], [29, p. 155]) who reported that the posterior Dirichlet parameters should be simulated using the beta distribution approach. Table 1 shows how the posterior values estimated from algorithm are very close to the true ones.

ParameterTrue valuePosterior statistics
MeanStandard deviationConfidence interval (5%)

Table 1.

Posterior inference for the parameters of the MAR(1)HMM model.


6. Conclusion

We have extended the method of Chib [21] for block update estimation of the states to a MAR(1)HMM model. Furthermore, we would like to point out that our model can easily be extended to include missing observations, as we should only add an extra step in each MCMC iteration to estimate the missing observations. Also, we can estimate the autoregressive model for different values of the autoregressive order, p≥ 1, by evaluating the Bayesian information criterion to select the best order that fits the observations of the model. Our model would capture the complexity and the dynamics of the evolution of breast cancer by introducing the latent states; the probabilities of transition between the latent states allow to compare among the effects of treatments on slowing or accelerating the transition of the disease from one health stage to another the autoregressive parameter mean values corresponding to different stages of the disease would guide medical doctors and scientists to monitor patients in different phases of the disease. The model incorporates individual observations with different lengths.

Last but not least, we like to mention the utilities of switching diffusion processes in addressing and analyzing many complicated applications such as in finance and risk management. Our future work would be to apply these processes to explore disease progression, because they are characterized by the coexistence of continuous dynamics and discrete events as well as their interactions.



We would like to thank the editorial staff for the comments that helped in improving this work. Also, we would like to thank the supporters of this work: The Lalla Salma Foundation Prevention and Treatment of Cancer, Rabat, Morocco; and the Germano-Morrocan Program for Scientific Research PMARS 2015-060.


  1. 1.Samy N, Ragab HM, El Maksoud NA, Shaalan M. Prognostic significance of serum Her2/neu, BCL2, CA15-3 and CEA in breast cancer patients: A short follow up. Cancer Biomarkers. 2009;6:63‐72
  2. 2.Benmiloud B, Piczunski W. Estimation des parametres dans les chaines de markov cachees et segmentation. Traitement du Signal. 1995;12:433–454
  3. 3.Boys R, Handerson D. A Bayesian approach to DNA sequence segmentation (with discussion). Biometrics. 2004;60:573–588
  4. 4.Guihenneuc-Jouyaux C, Richardson S, Longini IM Jr. Modeling disease progression by a hidden Markov process: Application to characterizing CD4 cell decline. Biometrics. 2000;56:733–741
  5. 5.Albert J, Chib S. Bayes inference via Gibbs sampling of autoregressive time series subject to Markov mean and variance shifts. Journal of Business and Economic Statistics. 1993;11:1–15
  6. 6.Korolkiewickz M, Elliot J. A hidden Markov model of credit quality. Journal of Economic Dynamics and Control. 2008;32:3807–3819
  7. 7.Zeng Y, Frias J. A novel HMM-based clustering algorithm for the analysis of gene expression time-course data. Computational Statistics and Data Analysis. 2006;50:2472–2494
  8. 8.Zucchini W, MacDonald I. Hidden Markov Models for Time Series: An Introduction Using R. New York: Springer; 2009
  9. 9.Ailliot P, Monbet V. Markov-switching autoregressive models for wind time series. Environmental Modelling & Software. 2012;30:92–101
  10. 10.Farcomeni A, Arima S. A Bayesian autoregressive three state hidden Markov model for identifying switching monotonic regimes in microarray time course data. Statistical Applications in Genetics and Molecular Biology. 2013;23:467–480
  11. 11.Hamilton J. A new approach to the economic analysis of nonstationary time series and the business cycle. Econometrica. 1989;57:357–384
  12. 12.Kim C, Kim J. Bayesian inference in regime-switching ARMA models with absorbing states: The dynamics of the ex-ante real interest rate under regime shifts. Journal of Business and Economic Statistics. 2015;33:566–578
  13. 13.Fitzpatrick, M. and Marchev, D. Ef?cient bayesian estimation of the multivariate double chain markov model. Statistics and Computing, 2013;23(4):467-480
  14. 14.Lindley D. The philosophy of statistics. The Statistician. 2000;49:293–337
  15. 15.Bolstad, W. M. (2007). Introduction to Bayesian Statistics. John Wiley and Sons, Inc., Hoboken, New Jersey; 2007
  16. 16.Gelman A, Shalizi CR. Philosophy and the practice of Bayesian statistics in the social sciences. British Journal of Mathematical and Statistical Psychology. 2013;66:8–38
  17. 17.Hobert, J.P. The data augmentation algorithm: Theory and methodology. In: Brooks S, Gelman A, Jones GL, Meng X-L, editors. The Handbook of Markov Chain Monte Carlo. Boca Raton, FL: Chapman and Hall/CRC; 2011. p. 253
  18. 18.Tuyl F, Gerlach R, Mengersen K. Posterior predictive arguments in favor of the Bayes-Laplace priors as the consensus prior for the binomial and multinomial parameters. Bayesian Analysis. 2013;4:151–158
  19. 19.Cappe O, Moulines E, Ryden T. Inference in Hidden Markov Models. New York: Springer-Verlag; 2005
  20. 20.Sampietro S. Mixture of autoregressive components for modeling financial market volatility. LIUC Papers. Serie Metodi quantitativi 16. 2005
  21. 21.Chib S. Calculating posterior distributions and model estimates in Markov mixtures models. Journal of Econometrics. 1996;75:79–98
  22. 22.Laessig D, Nagel D, Heinemann V, Untch M, Kahlert S, Bauerfeind I, Stieber P. Importance of CEA and CA15-3 during disease progression in metastatic breast cancer patients. Anticancer Research. 2007;27:1963–1968
  23. 23.Gelman A. Inference and monitoring convergence. In: Gilks W, Richardson S, Spiegelhalter D, editors. Markov Chain Monte Carlo in Practice. London: Chapman and Hall/CRC; 1995. p. 131
  24. 24.Fruhwirth-Schnatter S. Finite Mixture and Markov Switching Models. New York: Springer; 2006
  25. 25.Celoux G. Bayesian inference for mixture: The label switching problem. In: Payne R, Green P, editors. Proceedings in Computational Statistics. Heidelberg: Physica; 1998. pp. 227–232
  26. 26.Fruhwirth-Schnatter S. Markov Chain Monte Carlo estimation of classical and dynamic switching and mixture models. Journal of the American Statistical Association. 2001;96:194–209
  27. 27.Hurn M, Justel A, Rober C. Estimating mixtures of regressions. Journal of Computational and Graphical Statistics. 2003;79:55–79
  28. 28.Kim C, Nelson C. State-Space Models with Regime Switching: Classical and Gibbs Sampling Approaches with Applications. Cambridge, MA: MIT Press; 1999
  29. 29.Krozlig H. Markov-Switching Vector Autoregressions. Berlin, Heidelberg: Springer-Verlag; 1997

Written By

Hamid El Maroufy, El Houcine Hibbah, Abdelmajid Zyad and Taib Ziad

Submitted: December 12th, 2016Reviewed: June 8th, 2017Published: November 2nd, 2017