Stochastic Mixed LQR/H∞ Control for Linear Discrete-Time Systems

1. Introduction

Mixed $$H_2/H_{\infty }$$ control has received much attention in the past two decades, see Bernstein & Haddad (1989), Doyle et al. (1989b), Haddad et al. (1991), Khargonekar & Rotea (1991), Doyle et al. (1994), Limebeer et al. (1994), Chen & Zhou (2001) and references therein. The mixed $$H_2/H_{\infty }$$ control problem involves the following linear continuous-time systems

$$\begin{array}{c}\dot{x}(t)=Ax(t)+B_0{w}_0(t)+B_{1}w(t)+B_{2}u(t),x(0)=x_0\\ z(t)=C_{1}x(t)+D_{12}u(t)\\ y(t)=C_{2}x(t)+D_{20}{w}_0(t)+D_{21}w(t)\end{array}$$ E1

where, $$x(t)\in R^n$$ is the state, $$u(t)\in R^m$$ is the control input, $$w_0(t)\in R^{q_1}$$ is one disturbance input, $$w(t)\in R^{q_2}$$ is another disturbance input that belongs to $$L_2[\mathrm{0,}\infty )$$ , $$y(t)\in R^r$$ is the measured output.

Bernstein & Haddad (1989) presented a combined LQG/ $$H_{\infty }$$ control problem. This problem is defined as follows: Given the stabilizable and detectable plant (1) with $$w_0(t)=0$$ and the expected cost function

$$J(A_c,B_c,C_c)=\underset{t\to \infty }{\lim }E\{x^T(t)Qx(t)+u^T(t)Ru(t)\}$$ E2

determine an $$n$$ th order dynamic compensator

$$\begin{array}{c}{\dot{x}}_c(t)=A_{c}x(t)+B_{c}y(t)\\ u(t)=C_c{x}_c(t)\end{array}$$ E3

which satisfies the following design criteria: (i) the closed-loop system (1) (3) is stable; (ii) the closed-loop transfer matrix $$T_{zw}$$ from the disturbance input $$w$$ to the controlled output $$z$$ satisfies $${\Vert T_{zw}\Vert }_{\infty }<\gamma$$ ; (iii) the expected cost function $$J(A_c,B_c,C_c)$$ is minimized; where, the disturbance input $$w$$ is assumed to be a Gaussian white noise. Bernstein & Haddad (1989) considered merely the combined LQG/ $$H_{\infty }$$ control problem in the special case of $$Q=C_1^T{C}_1$$ and $$R=D_{12}^T{D}_{12}$$ and $$C_1^T{D}_{12}=0$$ . Since the expected cost function $$J(A_c,B_c,C_c)$$ equals the square of the $$H_2$$ -norm of the closed-loop transfer matrix $$T_{zw}$$ in this case, the combined LQG/ $$H_{\infty }$$ problem by Bernstein & Haddad (1989) has been recognized to be a mixed $$H_2/H_{\infty }$$ problem. In Bernstein & Haddad (1989), they considered the minimization of an “upper bound” of $${\Vert T_{zw}\Vert }_2^2$$ subject to $${\Vert T_{zw}\Vert }_{\infty }<\gamma$$ , and solved this problem by using Lagrange multiplier techniques. Doyle et al. (1989b) considered a related output feedback mixed $$H_2/H_{\infty }$$ problem (also see Doyle et al. 1994). The two approaches have been shown in Yeh et al. (1992) to be duals of one another in some sense. Haddad et al. (1991) gave sufficient conditions for the exstence of discrete-time static output feedback mixed $$H_2/H_{\infty }$$ controllers in terms of coupled Riccati equations. In Khargonekar & Rotea (1991), they presented a convex optimisation approach to solve output feedback mixed $$H_2/H_{\infty }$$ problem. In Limebeer et al. (1994), they proposed a Nash game approach to the state feedback mixed $$H_2/H_{\infty }$$ problem, and gave necessary and sufficient conditions for the existence of a solution of this problem. Chen & Zhou (2001) generalized the method of Limebeer et al. (1994) to output feedback multiobjective $$H_2/H_{\infty }$$ problem. However, up till now, no approach has involved the combined LQG/ $$H_{\infty }$$ control problem (so called stochastic mixed LQR/ $$H_{\infty }$$ control problem) for linear continuous-time systems (1) with the expected cost function (2), where, $$Q\ge 0$$ and $$R>0$$ are the weighting matrices, $$w_0(t)$$ is a Gaussian white noise, and $$w(t)$$ is a disturbance input that belongs to $$L_2[\mathrm{0,}\infty )$$ .

In this chapter, we consider state feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem for linear discrete-time systems. The deterministic problem corresponding to this problem (so called mixed LQR/ $$H_{\infty }$$ control problem) was first considered by Xu (2006). In Xu (2006), an algebraic Riccati equation approach to state feedback mixed quadratic guaranteed cost and $$H_{\infty }$$ control problem (so called state feedback mixed QGC/ $$H_{\infty }$$ control problem) for linear discrete-time systems with uncertainty was presented. When the parameter uncertainty equals zero, the discrete-time state feedback mixed QGC/ $$H_{\infty }$$ control problem reduces to the discrete-time state feedback mixed LQR/ $$H_{\infty }$$ control problem. Xu (2011) presented respectively a state space approach and an algebraic Riccati equation approach to discrete-time state feedback mixed LQR/ $$H_{\infty }$$ control problem, and gave a sufficient condition for the existence of an admissible state feedback controller solving this problem.

On the other hand, Geromel & Peres (1985) showed a new stabilizability property of the Riccati equation solution, and proposed, based on this new property, a numerical procedure to design static output feedback suboptimal LQR controllers for linear continuous-time systems. Geromel et al. (1989) extended the results of Geromel & Peres (1985) to linear discrete-time systems. In the fact, comparing this new stabilizability property of the Riccati equation solution with the existing results (de Souza & Xie 1992, Kucera & de Souza 1995, Gadewadikar et al. 2007, Xu 2008), we can show easily that the former involves sufficient conditions for the existence of all state feedback suboptimal LQR controllers. Untill now, the technique of finding all state feedback controllers by Geromel & Peres (1985) has been extended to various control problems, such as, static output feedback stabilizability (Kucera & de Souza 1995), $$H_{\infty }$$ control problem for linear discrete-time systems (de Souza & Xie 1992), $$H_{\infty }$$ control problem for linear continuous-time systems (Gadewadikar et al. 2007), mixed LQR/ $$H_{\infty }$$ control problem for linear continuous-time systems (Xu 2008).

The objective of this chapter is to solve discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem by combining the techniques of Xu (2008 and 2011) with the well known LQG theory. There are three motivations for developing this problem. First, Xu (2011) parametrized a central controller solving the discrete-time state feedback mixed LQR/ $$H_{\infty }$$ control problem in terms of an algebraic Riccati equation. However, no stochastic interpretation was provided. This paper thus presents a central solution to the discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem. This result may be recognied to be a stochastic interpretation of the discrete-time state feedback mixed LQR/ $$H_{\infty }$$ control problem considered by Xu (2011). The second motivation for our paper is to present a characterization of all admissible state feedback controllers for solving discrete-time stochastic mixed LQR/ $$H_{\infty }$$ control problem for linear continuous-time systems in terms of a single algebraic Riccati equation with a free parameter matrix, plus two constrained conditions: One is a free parameter matrix constrained condition on the form of the gain matrix, another is an assumption that the free parameter matrix is a free admissible controller error. The third motivation for our paper is to use the above results to solve the discrete-time static output feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem.

This chapter is organized as follows: Section 2 introduces several preliminary results. In Section 3, first,we define the state feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem for linear discrete-time systems. Secondly, we give sufficient conditions for the existence of all admissible state feedback controllers solving the discrete-time stochastic mixed LQR/ $$H_{\infty }$$ control problem. In the rest of this section, first, we parametrize a central discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ controller, and show that this result may be recognied to be a stochastic interpretation of discrete-time state feedback mixed LQR/ $$H_{\infty }$$ control problem considered by Xu (2011). Secondly, we propose a numerical algorithm for calclulating a kind of discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ controllers. Also, we compare our main result with the related well known results. As a special case, Section 5 gives sufficient conditions for the existence of all admissible static output feedback controllers solving the discrete-time stochastic mixed LQR/ $$H_{\infty }$$ control problem, and proposes a numerical algorithm for calculating a discrete-time static output feedback stochastic mixed LQR/ $$H_{\infty }$$ controller. In Section 6, we give two examples to illustrate the design procedures and their effectiveness. Section 7 is conclusion.

2. Preliminaries

In this section, we will review several preliminary results. First, we introduce the new stabilizability property of Riccati equation solutions for linear discrete-time systems which was presented by Geromel et al. (1989). This new stabilizability property involves the following linear discrete-time systems

$$\begin{array}{c}x(k+1)=Ax(k)+Bu(k);x(0)=x_0\\ y(k)=Cx(k)\end{array}$$ E4

with quadratic performance index

$$J_2:={\displaystyle \sum _{k=0}^{\infty }\{x^T(k)Qx(k)+u^T(k)Ru(k)\}}$$

under the influence of state feedback of the form

$$u(k)=Kx(k)$$ E5

where, $$x(k)\in R^n$$ is the state, $$u(k)\in R^m$$ is the control input, $$y(k)\in R^r$$ is the measured output, $$Q=Q^T\ge 0$$ and $$R=R^T>0$$ . We make the following assumptions

Assumption 2.1 $$(A,B)$$ is controllable.

Assumption 2.2 $$(A,Q^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.})$$ is observable.

Define a discrete-time Riccati equation as follows:

$$A^{T}SA-A-A^{T}SB{(R+B^{T}SB)}^{-1}{B}^{T}SA+Q=0$$ E6

For simplicity the discrete-time Riccati equation (6) can be rewritten as

$${\Pi }_d(S)=Q$$ E7

Geromel & Peres (1985) showed a new stabilizability property of the Riccati equation solution, and proposed, based on this new property, a numerical procedure to design static output feedback suboptimal LQR controllers for linear continuous-time systems. Geromel et al. (1989) extended this new stabilizability property displayed in Geromel & Peres (1985) to linear discrete-time systems. This resut is given by the following theorem.

Theorem 2.1 (Geromel et al. 1989) For the matrix $$L\in R^{m\times n}$$ such that

$$K=-{(R+B^{T}SB)}^{-1}{B}^{T}SA+L$$ E8

holds, $$S\in R^{n\times n}$$ is a positive definite solution of the modified discrete-time Riccati equation

$${\Pi }_d(S)=Q+L^T(R+B^{T}SB)L$$ E9

Then the matrix $$(A+BK)$$ is stable.

When these conditions are met, the quadratic cost function $$J_2$$ is given by

$$J_2=x^T(0)Sx(0)$$

Second, we introduce the well known discrete-time bounded real lemma (see Zhou et al., 1996; Iglesias & Glover, 1991; de Souza & Xie, 1992).

Lemma 2.1 (Discrete Time Bounded Real Lemma)

Suppose that $$\gamma >0$$ , $$M(z)=\left[\begin{array}{cc}A& B\\ C& D\end{array}\right]\in R{H}_{\infty }$$ , then the following two statements are equivalent:

i. $${\Vert M(z)\Vert }_{\infty }<\gamma$$ .

ii. There exists a stabilizing solution $$X\ge 0$$ ( $$X>0$$ if $$(C,A)$$ is observable ) to the discrete-time Riccati equation

$$A^{T}XA-X+{\gamma }^{-2}(A^{T}XB+C^{T}D)U_1^{-1}(B^{T}XA+D^{T}C)+C^{T}C=0$$

such that $$U_1=I-{\gamma }^{-2}(D^{T}D+B^{T}XB)>0$$ .

Next, we will consider the following linear discrete-time systems

$$\begin{array}{c}x(k+1)=Ax(k)+B_{1}w(k)+B_{2}u(k)\\ z(k)=C_{1}x(k)+D_{12}u(k)\end{array}$$ E10

under the influence of state feedback of the form

$$u(k)=Kx(k)$$ E11

where, $$x(k)\in R^n$$ is the state, $$u(k)\in R^m$$ is the control input, $$w(k)\in R^q$$ is the disturbance input that belongs to $$L_2[\mathrm{0,}\infty )$$ , $$z(k)\in R^p$$ is the controlled output. Let $$x(0)=x_0$$ .

The associated with this systems is the quadratic performance index

$$J_2:={\displaystyle \sum _{k=0}^{\infty }\{x^T(k)Qx(k)+u^T(k)Ru(k)\}}$$ E12

where, $$Q=Q^T\ge 0$$ and $$R=R^T>0$$ .

The closed-loop transfer matrix from the disturbance input $$w$$ to the controlled output $$z$$ is

$$T_{zw}(z)=\left[\begin{array}{cc}{A}_K& B_K\\ C_K& 0\end{array}\right]:=C_K{(zI-A_K)}^{-1}{B}_K$$

where, $$A_K:=A+B_{2}K$$ , $$B_K:=B_1$$ , $$C_K:=C_1+D_{12}K$$ .

The following lemma is an extension of the discrete-time bounded real lemma ( see Xu 2011).

Lemma 2.2 Given the system (10) under the influence of the state feedback (11), and suppose that $$\gamma >0$$ , $$T_{zw}(z)\in R{H}_{\infty }$$ ; then there exists an admissible controller $$K$$ such that $${\Vert T_{zw}(z)\Vert }_{\infty }<\gamma$$ if there exists a stabilizing solution $$X_{\infty }\ge 0$$ to the discrete time Riccati equation

$$A_K^T{X}_{\infty }{A}_K-X_{\infty }+{\gamma }^{-2}{A}_K^T{X}_{\infty }{B}_K{U}_1^{-1}{B}_K^T{X}_{\infty }{A}_K+C_K^T{C}_K+Q+K^{T}RK=0$$ E13

such that $$U_1=I-{\gamma }^{-2}{B}_K^T{X}_{\infty }{B}_K>0$$ .

Proof: See the proof of Lemma 2.2 of Xu (2011). Q.E.D.

Finally, we review the result of discrete-time state feedback mixed LQR/ $$H_{\infty }$$ control problem. Xu (2011) has defined this problem as follows: Given the linear discrete-time systems (10)(11) with $$w\in$$ $$L_2\text{[0}\text{,}\infty \text{)}$$ and $$x(0)=x_0$$ , for a given number $$\gamma$$ $$>\mathrm{0,}$$ determine an admissible controller that achieves

$$\underset{w\in L_{2+}}{\sup }\underset{K}{\inf }\left\{J_2\right\}$$ subject to $${\Vert T_{zw}(z)\Vert }_{\infty }<\gamma$$ .

If this controller $$K$$ exists, it is said to be a discrete-time state feedback mixed LQR/ $$H_{\infty }$$ controller.

The following assumptions are imposed on the system

Assumption 2.3 $$(C_1,A)$$ is detectable.

Assumption 2.4 $$(A,B_2)$$ is stabilizable.

Assumption 2.5 $$D_{12}^T\left[\begin{array}{cc}{C}_1& D_{12}\end{array}\right]=\left[\begin{array}{cc}0& I\end{array}\right]$$ .

The solution to the problem defined in the above involves the discrete-time Riccati equation

$$A^T{X}_{\infty }A-X_{\infty }-A^T{X}_{\infty }\widehat{B}{({\widehat{B}}^T{X}_{\infty }\widehat{B}+\widehat{R})}^{-1}{\widehat{B}}^T{X}_{\infty }A+C_1^T{C}_1+Q=0$$ E14

where, $$\widehat{B}=\left[\begin{array}{cc}{\gamma }^{-1}{B}_1& B_2\end{array}\right]$$ , $$\widehat{R}=\left[\begin{array}{cc}-I& 0\\ 0& R+I\end{array}\right]$$ .

Xu (2011) has provided a solution to discrete-time state feedback mixed LQR/ $$H_{\infty }$$ control problem, this result is given by the following theorem.

Theorem 2.2 There exists a discrete-time state feedback mixed LQR/ $$H_{\infty }$$ controller if the discrete-time Riccati equation (14) has a stabilizing solution $$X_{\infty }$$ and $$U_1=$$ $$I-{\gamma }^{-2}{B}_1^T{X}_{\infty }{B}_1>0$$ .

Moreover, this discrete-time state feedback mixed LQR/ $$H_{\infty }$$ controller is given by

$$K=-U_2^{-1}{B}_2^T{U}_{3}A$$

where, $$U_2=R+I+B_2^T{U}_3{B}_2$$ , and $$U_3=X_{\infty }+{\gamma }^{-2}{X}_{\infty }{B}_1{U}_1^{-1}{B}_1^T{X}_{\infty }$$ .

In this case, the discrete-time state feedback mixed LQR/ $$H_{\infty }$$ controller will achieve

$$\underset{w\in L_{2+}}{\sup }\underset{K}{\inf }\left\{J_2\right\}=x_0^T(X_{\infty }+{\gamma }^{-2}{X}_w-X_z)x_0$$ subject to $${\Vert T_{zw}\Vert }_{\infty }<\gamma$$ .

where, $${\widehat{A}}_K=A_K+{\gamma }^{-2}{B}_K{U}_1^{-1}{B}_K^T{X}_{\infty }{A}_K$$ , $$X_w={\displaystyle \sum _{k=0}^{\infty }\{{({\widehat{A}}_K^k)}^T{A}_K^T{X}_{\infty }}{B}_K{U}_1^{-2}{B}_K^T{X}_{\infty }{A}_K{\widehat{A}}_K^k\}$$ , and $$X_z={\displaystyle \sum _{k=0}^{\infty }\{{({\widehat{A}}_K^k)}^T{C}_K^T{C}_K}{\widehat{A}}_K^k\}$$ .

3. State Feedback

In this section, we consider the following linear discrete-time systems

$$\begin{array}{c}x(k+1)=Ax(k)+B_0{w}_0(k)+B_{1}w(k)+B_{2}u(k)\\ z(k)=C_{1}x(k)+D_{12}u(k)\\ y(k)=C_{2}x(k)\end{array}$$ E15

with state feedback of the form

$$u(k)=Kx(k)$$ E16

where, $$x(k)\in R^n$$ is the state, $$u(k)\in R^m$$ is the control input, $$w_0(k)\in R^{q_1}$$ is one disturbance input, $$w(k)\in R^{q_2}$$ is another disturbance that belongs to $$L_2[\mathrm{0,}\infty )$$ , $$z(k)\in R^p$$ is the controlled output, $$y(k)\in R^r$$ is the measured output.

It is assumed that $$x(0)$$ is Gaussian with mean and covariance given by

$$\begin{array}{l}E\{x(0)\}={\overline{x}}_0\\ \mathrm{cov}\left\{x\left(0\right),x\left(0\right)\right\}:=E\left\{\left(x\left(0\right)-{\overline{x}}_0\right){\left(x\left(0\right)-{\overline{x}}_0\right)}^T\right\}=R_0\end{array}$$

The noise process $$w_0(k)$$ is a Gaussain white noise signal with properties

$$E\{w_0(k)\}=\mathrm{0,}E\{w_0(k)w_0^T(\tau )\}=R_1(k)\delta (k-\tau )$$

Furthermore, $$x(0)$$ and $$w_0(k)$$ are assumed to be independent, $$w_0(k)$$ and $$w(k)$$ are also assumed to be independent, where, $$E[•]$$ denotes expected value.

Also, we make the following assumptions:

Assumption 3.1 $$(C_1,A)$$ is detectable.

Assumption 3.2 $$(A,B_2)$$ is stabilizable.

Assumption 3.3 $$D_{12}^T\left[\begin{array}{cc}{C}_1& D_{12}\end{array}\right]=\left[\begin{array}{cc}0& I\end{array}\right]$$ .

The expected cost function corresponding to this problem is defined as follows:

$$J_E:=\underset{T\to \infty }{\lim }\frac{1}{T}E\{{\displaystyle \sum _{k=0}^T(x^T(k)Qx(k)+u^T(k)Ru(k)-{\gamma }^2{\Vert w\Vert }^2)\}}$$ E17

where, $$Q=Q^T\ge 0$$ , $$R=R^T>0$$ , and $$\gamma >0$$ is a given number.

As is well known, a given controller $$K$$ is called admissible (for the plant $$G$$ ) if $$K$$ is real-rational proper, and the minimal realization of $$K$$ internally stabilizes the state space realization (15) of $$G$$ .

Recall that the discrete-time state feedback optimal LQG problem is to find an admissible controller that minimizes the expected quadratic cost function (17) subject to the systems (15) (16) with $$w(k)=0$$ , while the discrete-time state feedback $$H_{\infty }$$ control problem is to find an admissible controller such that $${\Vert T_{zw}\Vert }_{\infty }<\gamma$$ subject to the systems (15) (16) for a given number $$\gamma >0$$ . While we combine the two problems for the systems (15) (16) with $$w\in$$ $$L_2[\mathrm{0,}\infty )$$ , the expected cost function (17) is a function of the control input $$u(k)$$ and disturbance input $$w(k)$$ in the case of $$\gamma$$ being fixed and $$x(0)$$ being Gaussian with known statistics and $$w_0(k)$$ being a Gaussain white noise with known statistics. Thus it is not possible to pose a discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem that achieves the minimization of the expected cost function (17) subject to $${\Vert T_{zw}\Vert }_{\infty }<\gamma$$ for the systems (15) (16) with $$w\in L_2[\mathrm{0,}\infty )$$ because the expected cost function (17) is an uncertain function depending on disturbance input $$w(k)$$ . In order to eliminate this difficulty, the design criteria of discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem should be replaced by the following design criteria:

$$\underset{w\in L_{2+}}{\sup }\underset{K}{\inf }\left\{J_E\right\}$$ subject to $${\Vert T_{zw}\Vert }_{\infty }<\gamma$$

because for all $$w\in L_2[\mathrm{0,}\infty )$$ , the following inequality always exists.

$$\underset{K}{\inf }\left\{J_E\right\}\le \underset{w\in L_{2+}}{\sup }\underset{K}{\inf }\left\{J_E\right\}$$

Based on this, we define the discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem as follows:

Discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem: Given the linear discrete-time systems (15) (16) satisfying Assumption 3.1-3.3 with $$w(k)\in L_2[\mathrm{0,}\infty )$$ and the expected cost functions (17), for a given number $$\gamma >0$$ , find all admissible state feedback controllers $$K$$ such that

$$\underset{w\in L_{2+}}{\sup }\left\{J_E\right\}$$ subject to $${\Vert T_{zw}\Vert }_{\infty }<\gamma$$

where, $$T_{zw}(z)$$ is the closed loop transfer matrix from the disturbance input $$w$$ to the controlled output $$z$$ .

If all these admissible controllers exist, then one of them $$K=K^{\ast }$$ will achieve the design criteria

$$\underset{w\in L_{2+}}{\sup }\underset{K}{\inf }\left\{J_E\right\}$$ subject to $${\Vert T_{zw}\Vert }_{\infty }<\gamma$$

and it is said to be a central discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ controller.

Remark 3.1 The discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem defined in the above is also said to be a discrete-time state feedback combined LQG/ $$H_{\infty }$$ control problem in general case. When the disturbance input $$w(k)=0$$ , this problem reduces to a discrete-time state feedback combined LQG/ $$H_{\infty }$$ control problem arisen from Bernstein & Haddad (1989) and Haddad et al. (1991).

Remark 3.2 In the case of $$w(k)=0$$ , it is easy to show (see Bernstein & Haddad 1989, Haddad et al. 1991) that $$J_E$$ in (17) is equivalent to the expected cost function

$$J_E=\underset{k\to \infty }{\lim }E\{x^T(k)Qx(k)+u^T(k)Ru(k)\}$$

Define $$Q=C_1^T{C}_1$$ and $$R=D_{12}^T{D}_{12}$$ and suppose that $$C_1^T{D}_{12}=0$$ , then $$J_E$$ may be rewritten as

$$\begin{array}{l}{J}_E=\underset{k\to \infty }{\lim }E\{x^T(k)Qx(k)+u^T(k)Ru(k)\}\\ =\underset{k\to \infty }{\lim }E\{x^T(k)C_1^T{C}_{1}x(k)+u^T(k)D_{12}^T{D}_{12}u(k)\}\\ =\underset{k\to \infty }{\lim }E\{z^T(k)z(k)\}\end{array}$$

Also, the controlled output $$z$$ may be expressed as

$$z=T_{z{w}_0}(z)w_0$$ E18

where, $$T_{z{w}_0}(z)=\left[\begin{array}{cc}{A}_K& B_0\\ C_K& 0\end{array}\right]$$ . If $$w_0$$ is white noise with indensity matrix $$I$$ and the closed-loop systems is stable then

$$J_E=\underset{k\to \infty }{\lim }E\{z^T(k)z(k)\}={\Vert T_{z{w}_0}\Vert }_2^2$$

This implies that the discrete-time state feedback combined LQG/ $$H_{\infty }$$ control problem in the special case of $$Q=C_1^T{C}_1$$ and $$R=D_{12}^T{D}_{12}$$ and $$C_1^T{D}_{12}=0$$ arisen from Bernstein & Haddad (1989) and Haddad et al. (1991) is a mixed $$H_2$$ / $$H_{\infty }$$ control problem.

Based on the above definition, we give sufficient conditions for the existence of all admissible state feedback controllers solving the discrete-time stochastic mixed LQR/ $$H_{\infty }$$ control problem by combining the techniques of Xu (2008 and 2011) with the well known LQG theory. This result is given by the following theorem.

Theorem 3.1 There exists a discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ controller if the following two conditions hold:

i. There exists a matrix $$\Delta K$$ such that

$$\Delta K=K+U_2^{-1}{B}_2^T{U}_{3}A$$ E19

and $$X_{\infty }$$ is a symmetric non-negative definite solution of the following discrete-time Riccati equation

$$\begin{array}{l}{A}^T{X}_{\infty }A-X_{\infty }-A^T{X}_{\infty }\widehat{B}{\left({\widehat{B}}^T{X}_{\infty }\widehat{B}+\widehat{R}\right)}^{-1}{\widehat{B}}^T{X}_{\infty }A\\ +C_1^T{C}_1+Q+\Delta K^T{U}_2\Delta K=0\end{array}$$ E20

and $${\widehat{A}}_c=A-\widehat{B}{({\widehat{B}}^T{X}_{\infty }\widehat{B}+\widehat{R})}^{-1}{\widehat{B}}^T{X}_{\infty }A$$ is stable and $$U_1=I-{\gamma }^{-2}{B}_1^T{X}_{\infty }{B}_1>0$$ ;

where, $$\widehat{B}=\left[\begin{array}{cc}{\gamma }^{-1}{B}_1& B_2\end{array}\right]$$ , $$\widehat{R}=\left[\begin{array}{cc}-I& 0\\ 0& R+I\end{array}\right]$$ , $$U_3=X_{\infty }+{\gamma }^{-2}{X}_{\infty }{B}_1{U}_1^{-1}{B}_1^T{X}_{\infty }$$ , $$U_2=R+I+$$ $$B_2^T{U}_3{B}_2$$ .

ii. $$\Delta K$$ is an admissible controller error.

In this case, the discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ controller will achieve

$$\underset{w\in L_{2+}}{\sup }\left\{J_E\right\}=\underset{T\to \infty }{\lim }\frac{1}{T}{\displaystyle \sum _{k=0}^{T}tr(B_0^T{X}_{\infty }}{B}_0{R}_1(k))$$ subject to $${\Vert T_{zw}\Vert }_{\infty }<\gamma$$

Remark 3.3 In Theorem 3.1, the controller error is defined to be the state feedback controller $$K$$ minus the suboptimal controller $$K^{\ast }=-U_2^{-1}{B}_2^T{U}_{3}A$$ , where, $$X_{\infty }\ge 0$$ satisfies the discrete-time Riccati equation (20), that is,

$$\Delta K=K-K^{\ast }$$

where, $$\Delta K$$ is the controller error, $$K$$ is the state feedback controller and $$K^{\ast }$$ is the suboptimal controller. Suppose that there exists a suboptimal controller $$K^{\ast }$$ such that $$A_{K^{\ast }}=A+B_2{K}^{\ast }$$ is stable, then $$K$$ and $$\Delta K$$ is respectively said to be an admissible controller and an admissible controller error if it belongs to the set

$$\Omega :=\{\Delta K:A_{K^{\ast }}+B_2\Delta K\text{ is stable }\}$$

Remark 3.4 The discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ controller satisfying the conditions i-ii displayed in Theorem 3.1 is not unique. All admissible state feedback controllers satisfying these two conditions lead to all discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ controllers.

Astrom (1971) has given the mean value of a quadratic form of normal stochastic variables. This result is given by the following lemma.

Lemma 3.1 Let $$x$$ be normal with mean $$m$$ and covariance $$R$$ . Then

$$E\left\{x^{T}Sx\right\}=m^{T}Sm+trSR$$

For convenience, let $$A_K=A+B_{2}K$$ , $$B_K=B_1$$ , $$C_K=C_1+D_{12}K$$ , $$A_{K^{\ast }}=A+B_2{K}^{\ast }$$ , $$B_{K^{\ast }}=B_1$$ , $$C_{K^{\ast }}=C_1+D_{12}{K}^{\ast }$$ , and $$K^{\ast }=-U_2^{-1}{B}_2^T{U}_{3}A$$ , where, $$X_{\infty }\ge 0$$ satisfies the discrete-time Riccati equation (20); then we have the following lemma.

Lemma 3.2 Suppose that the conditions i-ii of Theorem 3.1 hold, then the both $$A_{K^{\ast }}$$ and $$A_K$$ are stable.

Proof: Suppose that the conditions i-ii of Theorem 3.1 hold, then it can be easily shown by using the similar standard matrix manipulations as in the proof of Theorem 3.1 in de Souza & Xie (1992) that

$${({\widehat{B}}^T{X}_{\infty }\widehat{B}+\widehat{R})}^{-1}=\left[\begin{array}{cc}-U_1^{-1}+U_1^{-1}{\widehat{B}}_1{U}_2^{-1}{\widehat{B}}_1^T{U}_1^{-1}& U_1^{-1}{\widehat{B}}_1{U}_2^{-1}\\ U_2^{-1}{\widehat{B}}_1^T{U}_1^{-1}& U_2^{-1}\end{array}\right]$$

where, $${\widehat{B}}_1={\gamma }^{-1}{B}_1^T{X}_{\infty }{B}_2$$ .Thus we have

$$A^T{X}_{\infty }\widehat{B}{({\widehat{B}}^T{X}_{\infty }\widehat{B}+\widehat{R})}^{-1}{\widehat{B}}^T{X}_{\infty }A=-{\gamma }^2{A}^T{X}_{\infty }{B}_1{U}_1^{-1}{B}_1^T{X}_{\infty }A+A^T{U}_3{B}_2{U}_2^{-1}{B}_2^T{U}_{3}A$$

Rearranging the discrete-time Riccati equation (20), we get

$$\begin{array}{l}{X}_{\infty }=A^T{X}_{\infty }A+{\gamma }^{-2}{A}^T{X}_{\infty }{B}_1{U}_1^{-1}{B}_1^T{X}_{\infty }A-A^T{U}_3{B}_2{U}_2^{-1}{B}_2^T{U}_{3}A\\ +C_1^T{C}_1+Q+\Delta K^T{U}_2\Delta K\\ =A_{K^{\ast }}^T{X}_{\infty }{A}_{K^{\ast }}+{\gamma }^{-2}{A}_{K^{\ast }}^T{X}_{\infty }{B}_{K^{\ast }}{U}_1^{-1}{B}_{K^{\ast }}^T{X}_{\infty }{A}_{K^{\ast }}+C_{K^{\ast }}^T{C}_{K^{\ast }}+Q\\ +K^{\ast T}R{K}^{\ast }+\Delta K^T{U}_2\Delta K\end{array}$$

that is,

$$\begin{array}{l}{A}_{K^{\ast }}^T{X}_{\infty }{A}_{K^{\ast }}-X_{\infty }+{\gamma }^{-2}{A}_{K^{\ast }}^T{X}_{\infty }{B}_{K^{\ast }}{U}_1^{-1}{B}_{K^{\ast }}^T{X}_{\infty }{A}_{K^{\ast }}+C_{K^{\ast }}^T{C}_{K^{\ast }}+Q\\ +K^{\ast T}R{K}^{\ast }+\Delta K^T{U}_2\Delta K=0\end{array}$$ E21

Since the discrete-time Riccati equation (20) has a symmetric non-negative definite solution $$X_{\infty }$$ and $${\widehat{A}}_c=A-\widehat{B}{({\widehat{B}}^T{X}_{\infty }\widehat{B}+\widehat{R})}^{-1}{\widehat{B}}^T{X}_{\infty }A$$ is stable, and we can show that $${\widehat{A}}_c=A_{K^{\ast }}+$$ $${\gamma }^{-2}{B}_{K^{\ast }}{U}_1^{-1}{B}_{K^{\ast }}^T{X}_{\infty }{A}_{K^{\ast }}$$ , the discrete-time Riccati equation (21) also has a symmetric non-negative definite solution $$X_{\infty }$$ and $$A_{K^{\ast }}+{\gamma }^{-2}{B}_{K^{\ast }}{U}_1^{-1}{B}_{K^{\ast }}^T{X}_{\infty }{A}_{K^{\ast }}$$ also is stable. Hence, $$(U_1^{-1}{B}_{K^{\ast }}^T{X}_{\infty }{A}_{K^{\ast }},A_{K^{\ast }})$$ is detectable. Based on this, it follows from standard results on Lyapunov equations (see Lemma 2.7 a), Iglesias & Glover 1991) that $$A_{K^{\ast }}$$ is stable. Also, note that $$\Delta K$$ is an admissible controller error, so $$A_K=A_{K^{\ast }}+B_2\Delta K$$ is stable. Q. E. D.

Proof of Theorem 3.1: Suppose that the conditions i-ii hold, then it follows from Lemma 3.2 that the both $$A_{K^{\ast }}$$ and $$A_K$$ are stable. This implies that $$T_{zw}(z)\in R{H}_{\infty }$$ .

Define $$V(x(k))=x^T(k)X_{\infty }x(k)$$ , where, $$X_{\infty }$$ is the solution to the discrete-time Riccati equation (20), then taking the difference $$\Delta V(x(k))$$ , we get

$$\begin{array}{l}\Delta V(x(k))=x^T(k+1)X_{\infty }x(k+1)-x^T(k)X_{\infty }x(k)\\ =x^T(k)(A_K^T{X}_{\infty }{A}_K-X_{\infty })x(k)+2w^T(k)B_K^T{X}_{\infty }{A}_{K}x(k)\\ +w^T(k)B_K^T{X}_{\infty }{B}_{K}w(k)+2w_0^T(k)B_0^T{X}_{\infty }{A}_{K}x(k)\\ +2w_0^T(k)B_0^T{X}_{\infty }{B}_{1}w(k)+w_0^T(k)B_0^T{X}_{\infty }{B}_0{w}_0(k)\end{array}$$ E22

On the other hand, we can rewrite the discrete-time Riccati equation (20) by using the same standard matrix manipulations as in the proof of Lemma 3.2 as follows:

$$\begin{array}{l}{A}^T{X}_{\infty }A-X_{\infty }+{\gamma }^{-2}{A}^T{X}_{\infty }{B}_1{U}_1^{-1}{B}_1^T{X}_{\infty }A-A^T{U}_3{B}_2{U}_2^{-1}{B}_2^T{U}_{3}A\\ +C_1^T{C}_1+Q+\Delta K^T{U}_2\Delta K=0\end{array}$$

or equivalently

$$A_K^T{X}_{\infty }{A}_K-X_{\infty }+{\gamma }^{-2}{A}_K^T{X}_{\infty }{B}_K{U}_1^{-1}{B}_K^T{X}_{\infty }{A}_K+C_K^T{C}_K+Q+K^{T}RK=0$$ E23

It follows from Lemma 2.2 that $${\Vert T_{zw}\Vert }_{\infty }<\gamma$$ .Completing the squares for (22) and substituting (23) in (22), we get

$$\begin{array}{l}\Delta V(x(k))=-{\Vert z\Vert }^2+{\gamma }^2{\Vert w\Vert }^2-{\gamma }^2{\Vert U_1^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}(w-{\gamma }^{-2}{U}_1^{-1}{B}_K^T{X}_{\infty }{A}_{K}x)\Vert }^2\\ +x^T(k)(A_K^T{X}_{\infty }{A}_K-X_{\infty }+{\gamma }^{-2}{A}_K^T{X}_{\infty }{B}_K{U}_1^{-1}{B}_K^T{X}_{\infty }{A}_K+C_K^T{C}_K)x(k)\\ +2w_0^T(k)B_0^T{X}_{\infty }{A}_{K}x(k)+2w_0^T(k)B_0^T{X}_{\infty }{B}_{1}w(k)+w_0^T(k)B_0^T{X}_{\infty }{B}_0{w}_0(k)\\ =-{\Vert z\Vert }^2+{\gamma }^2{\Vert w\Vert }^2-{\gamma }^2{\Vert U_1^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}(w-{\gamma }^{-2}{U}_1^{-1}{B}_K^T{X}_{\infty }{A}_{K}x)\Vert }^2-x^T(k)(Q+K^{T}RK)x(k)\\ +2w_0^T(k)B_0^T{X}_{\infty }{A}_{K}x(k)+2w_0^T(k)B_0^T{X}_{\infty }{B}_{1}w(k)+w_0^T(k)B_0^T{X}_{\infty }{B}_0{w}_0(k)\end{array}$$

Thus, we have

$$\begin{array}{l}{J}_E=\underset{T\to \infty }{\lim }\frac{1}{T}E\{{\displaystyle \sum _{k=0}^T(x^T(k)Qx(k)+u^T(k)Ru(k)-{\gamma }^2}{\Vert w\Vert }^2)\}\\ =\underset{T\to \infty }{\lim }\frac{1}{T}E\{{\displaystyle \sum _{k=0}^T(-\Delta V(x(k))-{\Vert z\Vert }^2}-{\gamma }^2{\Vert U_1^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}(w-{\gamma }^{-2}{U}_1^{-1}{B}_K^T{X}_{\infty }{A}_{K}x)\Vert }^2\\ +2w_0^T(k)B_0^T{X}_{\infty }{A}_{K}x(k)+2w_0^T(k)B_0^T{X}_{\infty }{B}_{1}w(k)+w_0^T(k)B_0^T{X}_{\infty }{B}_0{w}_0(k))\}\\ \le \underset{T\to \infty }{\lim }\frac{1}{T}E\{{\displaystyle \sum _{k=0}^T(-\Delta V(x(k))+2w_0^T(k)B_0^T{X}_{\infty }}{A}_{K}x(k)+2w_0^T(k)B_0^T{X}_{\infty }{B}_{1}w(k)\\ +w_0^T(k)B_0^T{X}_{\infty }{B}_0{w}_0(k))\}\end{array}$$

Note that $$x(\infty )={\lim }_{T\to \infty }x(T)=0$$ and

$$\begin{array}{l}x(k)=A_K^k{x}_0+{\displaystyle \sum _{i=0}^{k-1}{A}_K^{k-i-1}{B}_0{w}_0(i)+{\displaystyle \sum _{i=0}^{k-1}{A}_K^{k-i-1}{B}_1}}w(i)\\ w_0^T(k)B_0^T{X}_{\infty }{A}_{K}x(k)=w_0^T(k)B_0^T{X}_{\infty }{A}_K{A}_K^k{x}_0\\ +w_0^T(k)B_0^T{X}_{\infty }{A}_K{\displaystyle \sum _{i=0}^{k-1}{A}_K^{k-i-1}{B}_0{w}_0(i)+w_0^T(k)B_0^T{X}_{\infty }}{A}_K{\displaystyle \sum _{i=0}^{k-1}{A}_K^{k-i-1}{B}_{0}w(i)}\end{array}$$

we have

$$\begin{array}{r}E\{{\displaystyle \sum _{k=0}^T{w}_0^T(k)B_0^T{X}_{\infty }}{A}_{K}x(k)\}=E\{{\displaystyle \sum _{k=0}^T(w_0^T(k)B_0^T{X}_{\infty }}{A}_K{\displaystyle \sum _{i=0}^{k-1}{A}_K^{k-i-1}{B}_0{w}_0(i))\}}\\ ={\displaystyle \sum _{k=0}^T{\displaystyle \sum _{i=0}^{k-1}tr\{B_0^T{X}_{\infty }}}{A}_K^{k-i}{B}_{0}E(w_0(i)w_0^T(k))\}=0\end{array}$$

Based on the above, it follows from Lemma 3.1 that

$$\begin{array}{l}\underset{w\in L_{2+}}{\sup }\left\{J_E\right\}=\underset{T\to \infty }{\lim }\frac{1}{T}E\{x^T(0)X_{\infty }x(0)+{\displaystyle \sum _{k=0}^T{w}_0^T(k)B_0^T{X}_{\infty }}{B}_0{w}_0(k)\}\\ =\underset{T\to \infty }{\lim }\frac{1}{T}\{{\overline{x}}_0^T{X}_{\infty }{\overline{x}}_0+tr(X_{\infty }{R}_0)+{\displaystyle \sum _{k=0}^{T}tr(B_0^T{X}_{\infty }}{B}_0{R}_1(k))\}\\ =\underset{T\to \infty }{\lim }\frac{1}{T}{\displaystyle \sum _{k=0}^{T}tr(B_0^T{X}_{\infty }}{B}_0{R}_1(k))\end{array}$$

Thus, we conclude that

$$\underset{w\in L_{2+}}{\sup }\left\{J_E\right\}=\underset{T\to \infty }{\lim }\frac{1}{T}{\displaystyle \sum _{k=0}^{T}tr(B_0^T{X}_{\infty }}{B}_0{R}_1(k))$$ subject to $${\Vert T_{zw}\Vert }_{\infty }<\gamma$$ Q.E.D.

In the rest of this section, we give several discussions.

A. A Central Discrete-Time State Feedback Stochastic Mixed LQR/ $$H_{\infty }$$ Controller

We are to find a central solution to the discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem.This central solution involves the discrete-time Riccati equation

$$A^T{X}_{\infty }A-X_{\infty }-A^T{X}_{\infty }\widehat{B}{({\widehat{B}}^T{X}_{\infty }\widehat{B}+\widehat{R})}^{-1}{\widehat{B}}^T{X}_{\infty }A+C_1^T{C}_1+Q=0$$ E24

where, $$\widehat{B}=\left[\begin{array}{cc}{\gamma }^{-1}{B}_1& B_2\end{array}\right]$$ , $$\widehat{R}=\left[\begin{array}{cc}-I& 0\\ 0& R+I\end{array}\right]$$ . Using the similar argument as in the proof of Theorem 3.1 in Xu (2011), the expected cost function $$J_E$$ can be rewritten as:

$$\begin{array}{l}{J}_E=\underset{T\to \infty }{\lim }\frac{1}{T}E\{{\displaystyle \sum _{k=0}^T(x^T(k)Qx(k)+u^T(k)Ru(k)-{\gamma }^2{\Vert w\Vert }^2)\}}\\ =\underset{T\to \infty }{\lim }\frac{1}{T}E\{{\displaystyle \sum _{k=0}^T[-\Delta V(x(k))-{\Vert z\Vert }^2}-{\gamma }^2{\Vert U_1^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}(w-{\gamma }^{-2}{U}_1^{-1}{B}_K^T{X}_{\infty }{A}_{K}x)\Vert }^2\\ +x^T(A_K^T{X}_{\infty }{A}_K-X_{\infty }+{\gamma }^{-2}{A}_K^T{X}_{\infty }{B}_K{U}_1^{-1}{B}_K^T{X}_{\infty }{A}_K+C_K^T{C}_K+Q+K^{T}RK)x\\ +2w_0^T(k)B_0^T{X}_{\infty }{A}_{K}x(k)+2w_0^T(k)B_0^T{X}_{\infty }{B}_{1}w(k)+w_0^T(k)B_0^T{X}_{\infty }{B}_0{w}_0(k)]\}\end{array}$$ E25

Note that

$$\begin{array}{l}{A}_K^T{X}_{\infty }{A}_K-X_{\infty }+{\gamma }^{-2}{A}_K^T{X}_{\infty }{B}_K{U}_1^{-1}{B}_K^T{X}_{\infty }{A}_K+C_K^T{C}_K+Q+K^{T}RK\\ =A^T{X}_{\infty }A-X_{\infty }-A^T{X}_{\infty }\widehat{B}{\left({\widehat{B}}^T{X}_{\infty }\widehat{B}+\widehat{R}\right)}^{-1}{\widehat{B}}^T{X}_{\infty }A+C_1^T{C}_1+Q\\ +{\left(K+U_2^{-1}{B}_2^T{U}_{3}A\right)}^T{U}_2\left(K+U_2^{-1}{B}_2^T{U}_{3}A\right)\end{array}$$ E26

It follows from (25) and (26) that

$$\begin{array}{l}{J}_E=\underset{T\to \infty }{\lim }\frac{1}{T}E\{{\displaystyle \sum _{k=0}^T(x^T(k)Qx(k)+u^T(k)Ru(k))\}}\\ =\underset{T\to \infty }{\lim }\frac{1}{T}E\{{\displaystyle \sum _{k=0}^T[-\Delta V(x(k))-{\Vert z\Vert }^2}+{\gamma }^2{\Vert w\Vert }^2-{\gamma }^2{\Vert U_1^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}(w-{\gamma }^{-2}{U}_1^{-1}{B}_K^T{X}_{\infty }{A}_{K}x)\Vert }^2\\ +{\Vert U_2^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}(K+U_2^{-1}{B}_2^T{U}_{3}A)x\Vert }^2+2w_0^T(k)B_0^T{X}_{\infty }{A}_{K}x(k)+w_0^T(k)B_0^T{X}_{\infty }{B}_0{w}_0(k)]\}\end{array}$$ E27

If $$K=-U_2^{-1}{B}_2^T{U}_{3}A$$ , then we get that

$$\begin{array}{l}\underset{w\in L_{2+}}{\sup }\underset{K}{\inf }\left\{J_E\right\}=\underset{T\to \infty }{\lim }\frac{1}{T}\{{\overline{x}}_0^T{X}_{\infty }{\overline{x}}_0+tr(X_{\infty }{R}_0)+{\displaystyle \sum _{k=0}^{T}tr(B_0^T{X}_{\infty }}{B}_0{R}_1(k))\}\\ =\underset{T\to \infty }{\lim }\frac{1}{T}{\displaystyle \sum _{k=0}^{T}tr(B_0^T{X}_{\infty }}{B}_0{R}_1(k))\end{array}$$

by using Lemma 3.1 and the similar argument as in the proof of Theorem 3.1. Thus, we have the following theorem:

Theorem 3.2 There exists a central discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ controller if the discrete-time Riccati equation (24) has a stabilizing solution $$X_{\infty }\ge 0$$ and $$U_1=I-{\gamma }^{-2}{B}_1^T{X}_{\infty }{B}_1>0$$ .

Moreover, if this condition is met, the central discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ controller is given by

$$K=-U_2^{-1}{B}_2^T{U}_{3}A$$

where, $$U_3=X_{\infty }+{\gamma }^{-2}{X}_{\infty }{B}_1{U}_1^{-1}{B}_1^T{X}_{\infty }$$ , $$U_2=R+I+B_2^T{U}_3{B}_2$$ .

In this case, the central discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ controller will achieve

$$\underset{w\in L_{2+}}{\sup }\underset{K}{\inf }\left\{J_E\right\}=\underset{T\to \infty }{\lim }\frac{1}{T}{\displaystyle \sum _{k=0}^{T}tr(B_0^T{X}_{\infty }}{B}_0{R}_1(k))$$ subject to $${\Vert T_{zw}\Vert }_{\infty }<\gamma$$

Remark 3.5 When $$\Delta K=0$$ , Theorem 3.1 reduces to Theorem 3.2.

Remark 3.6 Notice that the condition displayed in Theorem 3.2 is the same as one displayed in Theroem 2.2. This implies that the result given by Theorem 3.2 may be recognied to be a stochastic interpretation of the discrete-time state feedback mixed LQR/ $$H_{\infty }$$ control problem considered by Xu (2011).

B. Numerical Algorithm

In order to calculate a kind of discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ controllers, we propose the following numerical algorithm.

Algorithm 3.1

Step 1: Fix the two weighting matrices $$Q$$ and $$R$$ , set $$i=0$$ , $$\Delta K_i=0$$ , $$U_{2(i)}=0$$ and a small scalar $$\delta$$ , and a matrix $$M$$ which is not zero matrix of appropriate dimensions.

Step 2: Solve the discrete-time Riccati equation

$$A^T{X}_{i}A-X_i-A^T{X}_i\widehat{B}{({\widehat{B}}^T{X}_i\widehat{B}+\widehat{R})}^{-1}{\widehat{B}}^T{X}_{i}A+C_1^T{C}_1+Q+\Delta K_i^T{U}_{2(i)}\Delta K_i=0$$

for $$X_i$$ symmetric non-negative definite such that

$${\widehat{A}}_{ci}=A-\widehat{B}{({\widehat{B}}^T{X}_i\widehat{B}+\widehat{R})}^{-1}{\widehat{B}}^T{X}_{i}A$$ is stable and $$U_{1(i)}=I-{\gamma }^{-2}{B}_1^T{X}_i{B}_1>0$$ .

Step 3: Calculate $$U_{3(i)}$$ , $$U_{2(i)}$$ and $$K_i$$ by using the following formulas

$$\begin{array}{l}{U}_{3(i)}=X_i+{\gamma }^{-2}{X}_i{B}_1{U}_{1(i)}^{-1}{B}_1^T{X}_i\\ U_{2(i)}=R+I+B_2^T{U}_{3(i)}{B}_2\\ K_i=-U_{2(i)}^{-1}{B}_2^T{U}_{3(i)}A+\Delta K_i\end{array}$$ E28

Step 4: Let $$\Delta K_{i+1}=\Delta K_i+\delta M$$ (or $$\Delta K_{i+1}=\Delta K_i-\delta M$$ ) and $$U_{2(i+1)}=U_{2(i)}$$ .

Step 5: If $$A_i=A+B_2{K}_i$$ is stable, that is, $$\Delta K_i$$ is an admissible controller error, then increase $$i$$ by $$1$$ , goto Step 2; otherwise stop.

Using the above algorithm, we obtain a kind of discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ controllers as follows:

$$\begin{array}{c}{K}_i=-U_{2(i)}^{-1}{B}_2^T{U}_{3(i)}A\pm i\delta M\\ (i=\mathrm{0,1,2,}\cdots ,n,\cdots )\end{array}$$

C. Comparison with Related Well Known Results

Comparing the result displayed in Theorem 3.1 with the earlier results, such as, Geromel & Peres (1985), Geromel et al. (1989), de Souza & Xie (1992), Kucera & de Souza (1995) and Gadewadikar et al. (2007); we know easily that all these earlier results are given in terms of a single algebraic Riccati equation with a free parameter matrix, plus a free parameter constrained condition on the form of the gain matrix. Although the result displayed in Theorem 3.1 is also given in terms of a single algebraic Riccati equation with a free parameter matrix, plus a free parameter constrained condition on the form of the gain matrix; but the free parameter matrix is also constrained to be an admissible controller error. In order to give some interpretation for this fact, we provided the following result of discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem by combining directly the proof of Theorem 3.1, and the technique of finding all admissible state feedback controllers by Geromel & Peres (1985) ( also see Geromel et al. 1989, de Souza & Xie 1992, Kucera & de Souza 1995).

Theorem 3.3 There exists a state feedback stochastic mixed LQR/ $$H_{\infty }$$ controller if there exists a matrix $$L$$ such that

$$L=K+U_2^{-1}{B}_2^T{U}_{3}A$$ E29

and $$X_{\infty }$$ is a symmetric non-negative definite solution of the following discrete-time Riccati equation

$$\begin{array}{l}{A}^T{X}_{\infty }A-X_{\infty }-A^T{X}_{\infty }\widehat{B}{\left({\widehat{B}}^T{X}_{\infty }\widehat{B}+\widehat{R}\right)}^{-1}{\widehat{B}}^T{X}_{\infty }A\\ +C_1^T{C}_1+Q+L^T{U}_{2}L=0\end{array}$$ E30

and $$A_K+{\gamma }^{-2}{B}_K{U}_1^{-1}{B}_K^T{X}_{\infty }{A}_K$$ is stable and $$U_1=I-{\gamma }^{-2}{B}_1^T{X}_{\infty }{B}_1>0$$ .

Where, $$\widehat{B}=\left[\begin{array}{cc}{\gamma }^{-1}{B}_1& B_2\end{array}\right]$$ , $$\widehat{R}=\left[\begin{array}{cc}-I& 0\\ 0& R+I\end{array}\right]$$ , $$U_3=X_{\infty }+{\gamma }^{-2}{X}_{\infty }{B}_1{U}_1^{-1}{B}_1^T{X}_{\infty }$$ , $$U_2=R+I+$$ $$B_2^T{U}_3{B}_2$$ .

Note that $$A_K=A_{K^{\ast }}+B_2\Delta K$$ , $$K=K^{\ast }+\Delta K$$ and

$${\widehat{A}}_c=A-\widehat{B}{({\widehat{B}}^T{X}_{\infty }\widehat{B}+\widehat{R})}^{-1}{\widehat{B}}^T{X}_{\infty }A=A_{K^{\ast }}+{\gamma }^{-2}{B}_{K^{\ast }}{U}_1^{-1}{B}_{K^{\ast }}^T{X}_{\infty }{A}_{K^{\ast }}$$

This implies that $$A_K+{\gamma }^{-2}{B}_K{U}_1^{-1}{B}_K^T{X}_{\infty }{A}_K$$ is stable if $${\widehat{A}}_c$$ is stable and $$\Delta K$$ is an admissible controller error. Thus we show easily that in the case of $$\Delta K=L$$ , there exists a matrix $$L$$ such that (29) holds, where, $$X_{\infty }$$ is a symmetric non-negative definite solution of discrete-time Riccati equation (30) and $$A_K+{\gamma }^{-2}{B}_K{U}_1^{-1}{B}_K^T{X}_{\infty }{A}_K$$ is stable if the conditions i-ii of Theorem 3.1 hold.

At the same time, we can show also that if $$\Delta K=L$$ is an admissble controller error, then the calculation of the algotithm 3.1 will become easilier. For an example, for a given admissible controller error $$\Delta K_i$$ , the step 2 of algorithm 3.1 is to solve the discrete-time Riccati equation

$$A^T{X}_{i}A-X_i-A^T{X}_i\widehat{B}{({\widehat{B}}^T{X}_i\widehat{B}+\widehat{R})}^{-1}{\widehat{B}}^T{X}_{i}A+\widehat{Q}=0$$

for $$X_i$$ being a stabilizing solution, where, $$\widehat{Q}=C_1^T{C}_1+Q+\Delta K_i^T{U}_{2(i)}\Delta K_i$$ . Since $${\widehat{A}}_{ci}=A-$$ $$\widehat{B}{({\widehat{B}}^T{X}_i\widehat{B}+\widehat{R})}^{-1}{\widehat{B}}^T{X}_{i}A$$ is stable and $$\Delta K_i$$ is an admissible controller error, so $$A_K+{\gamma }^{-2}{B}_K{U}_{1(i)}^{-1}{B}_K^T{X}_i{A}_K$$ is stable. This implies the condition ii displayed in Theorem 3.1 makes the calculation of the algorithm 3.1 become easier.

4. Static Output Feedback

This section consider discrete-time static output feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem. This problem is defined as follows:

Discrete-time static ouput feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem: Consider the system (15) under the influence of static output feedback of the form

$$u(k)=F_{\infty }y(k)$$

with $$w\in L_2[\mathrm{0,}\infty )$$ , for a given number $$\gamma >0$$ , determine an admissible static output feedback controller $$F_{\infty }$$ such that

$$\underset{w\in L_{2+}}{\sup }\left\{J_E\right\}$$ subject to $${\Vert T_{zw}\Vert }_{\infty }<\gamma$$

If this admissible controller exists, it is said to be a discrete-time static output feedback stochastic mixed LQR/ $$H_{\infty }$$ controller. As is well known, the discrete-time static output feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem is equivalent to the discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem for the systems (15) (16), where, $$K$$ is constrained to have the form of $$K=F_{\infty }{C}_2$$ . This problem is also said to be a structural constrained state feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem.Based the above, we can obtain all solution to discrete-time static output feedback stochastic mixed LQR/ $$H_{\infty }$$ control problem by using the result of Theorem 3.1 as follows:

Theorem 4.1 There exists a discrete-time static output feedback stochastic mixed LQR/ $$H_{\infty }$$ controller if the following two conditions hold:

i.There exists a matrix $$\Delta K$$ such that

$$\Delta K=F{C}_2+U_2^{-1}{B}_2^T{U}_{3}A$$ E31

and $$X_{\infty }$$ is a symmetric non-negative definite solution of the following discrete-time Riccati equation

$$\begin{array}{l}{A}^T{X}_{\infty }A-X_{\infty }-A^T{X}_{\infty }\widehat{B}{\left({\widehat{B}}^T{X}_{\infty }\widehat{B}+\widehat{R}\right)}^{-1}{\widehat{B}}^T{X}_{\infty }A\\ +C_1^T{C}_1+Q+\Delta K^T{U}_2\Delta K=0\end{array}$$ E32

and $${\widehat{A}}_c=A-\widehat{B}{({\widehat{B}}^T{X}_{\infty }\widehat{B}+\widehat{R})}^{-1}{\widehat{B}}^T{X}_{\infty }A$$ is stable and $$U_1=I-{\gamma }^{-2}{B}_1^T{X}_{\infty }{B}_1>0$$ .

Where, $$\widehat{B}=\left[\begin{array}{cc}{\gamma }^{-1}{B}_1& B_2\end{array}\right]$$ , $$\widehat{R}=\left[\begin{array}{cc}-I& 0\\ 0& R+I\end{array}\right]$$ , $$U_3=X_{\infty }+{\gamma }^{-2}{X}_{\infty }{B}_1{U}_1^{-1}{B}_1^T{X}_{\infty }$$ , $$U_2=R+I+$$ $$B_2^T{U}_3{B}_2$$ .

ii. $$\Delta K$$ is an admissible controller error.

In this case, the discrete-time static output feedback stochastic mixed LQR/ $$H_{\infty }$$ controller will achieve

$$\underset{w\in L_{2+}}{\sup }\left\{J_E\right\}=\underset{T\to \infty }{\lim }\frac{1}{T}{\displaystyle \sum _{k=0}^{T}tr(B_0^T{X}_{\infty }}{B}_0{R}_1(k))$$ subject to $${\Vert T_{zw}\Vert }_{\infty }<\gamma$$

Remark 4.1 In Theorem 4.1, define a suboptimal controller as $$K^{\ast }=-U_2^{-1}{B}_2^T{U}_{3}A$$ , then $$\Delta K=F_{\infty }{C}_2-K^{\ast }$$ . As is discussed in Remark 3.1, suppose that there exists a suboptimal controller $$K^{\ast }$$ such that $$A_{K^{\ast }}=A+B_2{K}^{\ast }$$ is stable, then $$\Delta K$$ is an admissible controller error if it belongs to the set:

$$\Omega :=$$ { $$\Delta K:A+B_2{F}_{\infty }{C}_2$$ is stable}

It should be noted that Theorem 4.1 does not tell us how to calculate a discrete-time static output feedback stochastic mixed LQR/ $$H_{\infty }$$ controller $$F_{\infty }$$ . In order to do this, we present, based on the algorithms proposed by Geromel & Peres (1985) and Kucera & de Souza (1995), a numerical algorithm for computing a discrete-time static output feedback stochastic mixed LQR/ $$H_{\infty }$$ controller $$F_{\infty }$$ and a solution $$X_{\infty }$$ to discrete-time Riccati equation (32). This numerical algorithm is given as follows:

Algorithm 4.1

Step 1: Fix the two weighting matrices $$Q$$ and $$R$$ , set $$i=0$$ , $$\Delta K_i=0$$ , and $$U_{2(i)}=0$$ .

Step 2: Solve the discrete-time Riccati equation

$$\begin{array}{l}{A}^T{X}_{i}A-X_i-A^T{X}_i\widehat{B}{\left({\widehat{B}}^T{X}_i\widehat{B}+\widehat{R}\right)}^{-1}{\widehat{B}}^T{X}_{i}A\\ +C_1^T{C}_1+Q+\Delta K_i^T{U}_{2(i)}\Delta K_i=0\end{array}$$

for $$X_i$$ symmetric non-negative definite such that

$${\widehat{A}}_{ci}=A-\widehat{B}{({\widehat{B}}^T{X}_i\widehat{B}+\widehat{R})}^{-1}{\widehat{B}}^T{X}_{i}A$$ is stable and $$U_{1(i)}=I-{\gamma }^{-2}{B}_1^T{X}_i{B}_1>0$$ .

Step 3: Calculate $$U_{3(i+1)}$$ , $$U_{2(i+1)}$$ and $$\Delta K_{i+1}$$ by using the following formulas

$$\begin{array}{l}{U}_{3(i+1)}=X_i+{\gamma }^{-2}{X}_i{B}_1{U}_{1(i)}^{-1}{B}_1^T{X}_i\\ U_{2(i+1)}=R+I+B_2^T{U}_{3(i+1)}{B}_2\\ \Delta K_{i+1}=-U_{2(i+1)}^{-1}{B}_2^T{U}_{3(i+1)}A\left(C_2^T{\left(C_2{C}_2^T\right)}^{-1}{C}_2-I\right)\end{array}$$

Step 4: If $$\Delta K_{i+1}$$ is an admissible controller error, then increase $$i$$ by $$1$$ , and goto Step 2; otherwise stop.

If the four sequences $$X_0,X_1,\cdots ,X_i,\cdots$$ , $$U_{1(1)},U_{1(2)},\cdots ,U_{1(i)},\cdots$$ , $$U_{2(1)},U_{2(2)},\cdots ,U_{2(i)},\cdots$$ , and $$U_{3(1)},U_{3(2)},\cdots ,U_{3(i)},\cdots$$ converges, say to $$X_{\infty }$$ , $$U_1$$ , $$U_2$$ and $$U_3$$ , respectively; then the both two conditions displayed in Theorem 4.1 are met. In this case, a discrete-time static output feedback stochastic mixed LQR/ $$H_{\infty }$$ controllers is parameterized as follows:

$$F_{\infty }=-U_2^{-1}{B}_2^T{U}_{3}A{C}_2^T{(C_2{C}_2^T)}^{-1}$$

In this chapter, we will not prove the convergence of the above algorithm. This will is another subject.

5. Numerical Examples

In this section, we present two examples to illustrate the design methods displayed in Section 3 and 4 respectively.

Example 5.1 Consider the following linear discrete-time system (15) under the influence of state feedback of the form $$u(k)=Kx(k)$$ , its parameter matrices are

$$\begin{array}{l}A=\left[\begin{array}{cc}0& 2\\ 4& 0.2\end{array}\right],B_0=\left[\begin{array}{c}0.1\\ 0.2\end{array}\right],B_1=\left[\begin{array}{c}0.5\\ 0.3\end{array}\right],B_2=\left[\begin{array}{c}1\\ 0\end{array}\right]\\ C_1=\left[\begin{array}{cc}1& 0\\ 0& 0\end{array}\right],C_2=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right],D_{12}=\left[\begin{array}{c}0\\ 1\end{array}\right]\end{array}$$

The above system satisfies Assumption 3.1-3.3, and the open-loop poles of this system are $$p_1=$$ $$-2.7302$$ , $$p_2=2.9302$$ ; thus it is open-loop unstable.

Let $$R=1$$ , $$Q=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right]$$ , $$\gamma =9.5$$ , $$\delta =0.01$$ , $$M=\left[\begin{array}{cc}-0.04& -1.2\end{array}\right]$$ ; by using algorithm 3.1, we solve the discrete-time Riccati equation (20) to get $$X_i$$ , $$U_{1(i)}$$ , $$K_i$$ ( $$i=\mathrm{0,1,2,}\cdots \mathrm{,10})$$ and the corresponding closed-loop poles. The calculating results of algorithm 3.1 are listed in Table 1.

It is shown in Table 1 that when the iteration index $$i=10$$ , $$X_{10}0$$ and $$U_{1(10)}=-0.2927$$ $$0$$ , thus the discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ controller does not exist in this case. Of course, Table 1 does not list all discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ controllers because we do not calculating all these controllers by using Algorithm in this example. In order to illustrate further the results, we give the trajectories of state of the system (15) with the state feedback of the form $$u(k)=Kx(k)$$ for the resulting discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ controller $$K=$$ $$\left[\begin{array}{cc}-0.3071& -2.0901\end{array}\right]$$ . The resulting closed-loop system is

$$\begin{array}{l}x(k+1)=(A+B_{2}K)x(k)+B_0{w}_0(k)+B_{1}w(k)\\ z(k)=(C_1+D_{12}K)x(k)\end{array}$$

where, $$A+B_{2}K=\left[\begin{array}{cc}-0.3071& -0.0901\\ 4.0000& 0.2000\end{array}\right]$$ , $$C_1+D_{12}K=\left[\begin{array}{cc}1.0000& 0\\ -0.3071& -2.0901\end{array}\right]$$ .

IterationIndex $$i$$	Solution of DARE $$X_i$$	AdditionalCondition $$U_{1(i)}$$	State FeedbackController $$K_i$$	The Closed-Loop Poles
0	$$X_0>0$$	$$0.5571$$	$$\left[\begin{array}{cc}-0.2886& -1.9992\end{array}\right]$$	$$p_1=-0.2951$$,; $$p_2=0.2065$$
1	$$X_1>0$$	$$0.5553$$	$$\left[\begin{array}{cc}-0.2892& -2.0113\end{array}\right]$$	$$p_1=-0.1653$$,; $$p_2=0.0761$$
2	$$X_2>0$$	$$0.5496$$	$$\left[\begin{array}{cc}-0.2903& -2.0237\end{array}\right]$$	$$p_{\mathrm{1,2}}=-0.0451$$; $$\pm j0.1862$$
3	$$X_3>0$$	$$0.5398$$	$$\left[\begin{array}{cc}-0.2918& -2.0363\end{array}\right]$$	$$p_{\mathrm{1,2}}=-0.0459$$; $$\pm j0.2912$$
4	$$X_4>0$$	$$0.5250$$	$$\left[\begin{array}{cc}-0.2940& -2.0492\end{array}\right]$$	$$p_{\mathrm{1,2}}=-0.0470$$; $$\pm j0.3686$$
5	$$X_5>0$$	$$0.5040$$	$$\left[\begin{array}{cc}-0.2969& -2.0624\end{array}\right]$$	$$p_{\mathrm{1,2}}=-0.0485$$; $$\pm j0.4335$$
6	$$X_6>0$$	$$0.4739$$	$$\left[\begin{array}{cc}-0.3010& -2.0760\end{array}\right]$$	$$p_{\mathrm{1,2}}=-0.0505$$; $$\pm j0.4911$$
7	$$X_7>0$$	$$0.4295$$	$$\left[\begin{array}{cc}-0.3071& -2.0901\end{array}\right]$$	$$p_{\mathrm{1,2}}=-0.0535$$; $$\pm j0.5440$$
8	$$X_8>0$$	$$0.3578$$	$$\left[\begin{array}{cc}-0.3167& -2.1049\end{array}\right]$$	$$p_{\mathrm{1,2}}=-0.0584$$; $$\pm j0.5939$$
9	$$X_9>0$$	$$0.2166$$	$$\left[\begin{array}{cc}-0.3360& -2.1212\end{array}\right]$$	$$p_{\mathrm{1,2}}=-0.0680$$; $$\pm j0.6427$$
10	$$X_{10}>0$$	$$-0.2927$$

Table 1.

The calculating results of algorithm 3.1.

To determine the mean value function, we take mathematical expectation of the both hand of the above two equations to get

$$\begin{array}{l}\overline{x}(k+1)=(A+B_{2}K)\overline{x}(k)+B_{1}w(k)\\ \overline{z}(k)=(C_1+D_{12}K)\overline{x}(k)\end{array}$$

where, $$E\{x(k)\}=\overline{x}(k)$$ , $$E\{z(k)\}=\overline{z}(k)$$ , $$E\{x(0)\}={\overline{x}}_0$$ .

Let $$w(k)={\gamma }^{-2}{U}_1^{-1}{B}_1^T{X}_{\infty }(A+B_{2}K)\overline{x}(k)$$ , then the trajectories of mean values of states of resulting closed-loop system with $${\overline{x}}_0={\left[\begin{array}{cc}3& 2\end{array}\right]}^T$$ are given in Fig. 1.

Figure 1.

The trajectories of mean values of states of resulting system in Example 5.1.

Example 5.2 Consider the following linear discrete-time system (15) with static output feedback of the form $$u(k)=F_{\infty }y(k)$$ , its parameter matrices are as same as Example 5.1.

When $$C_2$$ is quare and invertible, that is, all state variable are measurable, we may assume without loss of generality that $$C_2=I$$ ; let $$\gamma =6.5$$ , $$R=1$$ and $$Q=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right]$$ , by solving the discrete-time Riccati equation (24), we get that the central discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ controller displayed in Theorem 3.2 is

$$K^{\ast }=\left[\begin{array}{cc}-0.3719& -2.0176\end{array}\right]$$

and the poles of resulting closed-loop system are $$p_1=-0.1923$$ , $$p_2=0.0205$$ .

When $$C_2=\left[\begin{array}{cc}1& 5.4125\end{array}\right]$$ , let $$\gamma =6.5$$ , $$R=1$$ , $$Q=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right]$$ , by using Algorithm 4.1, we solve the discrete-time Riccati equation (32) to get

$$X_{\infty }=\left[\begin{array}{cc}148.9006& 8.8316\\ 8.8316& 9.5122\end{array}\right]>\mathrm{0,}{U}_1=0.0360$$

Thus the discrete-time static output feedback stochastic mixed LQR/ $$H_{\infty }$$ controller displayed in Theorem 4.1 is $$F_{\infty }=-0.3727$$ . The resulting closed-loop system is

$$\begin{array}{l}x(k+1)=(A+B_2{F}_{\infty }{C}_2)x(k)+B_0{w}_0(k)+B_{1}w(k)\\ z(k)=(C_1+D_{12}{F}_{\infty }{C}_2)x(k)\end{array}$$

where, $$A+B_2{F}_{\infty }{C}_2=\left[\begin{array}{cc}-0.3727& -0.0174\\ 4.0000& 0.2000\end{array}\right]$$ , $$C_1+D_{12}{F}_{\infty }{C}_2=\left[\begin{array}{cc}1.0000& 0\\ -0.3727& -2.0174\end{array}\right]$$ .

Taking mathematical expectation of the both hand of the above two equations to get

$$\begin{array}{l}\overline{x}(k+1)=(A+B_2{F}_{\infty }{C}_2)\overline{x}(k)+B_{1}w(k)\\ z(k)=(C_1+D_{12}{F}_{\infty }{C}_2)\overline{x}(k)\end{array}$$

where, $$E\{x(k)\}=\overline{x}(k)$$ , $$E\{z(k)\}=\overline{z}(k)$$ , $$E\{x(0)\}={\overline{x}}_0$$ .

Let $$w(k)={\gamma }^{-2}{U}_1^{-1}{B}_1^T{X}_{\infty }(A+B_2{F}_{\infty }{C}_2)\overline{x}(k)$$ , then the trajectories of mean values of states of resulting closed-loop system with $${\overline{x}}_0={\left[\begin{array}{cc}1& 2\end{array}\right]}^T$$ are given in Fig. 2.

Figure 2.

The trajectories of mean values of states of resulting system in Example 5.2.

6. Conclusion

In this chapter, we provide a characterization of all state feedback controllers for solving the discrete-time stochastic mixed LQR/ $$H_{\infty }$$ control problem for linear discrete-time systems by the technique of Xu (2008 and 2011) with the well known LQG theory. Sufficient conditions for the existence of all state feedback controllers solving the discrete-time stochastic mixed LQR/ $$H_{\infty }$$ control problem are given in terms of a single algebraic Riccati equation with a free parameter matrix, plus two constrained conditions: One is a free parameter matrix constrained condition on the form of the gain matrix, another is an assumption that the free parameter matrix is a free admissible controller error. Also, a numerical algorithm for calculating a kind of discrete-time state feedback stochastic mixed LQR/ $$H_{\infty }$$ controllers are proposed. As one special case, the central discrete-time state feedback stochastic mixed LQR./ $$H_{\infty }$$ controller is given in terms of an algebraic Riccati equation. This provides an interpretation of discrete-time state feedback mixed LQR/ $$H_{\infty }$$ control problem. As another special case, sufficient conditions for the existence of all static output feedback controllers solving the discrete-time stochastic mixed LQR/ $$H_{\infty }$$ control problem are given. A numerical algorithm for calculating a static output feedback stochastic mixed LQR/ $$H_{\infty }$$ controller is also presented.