## 1. Introduction

Mixed

where,

Bernstein & Haddad (1989) presented a combined LQG/

determine an

which satisfies the following design criteria: (i) the closed-loop system (1) (3) is stable; (ii) the closed-loop transfer matrix

In this chapter, we consider state feedback stochastic mixed LQR/

On the other hand, Geromel & Peres (1985) showed a new stabilizability property of the Riccati equation solution, and proposed, based on this new property, a numerical procedure to design static output feedback suboptimal LQR controllers for linear continuous-time systems. Geromel et al. (1989) extended the results of Geromel & Peres (1985) to linear discrete-time systems. In the fact, comparing this new stabilizability property of the Riccati equation solution with the existing results (de Souza & Xie 1992, Kucera & de Souza 1995, Gadewadikar et al. 2007, Xu 2008), we can show easily that the former involves sufficient conditions for the existence of all state feedback suboptimal LQR controllers. Untill now, the technique of finding all state feedback controllers by Geromel & Peres (1985) has been extended to various control problems, such as, static output feedback stabilizability (Kucera & de Souza 1995),

The objective of this chapter is to solve discrete-time state feedback stochastic mixed LQR/

This chapter is organized as follows: Section 2 introduces several preliminary results. In Section 3, first,we define the state feedback stochastic mixed LQR/

## 2. Preliminaries

In this section, we will review several preliminary results. First, we introduce the new stabilizability property of Riccati equation solutions for linear discrete-time systems which was presented by Geromel et al. (1989). This new stabilizability property involves the following linear discrete-time systems

with quadratic performance index

under the influence of state feedback of the form

where,

*Assumption 2.1*

*Assumption 2.2*

Define a discrete-time Riccati equation as follows:

For simplicity the discrete-time Riccati equation (6) can be rewritten as

Geromel & Peres (1985) showed a new stabilizability property of the Riccati equation solution, and proposed, based on this new property, a numerical procedure to design static output feedback suboptimal LQR controllers for linear continuous-time systems. Geromel et al. (1989) extended this new stabilizability property displayed in Geromel & Peres (1985) to linear discrete-time systems. This resut is given by the following theorem.

*Theorem 2.1* (Geromel et al. 1989) For the matrix

holds,

Then the matrix

When these conditions are met, the quadratic cost function

Second, we introduce the well known discrete-time bounded real lemma (see Zhou et al., 1996; Iglesias & Glover, 1991; de Souza & Xie, 1992).

*Lemma 2.1 (Discrete Time Bounded Real Lemma)*

Suppose that

i.

ii. There exists a stabilizing solution

such that

Next, we will consider the following linear discrete-time systems

under the influence of state feedback of the form

where,

The associated with this systems is the quadratic performance index

where,

The closed-loop transfer matrix from the disturbance input

where,

The following lemma is an extension of the discrete-time bounded real lemma ( see Xu 2011).

*Lemma 2.2* Given the system (10) under the influence of the state feedback (11), and suppose that

such that

*Proof:* See the proof of Lemma 2.2 of Xu (2011). Q.E.D.

Finally, we review the result of discrete-time state feedback mixed LQR/

If this controller

The following assumptions are imposed on the system

*Assumption 2.3*

*Assumption 2.4*

*Assumption 2.5*

The solution to the problem defined in the above involves the discrete-time Riccati equation

where,

Xu (2011) has provided a solution to discrete-time state feedback mixed LQR/

*Theorem 2.2* There exists a discrete-time state feedback mixed LQR/

Moreover, this discrete-time state feedback mixed LQR/

where,

In this case, the discrete-time state feedback mixed LQR/

where,

## 3. State Feedback

In this section, we consider the following linear discrete-time systems

with state feedback of the form

where,

It is assumed that

The noise process

Furthermore,

Also, we make the following assumptions:

*Assumption 3.1*

*Assumption 3.2*

*Assumption 3.3*

The expected cost function corresponding to this problem is defined as follows:

where,

As is well known, a given controller

Recall that the discrete-time state feedback optimal LQG problem is to find an admissible controller that minimizes the expected quadratic cost function (17) subject to the systems (15) (16) with

because for all

Based on this, we define the discrete-time state feedback stochastic mixed LQR/

Discrete-time state feedback stochastic mixed LQR/

where,

If all these admissible controllers exist, then one of them

and it is said to be a central discrete-time state feedback stochastic mixed LQR/

*Remark 3.1* The discrete-time state feedback stochastic mixed LQR/

*Remark 3.2* In the case of

Define

Also, the controlled output

where,

This implies that the discrete-time state feedback combined LQG/

Based on the above definition, we give sufficient conditions for the existence of all admissible state feedback controllers solving the discrete-time stochastic mixed LQR/

*Theorem 3.1* There exists a discrete-time state feedback stochastic mixed LQR/

i. There exists a matrix

and

and

where,

ii.

In this case, the discrete-time state feedback stochastic mixed LQR/

*Remark 3.3* In Theorem 3.1, the controller error is defined to be the state feedback controller

where,

*Remark 3.4* The discrete-time state feedback stochastic mixed LQR/

Astrom (1971) has given the mean value of a quadratic form of normal stochastic variables. This result is given by the following lemma.

*Lemma 3.1* Let

For convenience, let

*Lemma 3.2* Suppose that the conditions i-ii of Theorem 3.1 hold, then the both

Proof: Suppose that the conditions i-ii of Theorem 3.1 hold, then it can be easily shown by using the similar standard matrix manipulations as in the proof of Theorem 3.1 in de Souza & Xie (1992) that

where,

Rearranging the discrete-time Riccati equation (20), we get

that is,

Since the discrete-time Riccati equation (20) has a symmetric non-negative definite solution

Proof of Theorem 3.1: Suppose that the conditions i-ii hold, then it follows from Lemma 3.2 that the both

Define

(22) |

On the other hand, we can rewrite the discrete-time Riccati equation (20) by using the same standard matrix manipulations as in the proof of Lemma 3.2 as follows:

or equivalently

It follows from Lemma 2.2 that

Thus, we have

Note that

we have

Based on the above, it follows from Lemma 3.1 that

Thus, we conclude that

In the rest of this section, we give several discussions.

*A. A Central Discrete-Time State Feedback Stochastic Mixed LQR/*
*Controller*

We are to find a central solution to the discrete-time state feedback stochastic mixed LQR/

where,

(25) |

Note that

(26) |

It follows from (25) and (26) that

(27) |

If

by using Lemma 3.1 and the similar argument as in the proof of Theorem 3.1. Thus, we have the following theorem:

*Theorem 3.2* There exists a central discrete-time state feedback stochastic mixed LQR/

Moreover, if this condition is met, the central discrete-time state feedback stochastic mixed LQR/

where,

In this case, the central discrete-time state feedback stochastic mixed LQR/

*Remark 3.5* When

*Remark 3.6* Notice that the condition displayed in Theorem 3.2 is the same as one displayed in Theroem 2.2. This implies that the result given by Theorem 3.2 may be recognied to be a stochastic interpretation of the discrete-time state feedback mixed LQR/

*B. Numerical Algorithm*

In order to calculate a kind of discrete-time state feedback stochastic mixed LQR/

*Algorithm 3.1*

Step 1: Fix the two weighting matrices

Step 2: Solve the discrete-time Riccati equation

for

Step 3: Calculate

Step 4: Let

Step 5: If

Using the above algorithm, we obtain a kind of discrete-time state feedback stochastic mixed LQR/

*C. Comparison with Related Well Known Results*

Comparing the result displayed in Theorem 3.1 with the earlier results, such as, Geromel & Peres (1985), Geromel et al. (1989), de Souza & Xie (1992), Kucera & de Souza (1995) and Gadewadikar et al. (2007); we know easily that all these earlier results are given in terms of a single algebraic Riccati equation with a free parameter matrix, plus a free parameter constrained condition on the form of the gain matrix. Although the result displayed in Theorem 3.1 is also given in terms of a single algebraic Riccati equation with a free parameter matrix, plus a free parameter constrained condition on the form of the gain matrix; but the free parameter matrix is also constrained to be an admissible controller error. In order to give some interpretation for this fact, we provided the following result of discrete-time state feedback stochastic mixed LQR/

*Theorem 3.3* There exists a state feedback stochastic mixed LQR/

and

and

Where,

Note that

This implies that

At the same time, we can show also that if

for

## 4. Static Output Feedback

This section consider discrete-time static output feedback stochastic mixed LQR/

Discrete-time static ouput feedback stochastic mixed LQR/

with

If this admissible controller exists, it is said to be a discrete-time static output feedback stochastic mixed LQR/

*Theorem 4.1* There exists a discrete-time static output feedback stochastic mixed LQR/

i.There exists a matrix

and

and

Where,

ii.

In this case, the discrete-time static output feedback stochastic mixed LQR/

*Remark 4.1* In Theorem 4.1, define a suboptimal controller as

It should be noted that Theorem 4.1 does not tell us how to calculate a discrete-time static output feedback stochastic mixed LQR/

*Algorithm 4.1*

Step 1: Fix the two weighting matrices

Step 2: Solve the discrete-time Riccati equation

for

Step 3: Calculate

Step 4: If

If the four sequences

In this chapter, we will not prove the convergence of the above algorithm. This will is another subject.

## 5. Numerical Examples

In this section, we present two examples to illustrate the design methods displayed in Section 3 and 4 respectively.

*Example 5.1* Consider the following linear discrete-time system (15) under the influence of state feedback of the form

The above system satisfies Assumption 3.1-3.3, and the open-loop poles of this system are

Let

It is shown in Table 1 that when the iteration index

where,

To determine the mean value function, we take mathematical expectation of the both hand of the above two equations to get

where,

Let

*Example 5.2* Consider the following linear discrete-time system (15) with static output feedback of the form

When

and the poles of resulting closed-loop system are

When

Thus the discrete-time static output feedback stochastic mixed LQR/

where,

Taking mathematical expectation of the both hand of the above two equations to get

where,

Let

## 6. Conclusion

In this chapter, we provide a characterization of all state feedback controllers for solving the discrete-time stochastic mixed LQR/