## Abstract

In this chapter, the author presents a theoretical result on the optimal control of nonlinear dynamic systems. In this theoretical result, the author presents the optimal control problem for nonlinear dynamic systems and shows that this problem can be solved by utilizing the dynamic programming approach and the inverse optimal approach. The author employs the dynamic programming approach to derive the Hamilton-Jacobi-Bellman (H-J-B) equation associated with the optimal control problem for nonlinear dynamic systems. Then, the author presents an analytic way to solve the H-J-B equation with the help of the inverse optimal approach. Based on the theoretical result presented in this chapter, the author establishes an optimal control design for TS-type fuzzy systems that guarantees the global asymptotic stability of an equilibrium point and the optimality with respect to a cost function and provides good convergence rates of state trajectories to an equilibrium point. The author considers the three-axis attitude stabilization problem of a rigid body to illustrate the optimal control design method for TS-type fuzzy systems. The author designs the optimal three-axis attitude stabilizing control law for a rigid body based on this optimal control design method and analyzes its control performance by numerical simulations.

### Keywords

- intelligent system design
- nonlinear optimal control
- fuzzy systems
- rigid body motion
- rigid body attitude control

## 1. Introduction

Since Lotfi Aliasker Zadeh introduced the fuzzy set [1] and the fuzzy logic [2], these two concepts have been successfully applied to various kinds of fields existing in the earth (e.g., see references in [3]), and the usefulness of these two concepts have also been verified in many industries.

Fuzzy logic is basically a multi-valued logic that allows intermediate values to be defined between conventional evaluations like yes or no, true or false, and big or small. The main characteristic of fuzzy concept is that it can handle some complicated phenomena of systems with the help of the fuzzy and linguistic modeling. Thus, it is possible to design knowledge-oriented intelligent systems if we use the fuzzy logic in the design of systems. This is a very important characteristic of fuzzy logic.

Most physical systems are almost nonlinear dynamic systems. Conventional control design approaches use different approximation methods such as linear, piecewise linear, and lookup table approximations to handle their nonlinearities. The linear approximation method linearizes a nonlinear dynamic system about a single equilibrium point and provides a linearized design-model for it. Then, the controller is designed for the linearized design-model to satisfy a given control objective. The linear approximation method is relatively simple, but it tends to limit control performance. In addition, the controller designed by the linear approximation method is valid only under the assumption that the states of a nonlinear dynamic system operate closely around the considered equilibrium point, which is the basic limitation of the control design by the linear approximation method [4]. The piecewise linear approximation method requires the design of several linear controllers. Thus, it works better than the linear approximation method, although it is tedious to implement. The lookup table approximation method can improve control performance, but it is difficult to debug and tune. Moreover, in complex systems where multiple inputs exist, the lookup table approximation method may be very costly to implement because large memories are required to store a lookup table.

In many applications, the fuzzy control, based on the fuzzy logic, provides better control performance than linear, piecewise linear, or lookup table approximation methods because it provides an efficient framework to incorporate linguistic fuzzy information from human experts. The so-called intelligent control has emerged to use the expert information in the control community. Some tools for implementing the intelligent control can be referred to artificial neural network, genetic algorithm, and fuzzy logic. Note that the artificial neural network is a learning-based device whose design is motivated by the function of human brains and components thereof. And, the genetic algorithm is inspired by Charles Robert Darwin’s theory of evolution and works by creating many stochastic selection parameters to a problem. Usually, the expert information is represented by fuzzy terms like small, large, fast, and so on. Therefore, the fuzzy control is more adequate to implement the intelligent control with the expert information than the artificial neural network and the genetic algorithm.

Fuzzy controllers can perform the nonlinear control actions because fuzzy logic systems are capable of uniformly approximating any nonlinear function over a compact set to any degree of accuracy [5]. Thus, if the parameters of a fuzzy controller are carefully chosen, it is possible to design a fuzzy controller for nonlinear dynamic systems. Then, it is well known that fuzzy controllers are robust with respect to disturbances of systems because their operations are determined by fuzzy rules. Also, fuzzy controllers are customizable because it is easy to understand and modify their rules.

As the behaviors of dynamic systems become complex, the need of fuzzy scheme increases, and the linguistic analysis suggested by Zadeh [6] allows us to analyze the qualitative behaviors of systems with the fuzzy algorithms. As motivated by the study of [6], Mamdani and Assilian [7] proposed the configuration of a fuzzy system with fuzzifier and defuzzifier, and they applied the fuzzy logic to the control of a dynamic plant, and the fuzzy control has attracted a great deal of interest among researchers. Note that fuzzy system is a name for the system which has a direct relationship with fuzzy concepts (e.g., fuzzy sets and linguistic variables) and fuzzy logic [5]. Also, note that the function of fuzzifier is to map crisp points to fuzzy sets, and the function of defuzzifier is to map fuzzy sets to crisp points [5].

Subsequently, many successful applications of the fuzzy control have increased the need of theoretical analysis concerning the stability and performance of fuzzy control systems. Most of all, the stability of fuzzy control systems has often been required to be verified with theoretical arguments, and there have been several significant studies for designing the stabilizing controllers for fuzzy systems with rigorous stability proofs [8, 9, 10, 11, 12, 13, 14], in which the so-called TS-type fuzzy model proposed by Takagi and Sugeno [15] has mainly been used to represent fuzzy systems. Specifically, in [8, 9, 10, 11, 12, 13, 14], the authors made TS-type fuzzy models of dynamic systems with the IF-THEN fuzzy implication and fuzzy inference and designed the stabilizing control laws for TS-type fuzzy models. Then they applied the stabilizing control laws for TS-type fuzzy models to dynamic systems. In the fuzzy control design, the knowledge of an expert can be applied to the control design for dynamic systems by a linguistic expression such as the IF-THEN fuzzy implication.

Concerning the performance of fuzzy control systems, the optimality has often been considered as an important issue in the design of fuzzy control systems, and the conventional linear optimal control method [16] has been used to design the optimal control law for TS-type fuzzy systems. On the optimality issue of fuzzy control systems, Wang [17] developed the optimal fuzzy controller for linear time-invariant systems by utilizing the Pontryagin minimum principal. However, the design method of [17] does not have much practical implications because it may not be a good choice to use the fuzzy controller designed for linear systems directly as the controller for nonlinear fuzzy systems. Based on the linear quadratic optimal control theory [16], Wu and Lin [18] presented a design method of the optimal controllers for both continuous- and discrete-time fuzzy systems. The main strategy of [18] is to seek the optimal controller that minimizes a given performance index by solving the matrix Riccati differential equations or the steady-state algebraic Riccati equations. Later, Wu and Lin [19] addressed a quadratic optimal control problem for continuous-time fuzzy systems, which were represented by the so-called linear-like synthetic matrix form and developed a design scheme of the optimal fuzzy controller under finite or infinite horizon by utilizing the calculus-of-variation method. The study of [19] is also based on solving a steady-state algebraic Riccati-like equation, but it utilizes an efficient algorithm to design the global optimal fuzzy controller. Park et al. [20] addressed the optimal control problem for continuous-time TS-type fuzzy systems. However, the design method of [20] has less redundancy in choice of feedback gains and requires undesirable high feedback gains, which are the main drawbacks of [20] in the design of the optimal controller for fuzzy systems. Kim and Rhee [21] presented a response surface methodology, and they applied this methodology to the design of an optimal fuzzy controller for a plant. Wu and Lin [22] proposed a way to design a global optimal discrete-time fuzzy controller to control and stabilize a nonlinear discrete-time TS-type fuzzy system with finite or infinite horizon time. Chen and Liu [23] studied the problem of guaranteed cost control for TS-type fuzzy systems with a time-varying delayed state. Mirzaei et al. [24] proposed an optimized fuzzy controller for antilock braking systems to improve vehicle control during sudden braking. Lin, Wang, and Lee [25] investigated a geometric property of time-optimal control problem in the TS-type fuzzy model via Lie algebra and found the time-optimal controller as the bang-bang type with a finite number of switching by applying the maximum principle. Mostefai et al. [26] presented a fuzzy observer-based optimal control design for the compensation of nonlinear friction in a robot joint structure based on a fuzzy local modeling technique. Zhu [27] studied a fuzzy optimal control problem for a multistage fuzzy system to optimize the expected value of a fuzzy objective function subject to a multistage fuzzy system. Esfahani and Sichani [28] studied the problem of optimal fuzzy H_{∞}-tracking control design for nonlinear systems that are represented by using the TS fuzzy modeling scheme. Through these studies, the optimal control for fuzzy systems has quite been progressed.

Since TS-type fuzzy systems essentially have a nonlinear nature due to the IF-THEN fuzzy implication and fuzzy inference, we see that a nonlinear optimal control method is suitable for designing an optimal control law for TS-type fuzzy systems. In addition, when we design an optimal control law for TS-type fuzzy systems, it is often required that the control design should allow us to control the convergence rates of state trajectories to an equilibrium point. The decay rate of the closed-loop dynamics may be used to achieve this requirement in the control design. These observations motivate the author to study an optimal control of TS-type fuzzy systems that can provide good convergence rates of state trajectories to an equilibrium point in this chapter.

More specifically, in this chapter, the author presents a theoretical result on the optimal control of nonlinear dynamic systems. In this theoretical result, the author presents the optimal control problem for nonlinear dynamic systems and solves this problem by utilizing the dynamic programming approach [29] and the inverse optimal approach [30]. Note that Kalman [31] first proposed the inverse optimal approach to establish some gain and phase margins of a linear quadratic regulator. Also, note that the conventional direct optimal approach is based on seeking a stabilizing controller that minimizes a given performance index. On the other hand, the inverse optimal approach avoids the task of solving the Hamilton-Jacobi- Bellman (H-J-B) equation numerically but finds a stabilizing controller first and then shows its optimality with respect to a posteriorly determined performance index. In this chapter, the author employs the dynamic programming approach to derive the H-J-B equation associated with the optimal control problem for nonlinear dynamic systems. Then, the author presents an analytic way to solve the H-J-B equation with the help of the inverse optimal approach, by which the author establishes a systematic approach for designing the optimal controller for nonlinear dynamic systems. The resulting optimal controller takes the form of state feedback

Then, based on the theoretical result presented in this chapter, the author establishes an optimal control design for TS-type fuzzy systems that guarantees the global asymptotic stability of an equilibrium point and the optimality with respect to a cost function, which incorporates a penalty on the state and control input vectors, and provides good convergence rates of state trajectories to an equilibrium point. The problem appearing in this optimal control design for TS-type fuzzy systems is given as a linear matrix inequality (LMI)-based problem. From the results, the optimal controller can be found by a simple controller design procedure, which is essentially given as LMIs. The control design involving LMIs is particularly useful in practice because LMIs can be efficiently solved by recently developed interior-point methods (e.g., [32, 33]). One of the algorithms belonging to the interior-point methods can be found in [34], and an implementation of the algorithm in [34] is included in the LMI Control Toolbox of MATLAB [35], which will be used as the solver for the LMI-based problem appearing in the optimal control design.

Note that the optimal controller for a nonlinear dynamic system is in general designed for a linearized design-model, which is obtained by conventional linear approximation techniques, because it is sometimes difficult to solve the nonlinear optimal control problem associated with a nonlinear dynamic system. Clearly, the optimal controller designed for a linearized design-model guarantees its optimality only at an equilibrium point used to design a linearized design-model. Compared with a design-model obtained by conventional linear approximation techniques, the TS-type fuzzy model can be seen as a good design-model for approximating a nonlinear dynamic system because it retains the essential features of a nonlinear dynamic system with a linguistic description in terms of fuzzy IF-THEN rules, by which the TS-type fuzzy system is valid over a range of operating points within fuzzy sets. Thus, when we design the optimal controller for a nonlinear dynamic system, it is adequate to approximate a nonlinear dynamic system with a TS-type fuzzy model and to use a TS-type fuzzy model rather than a linearized design-model. In addition, one can expect that the optimal controller designed for a TS-type fuzzy system has a wider range of optimality than the optimal controller designed for a linearized design-model because the former guarantees the optimality over a range of operating points within fuzzy sets.

As a control design example in this chapter, the three-axis attitude stabilization problem of a rigid body is considered to illustrate the optimal control design method for TS-type fuzzy systems presented in this chapter. The attitude motion of a rigid body is basically represented by a set of two Equations [36]: (i) Euler’s dynamic equation, which describes the time derivative of the angular velocity vector and (ii) the kinematic equation, which relates the time derivatives of the orientation angles to the angular velocity vector. For representing the orientation angles of a rigid body, there exist several kinematic parameterizations such as Euler angles, Gibbs vector, Cayley-Rodrigues parameters, and modified Rodrigues parameters [37, 38], which are singular three-dimensional parameter representations, and quaternion (also called Euler parameters), which is a nonsingular four-dimensional parameter representation. Note that three-dimensional parameter representations exhibit singular orientations because the Jacobian matrix is singular for some orientations. On the other hand, the quaternion consists of four parameters subject to the unit length constraint and is a globally nonsingular parameter for describing the body orientation [36].

In this chapter, the equations of motion of a rigid body including dynamics and kinematics are considered. The kinematic equation of a rigid body considered in this chapter is described by the quaternion. The equations of motion of a rigid body considered in this chapter describe a system in cascade interconnection, and the backstepping method of [39] can be efficiently utilized to apply the optimal control design method presented in this chapter to the three-axis attitude stabilization problem of a rigid body.

The optimal attitude stabilization problem of a rigid body has been addressed by several researchers [40, 41, 42]. Also, there have been many studies which consider performance indices such as time and/or fuel in the formulation of the optimal attitude stabilization problem of a rigid body [43, 44, 45, 46, 47, 48, 49], in which the optimal regulation problems for angular velocity subsystem of a rigid body and for some quadratic performance indexes have mainly been addressed.

The optimal attitude control problem of the complete attitude motion of a rigid body, which includes dynamics as well as kinematics, has been investigated by many researchers: Carrington and Junkins [50] used a polynomial expansion approach to approximate the solution of H-J-B equation. Rotea et al. [51] showed that Lyapunov functions including a logarithmic term in the kinematic parameters result in linear controllers with a finite quadratic performance index. For the general quadratic performance index, they also presented sufficient conditions which guarantee the existence of a linear and suboptimal stabilizing controller. Tsiotras [52] derived a new class of globally asymptotically stabilizing feedback control laws as well as a family of exponentially stabilizing optimal control laws for the complete attitude motion of a nonsymmetric rigid body. Later, Tsiotras [53] presented a partial solution to the optimal regulation problem of a spinning rigid body by using the natural decomposition of the complete attitude motion into its kinematics and dynamics systems and the inherent passivity properties of these two systems. Bharadwaj et al. [54] presented a couple of new globally stabilizing attitude control laws based on minimal and exponential coordinates. Park and Tahk [55] have considered the problem of three-axis robust attitude stabilization of a rigid body with inertia uncertainties, and they have presented a class of new robust attitude control laws having relaxed feedback gain structures. Later, Park and Tahk [56] have extended their robust attitude control scheme of [55] to the optimal attitude control scheme by using the Hamilton-Jacobi theory of [57]. Also, Park et al. [58] have first addressed a game-theoretic approach to robust and optimal attitude stabilization of a rigid body with external disturbances.

Note that, in the case of robot arm control, since the arms or hand fingers can be viewed as actuators which maneuver the attitude of the held object, the results on the attitude control of a rigid body can be applied to the attitude control of a rigid payload held by the robot arm [59]. With this relation, there have been many studies concerning the attitude control problem of a rigid body, and some remarkable studies can be referred to [60, 61].

The rest of this chapter is composed as follows. In Section 2, the author presents a theoretical result on the optimal control of nonlinear dynamic systems. In Section 3, the author introduces TS-type fuzzy systems and presents an optimal control design for TS-type fuzzy systems. In Section 4, the author considers the three-axis attitude stabilization problem of a rigid body as a control design example and illustrates the effectiveness of the optimal control design for TS-type fuzzy systems. In Section 5, the author concludes this chapter with concluding remarks.

## 2. Nonlinear optimal control

Consider the nonlinear dynamic system given by

where

In general, we can find the optimal control law for the nonlinear dynamic system in (1) by numerically solving the corresponding Hamilton-Jacobi-Bellman (H-J-B) equation. However, this is a difficult task, and, thus, we may need a simple and efficient method to find optimal control law for the nonlinear dynamic system in (1). In the following, the author presents a theory to provide the optimal control law for the nonlinear dynamic system in (1) by circumventing the task of numerically solving the H-J-B equation.

*Proposition 1* [62]: For the nonlinear dynamic system in (1), suppose that there exists a radially unbounded and positive definite function

for all

is the optimal, globally asymptotically stabilizing control law for the system in (1) that minimizes the cost function

where

for all

*Proof:* First, the following condition holds by (2):

for all

Next, define

Substituting

which implies that

In proposition 1, we see that the globally asymptotically stabilizing control law in (3) for the nonlinear dynamic system in (1) can be found without the task of numerically solving the H-J-B equation in (7) and the control law in (3) is optimal with respect to the cost function in (4). The key point of this work is that we posteriorly determine the penalty on the state vector, which is

It is remarkable that as shown in (4, 5), we can adjust the penalty on the control input vector, which is

As shown in (3), since the constant

Now, if we consider a practical application, we know that any control law for dynamic systems must provide good convergence rates of state trajectories to an equilibrium point. For achieving this requirement in the design of a control law, the decay rate can be used as a design factor to dominate convergence rates of state trajectories to an equilibrium point. Note that the decay rate of a system is defined to be the largest constant

Proposition 2 [32]: If there exist a positive definite function

for all trajectories of a system, then the decay rate of a system is at least

*Proof:* If there exist a positive definite function

where

## 3. Optimal control of fuzzy systems

### 3.1. Fuzzy systems

The TS-type fuzzy model and the TS-type fuzzy control law for a system are given by the following IF-THEN fuzzy implications, respectively [15]:

*Plant rule i for a system:*

IF

*Control law rule i for a system:*

IF

In (10, 11),

If we follow the usual fuzzy inference method, we can represent the state equations of the TS-type fuzzy model and the TS-type fuzzy control law for a system as follows, respectively [9]:

and

where

and

### 3.2. Problem definition

The author defines two kinds of problems considered in this section. The first problem is to design a control law for the TS-type fuzzy system in (12) that achieves the global asymptotic stability of the equilibrium point

where

### 3.3. Optimal control design

We can use the results of propositions 1 and 2 to solve the problems defined in Section 3.2. Specifically, we define

Theorem 1 [62]: For the TS-type fuzzy system in (12), suppose that there exists a function

and

where

where

where

for all

*Proof:* Suppose that

where

Now, from propositions 1 and 2, assume that there exists a positive definite matrix

for all

for all

and

respectively. If all sets of inequalities in (24, 25) are pre- and post-multiplied by a positive definite matrix

Now, consider the TS-type fuzzy system with a common input matrix, which is described by

Then, the author presents the following theory:

Theorem 2 [62]: For the TS-type fuzzy system in (26), suppose that there exists a function

and

where

where

where

for all

*Proof:* Suppose that

where

Now, from propositions 1 and 2, assume that there exists a positive definite matrix

for all

for all

and

respectively. If all sets of inequalities in (36, 37) are pre- and post-multiplied by a positive definite matrix

Note that the problem appearing in Theorems 1 and 2 is to find a matrix

## 4. A control design example

As a control design example, the author considers the three-axis attitude stabilization problem of a rigid body and illustrates the effectiveness of the optimal control design for TS-type fuzzy systems presented in Section 3.

### 4.1. Rigid body model

First, the dynamic equation of the rotational motion of a rigid body is described as follows [36]:

where

and has the property of

Second, the kinematic equation of rotational motion of a rigid body described in terms of the quaternion is given as follows [36]:

where

where

With the notations of the Euler axis

### 4.2. Optimal control design

First, it is observed that two state equations given by (38) and (41) represent a system in cascade interconnection. That is, the angular velocity vector indirectly controls the kinematics system in (41). Thus, the angular velocity vector can be regarded as a virtual control input of the kinematics system in (41) to stabilize the kinematics system in (41). This observation gives the following theorem: Theorem 3 [62]: Consider the kinematics system in (41) with

where

*Proof:* With the control law

Now, consider the Lyapunov function candidate

for all

Next, we have to stabilize the dynamics system in (38) with making

where

As a numerical example, the author assumes that

where diag means the diagonal matrix,

*Plant rule i for the system in*(45):

IF

In (49),

the TS-type fuzzy model in (49) can be transformed into the following TS-type fuzzy system:

We can use the result of theorem 2 to design the optimal control law for the TS-type fuzzy system in (50) because the TS-type fuzzy system in (50) has the common input matrix B in (47). Then, we assume that

where

Since we can analyze the pure effect of the weight matrix

where

The author designs the other by solving (27) and (28) for the TS-type fuzzy system in (50) with

where

### 4.3. Numerical simulation results

As the numerical simulation model, the author uses the equations of rotational motion of a rigid body given by (38) and (41), where the inertia matrix is given by (48). The author assumes that the initial conditions at the initial time

With the optimal control laws

First, as shown in Figures 3–5, we see that

Finally, in Figures 3 and 4, we see that the state responses of a rigid body with

## 5. Conclusion

In this chapter, the author presented a theory on the optimal control of nonlinear dynamic systems by utilizing the dynamic programming approach and the inverse optimal approach. Specifically, the author employed the dynamic programming approach to derive the Hamilton-Jacobi-Bellman (H-J-B) equation associated with the optimal control problem for nonlinear dynamic systems and utilized the inverse optimal approach to avoid the task of solving the H-J-B equation numerically.

Then, the author established an optimal control design for TS-type fuzzy systems to achieve the global asymptotic stability of an equilibrium point, the optimality with respect to a cost function, and the good convergence rates of state trajectories to an equilibrium point. Based on this optimal control design, the author presented a systematic way for designing the optimal control law for TS-type fuzzy systems.

The author showed the usefulness of the optimal control design by considering the three-axis attitude stabilization problem of a rigid body. The optimal three-axis attitude stabilizing control law for a rigid body was designed, and its control performance was analyzed by numerical simulations. The numerical simulation results demonstrated that the optimal three-axis attitude stabilizing control law designed in this chapter provides desirable optimal control performance together with good convergence rates of state trajectories to an equilibrium point.

The author would like to suggest two further research topics: one is an extension of the study presented in this chapter toward the robust control design for TS-type fuzzy systems with system parametric uncertainties and external disturbances. The problem is to incorporate these robustness issues into the optimal control design approach, and the key to solve this problem may be found in many literatures addressing the robust control approaches such as loop-transfer-recovery approach, guaranteed-cost approach, stochastic approach, and state-estimation approach. However, the difficulty of combining these approaches with the optimal control design method comes from the fact that these approaches are mainly for linear systems. Therefore, the extension of these approaches to nonlinear dynamic systems may be requisite. The other is to develop the optimal control design of TS-type fuzzy systems for tracking problems. Tracking problems assume that the equilibrium point is not the zero state. In linear dynamic systems, tracking problems can be reduced to regulation problems under the assumption that the desired state is known or can be reduced to disturbance-rejection problems under the assumption that the disturbance signal is known. However, in many cases, the desired state or the disturbance signal is not known. Thus, in such cases, some alternatives such as minimax approach and proportional-integral control may be needed. The minimax approach is for the worst-case design such that the disturbance signal maximizes the same performance index that the control input minimizes. And the proportional-integral control can be used to reject the constant disturbances. In the sequel, the minimax approach and the proportional-integral control may provide the solution to the tracking problem of TS-type fuzzy systems.