Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems

The design goal of a control system is to influence the behavior of dynamic systems to achieve some pre-determinate objectives. A control system is usually designed on the premise that an accurate knowledge of a given object and environment cannot be obtained in advance. It usually requires suitable methods to address the problems related to uncertain and highly complicated dynamic system identification. As a matter of fact, system identification is an important branch of research in the automatic control domain. However, the majority of methods for system identification and parameters' adjustment are based on linear analysis: therefore it is difficult to extend them to complex non-linear systems. Normally, a large amount of approximations and simplifications have to be performed and, unavoidably, they have a negative impact on the desired accuracy. Fortunately the characteristics of the Artificial Neural Network (ANN) approach, namely non-linear transformation and support to highly parallel operation, provide effective techniques for system identification and control, especially for non-linear systems [1-9]. The ANN approach has a high potential for identification and control applications mainly because: (1) it can approximate the nonlinear input-output mapping of a dynamic system [10]; (2) it enables to model the complex systems’ behavior and to achieve an accurate control through training, without a priori information about the structures or parameters of systems. Due to these characteristics, there has been a growing interest, in recent years, in the application of neural networks to dynamic system identification and control.


Introduction
The design goal of a control system is to influence the behavior of dynamic systems to achieve some pre-determinate objectives.A control system is usually designed on the premise that an accurate knowledge of a given object and environment cannot be obtained in advance.It usually requires suitable methods to address the problems related to uncertain and highly complicated dynamic system identification.As a matter of fact, system identification is an important branch of research in the automatic control domain.However, the majority of methods for system identification and parameters' adjustment are based on linear analysis: therefore it is difficult to extend them to complex non-linear systems.Normally, a large amount of approximations and simplifications have to be performed and, unavoidably, they have a negative impact on the desired accuracy.Fortunately the characteristics of the Artificial Neural Network (ANN) approach, namely non-linear transformation and support to highly parallel operation, provide effective techniques for system identification and control, especially for non-linear systems [1][2][3][4][5][6][7][8][9].The ANN approach has a high potential for identification and control applications mainly because: (1) it can approximate the nonlinear input-output mapping of a dynamic system [10]; (2) it enables to model the complex systems' behavior and to achieve an accurate control through training, without a priori information about the structures or parameters of systems.Due to these characteristics, there has been a growing interest, in recent years, in the application of neural networks to dynamic system identification and control.
"Depth" and "resolution ratio" are the main characteristics to measure the dynamic memory performance of neural networks [11]."Depth" denotes how far information can be memorized; "resolution ratio" denotes how much information in input sequences of neural networks can be retained.The memory of time-delay units is of lower depth and higher resolution ratio, while most recurrent neural networks, such as Elman neural networks, are higher depth and lower resolution ratio.The popular neural networks have much defect on dynamic memory performance.This chapter proposed a novel time-delay recurrent network model which has far more "depth" and "resolution ratio" in memory for identifying and controlling dynamic systems.The proposed identification and control schemes are examined by the numerical experiments for identifying and controlling some typical nonlinear systems.
The rest of this chapter is organized as follows.Section 2 proposes a novel time-delay recurrent neural network (TDRNN) by introducing the time-delay and recurrent mechanism; moreover, a dynamic recurrent back propagation algorithm is developed according to the gradient descent method.Section 3 derives the optimal adaptive learning rates to guarantee the global convergence in the sense of discrete-type Lyapunov stability.Thereafter, the proposed identification and control schemes based on TDRNN models are examined by numerical experiments in Section 4. Finally, some conclusions are made in Section 5.  Z  denotes a one-step time delay, the notation "" represents the memory neurons in the input layer with selffeedback gain  (0 1)    , which improves the resolution ratio of the inputs.

Time-delay recurrent neural network (TDRNN)
It is a type of recurrent neural network with different layers of neurons, namely: input nodes, hidden nodes, output nodes and, specific of the approach, context nodes.The input and output nodes interact with the outside environment, whereas the hidden and context nodes do not.The context nodes are used only to memorize previous activations of the output nodes.The feed-forward connections are modifiable, whereas the recurrent connections are fixed.More specifically, the proposed TDRNN possesses self-feedback links i zk uk uk i zk where 0, ,1 , Taking expansion for (1 ) zk , (2 ) zk ,…, (1) z by using Eq.( 5), then we have 01 From Eq.( 8) it can be seen that the memory neurons in the input layer include all the previous input information and the context nodes memorize previous activations of the output nodes, so the proposed TDRNN model has far higher memory depth than the popular neural networks.Furthermore, the neurons in the input layer can memory accurately the inputs from time k   to time k , and this is quite different from the memory performance of popular recurrent neural networks.If the delay step  is moderate large, the TDRNN possesses higher memory resolution ratio.
Let the kth desired output of the system be () d y k .We can then define the error as Differentiating E with respect to W respectively, according to the gradient descent method, we obtain the following equations 30 3, ( ) ( 1,2, , ; 1,2, , ) which form the learning algorithm for the proposed TDRNN, where If () gx is taken as a linear function, then () 1 i g   .Clearly, Eqs. ( 11) and ( 14) possess recurrent characteristics.

Convergence of proposed time-delay recurrent neural network
In Section 2, we have proposed a TDRNN model and derived its dynamic recurrent back propagation algorithm according to the gradient descent method.But the learning rates in the update rules have a direct effect on the stability of dynamic systems.More specifically, a large learning rate can make the modification of weights over large in each update step, and this will induce non-stability and non-convergence.On the other hand, a small learning rate will induce a lower learning efficiency.In order to train neural networks more efficiently, we propose three criterions of selecting proper learning rates for the dynamic recurrent back propagation algorithm based on the discrete-type Lyapunov stability analysis.The following theorems give sufficient conditions for the convergence of the proposed TDRNN when the functions () f  and () g  in Eqs. ( 4) and ( 2) are taken as sigmoidal function and linear function respectively.Suppose that the modification of the weights of the TDRNN is determined by Eqs.(10)(11)(12)(13)(14).For the convergence of the TDRNN we have the following theorems.
Proof.Define the Lyapunov energy function as follows.
Where , () () () And consequently, we can obtain the modification of the Lyapunov energy function Then the error during the learning process can be expressed as Furthermore, the modification of weights associated with the input and hidden layers is Hence, from Eqs.(18-20) we obtain 1 W represents an nr  dimensional vector and  denotes the Euclidean norm.
Notice that the activation function of the hidden neurons in the TDRNN is the sigmoidal type, we have 0( ) 1 / 4 fx  .Thus, 1 , 2 ,, ; 1 , 2 ,, ) According to the definition of the Euclidean norm we have Proof.Similarly, the error during the learning process can be expressed as Notice that the activation function of the hidden neurons in the TDRNN is the sigmoidal type, and neglect the dependence relation between () y k and the weights 2 ij w , we obtain According to the definition of the Euclidean norm we have 2 () Therefore, while Proof.Similarly, as the above proof, we have www.intechopen.com

Numerical results and discussion
The performance of the proposed time-delay recurrent neural network for identifying and controlling dynamic systems is examined by some typical test problems.We provide four examples to illustrate the effectiveness of the proposed model and algorithm.

Nonlinear time-varying system identification
We have carried out the identification for the following nonlinear time-varying system using the TDRNN model as an identifier.
Where () vk is Gauss white noise with zero mean and constant variance 0.1.The input of system is taken as ( ) sin(0.01 ) uk k   .
To evaluate the performance of the proposed algorithm, the numerical results are compared with those obtained by using Elman neural network (ENN).The Elman network is a typical www.intechopen.com Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems 123 recurrent network proposed by Elman [12].Some parameters on the TDRNN in our experiments are taken as follows.The number of hidden nodes is taken as 6, the weights are initialized in the interval [-2, 2] randomly, besides,  ,  and  are set as 0.4, 0.6, 0.4 respectively.The number of hidden nodes in the ENN is also taken as 6.
Figure 2 shows the identification result, where the "Actual curve" is the real output curve of the dynamic system, represented by the solid line; the "Elman curve" is the output curve identified using the ENN model, and represented by the dash line; the "TDRNN curve" is the output curve identified by the proposed TDRNN model, and represented by the dash dot line.Figure 3 shows the identification error curves obtained with the TDRNN and ENN respectively, in which the error is the absolute value of the difference between identification result and the actual output.From the two figures it can be seen that the proposed method is superior to the ENN method.These results demonstrate the power and potential of the proposed TDRNN model for identifying nonlinear systems.

Bilinear DGP system control
In this section, we control the following bilinear DGP system using the TDRNN model as a controller.
( ) 0 .5 0 .4 (1 ) 0 .4 (1 ) (1 ) ( ) The system output at an arbitrary time is influenced by all the past information.The control reference curves are respectively taken as: 1. Line type: The parameters on the TDRNN in the experiments are taken as follows.The number of hidden nodes is taken as 6, the weights are initialized in the interval [-2, 2] randomly, besides,  ,  and  are set as 0.3, 0.6, 0.4 respectively.Figures 4 and 5 show the control results.Figure 4 shows the control curve using the proposed TDRNN model when the control reference is taken as a line type.Figure 5 shows the control curve when the reference is taken as a quadrate wave type.From these results it can be seen that the proposed control model and algorithm possess a satisfactory control precision.

Inverted pendulum control
The inverted pendulum system is one of the classical examples used in many experiments dealing with classical as well as modern control, and it is often used to test the effectiveness of different controlling schemes [13][14][15][16].So in this chapter, to examine the effectiveness of the proposed TDRNN model, we investigate the application of the TDRNN to the control of inverted pendulums.
The inverted pendulum system used here is shown in Fig. 6, which is formed from a cart, a pendulum and a rail for defining position of cart.The Pendulum is hinged on the center of the top surface of the cart and can rotate around the pivot in the same vertical plane with the rail.The cart can move right or left on the rail freely. x The dynamic equation of the inverted pendulum system can be expressed as the following two nonlinear differential equations.
(s i n c o s ) s gn( ) Where the parameters, M and m are respectively the mass of the cart and the mass of the pendulum in unit (kg), 2 9.81 gm s  is the gravity acceleration, l is the half length of the pendulum in unit (m), F is the control force in the unit (N) applied horizontally to the cart, c u is the friction coefficient between the cart and the rail, p u is the friction coefficient between the pendulum pole and the cart.The variables  ,   ,   represent the angle between the pendulum and upright position, the angular velocity and the angular acceleration of the pendulum, respectively.Moreover, given that clockwise direction is positive.The variables x , x  , x  denote the displacement of the cart from the rail origin, its velocity, its acceleration, and right direction is positive.
We use the variables  and x to control inverted pendulum system.The control goal is to make  approach to zero by adjusting F, with the constraint condition that x is in a given interval.The control block diagram of the inverted pendulum system is shown in Figure 7.
The TDRNN controller adopts variables  and x as two input items.
Inverted pendulum x Fig. 7. Control block diagram of inverted pendulum system In the numerical experiments, the motion of the inverted pendulum system is simulated by numerical integral.The parameter setting is listed in the Table 1.Besides, the number of hidden nodes is taken as 6, the weights are initialized in the interval [-3, 3] randomly, the parameters  ,  and  on the TDRNN are set as 0.3, 0.6, 0.4 respectively.The control goals are to control the absolute value of  within 10º and make it approximate to zero as closely as possible, with the constraint condition of the absolute value of x within 3.0m.The control results are shown in Figures 8 and 9, and the sampling interval is taken as 1 Tm s 

012345
. Figures 8 and 9 respectively show the control curve of the angle  and the control curve of the displacement x .From Figure 8, it can be seen that the fluctuation degree of  is large at the initial stage, as time goes on, the fluctuation degree becomes smaller and smaller, and it almost reduces to zero after 3 seconds.Figure 9 shows that the change trend of x is similar to that of  except that it has a small slope.These results demonstrate the proposed control scheme based on the TDRNN can effectively perform the control for inverted pendulum system.

Ultrasonic motor control
In this section, a dynamic system of the ultrasonic motor (USM) is considered as an example of a highly nonlinear system.The simulation and control of the USM are important problems in the applications of the USM.According to the conventional control theory, an accurate mathematical model should be set up.But the USM has strongly nonlinear speed characteristics that vary with the driving conditions and its operational characteristics depend on many factors.Therefore, it is difficult to perform effective control to the USM using traditional methods based on mathematical models of systems.Our numerical experiments are performed using the model of TDRNN for the speed control of a longitudinal oscillation USM [17] shown in Figure 10.

Vibratory piece
Fig. 10.Schematic diagram of the motor Some parameters on the USM model are taken as: driving frequency 27.8 kHZ , amplitude of driving voltage 300 V , allowed output moment 2.5 kg cm  , rotation speed 3.8 / ms .Besides, the number of hidden nodes of the TDRNN is taken as 5, the weights are initialized in the interval [-3, 3] randomly, the parameters  ,  and  on the TDRNN are taken as 0.4, 0.6, 0.4 respectively.The input of the TDRNN is the system control error in the last time, and the output of the TDRNN, namely the control parameter of the USM is taken as the frequency of the driving voltage.
Figure 11 shows the speed control curves of the USM using the three different control strategies when the control speed is taken as 3.6 / ms .In the figure the dotted line a represents the speed control curve based on the method presented by Senjyu et al. [18], the solid line b represents the speed control curve using the method presented by Shi et al. [19] and the solid line c represents the speed curve using the method proposed in this paper.Simulation results show the stable speed control curves and the fluctuation amplitudes obtained by using the three methods.The fluctuation degree is defined as where max min , VV and ave V represent the maximum, minimum and average values of the speeds.From Figure 11 it can be seen that the fluctuation degrees when using the methods proposed by Senjyu and Shi are 5.7% and 1.9% respectively, whereas, it is just 1.1% when using the method in this paper.Figure 12 shows the speed control curves of the reference speeds vary as step types.From the figures it can be seen that this method possesses good control precision.

Conclusions
This chapter proposes a time-delay recurrent neural network (TDRNN) with better performance in memory than popular neural networks by employing the time-delay and recurrent mechanism.Subsequently, the dynamic recurrent back propagation algorithm for the TDRNN is developed according to the gradient descent method.Furthermore, to train neural networks more efficiently, we propose three criterions of selecting proper learning rates for the dynamic recurrent back propagation algorithm based on the discrete-type Lyapunov stability analysis.Besides, based on the TDRNN model, we have described, analyzed and discussed an identifier and an adaptive controller designed to identify and www.intechopen.comRecurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems 131 control nonlinear systems.Our numerical experiments show that the TDRNN has good effectiveness in the identification and control for nonlinear systems.It indicates that the methods described in this chapter can provide effective approaches for nonlinear dynamic systems identification and control.

Figure 1
Figure 1 depicts the proposed time-delay recurrent neural network (TDRNN) by introducing the time-delay and recurrent mechanism.In the figure,

Theorem 3 . 3 W
Lyapunov stability theory, this shows that the training error will converges to zero as t .This completes the proof.The stable convergence of the update rule(10) on is guaranteed if the learning rate 3 () k to the Lyapunov stability theory, this shows that the training error will converges to zero as t .This completes the proof.

Fig. 8 .Fig. 9 .
Fig. 8.Control curve of the angle  obtained by Senjyu b --obtained by Shi c --obtained in this paper

Fig. 11 .Fig. 12 .
Fig. 11.Comparison of speed control curves using different schemes Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems 117 with fixed coefficient  in the context nodes.Thus the output of the context nodes can be www.intechopen.com and  is the step number of time delay.()

Theorem 1 .
The stable convergence of the update rule (12) on )

Table 1 .
Parameter Setting of Inverted Pendulum