Tuning Artificial Neural Network Controller Using Particle Swarm Optimization Technique for Nonlinear System

Abstract

This chapter proposes an optimization technique of Artificial Neural Network (ANN) controller, of single-input single-output time-varying discrete nonlinear system. A bio-inspired optimization technique, Particle Swarm Optimization (PSO), is proposed to be applied in ANN to avoid any possibilities from local extreme condition. Further, a PSO based neural network controller is also developed to be integrated with the designed system to control a nonlinear systems. The simulation results of an example of nonlinear system demonstrate the effectiveness of the proposed approach using Particle Swarm Optimization approach in terms of reduced oscillations compared to classical neural network optimization method. MATLAB was used as simulation tool.

1. Introduction

We are interested, in this chapter, in adaptive system control of a class of single-input single-output (SISO) non-linear systems using neural network. In fact, this system control is a very general approach to adaptive control since one can combine in principle any parameter estimation scheme with any control strategy. In addition, its architecture is based on two neural network blocks corresponding to the system controller and the model identification of the dynamic behavior of the system [1, 2].

The use of artificial neural network (ANN) for identification, diagnosis, modeling and control has generated a lot of interest for quite some time now, because they have proved to be excellent function approximators, mapping any function to an arbitrary degree of accuracy, coupled with their ability for generalization, self-organization and self-learning [3].

Many architectures of neural networks are used. Among them, the most common and the most popular architecture is the multilayered perceptron, implemented with the standardized backpropagation algorithm. If the initial set of weights is not selected properly, this algorithm, employing a gradient descent search technique is seriously prone to getting trapped in local optimum solutions. However, the calculations could slowly occurred and may even overflow or fluctuate between the optima. These limitations encouraged researchers to look for more powerful optimizations techniques that can help reach the optimal solution in an improved fashion, guarantee the convergence of control system and increase the learning speed [3].

A lot of optimization techniques of neural network are widely used. Among them, particle swarm optimization (PSO), the subject of this chapter, studied in different papers like in [3], and originates from the behavioral simulation of fish schooling and bird flocking. The conceptual model, at some point in the evolution of the algorithm, was an optimizer following which a number of parameters extraneous to optimization were eliminated leading to the basic PSO algorithm [3].

This technique is used in many applications of neural network optimization like identification, control and modeling. For instance, in [4], the authors used the PSO based neural network optimization for prediction of diameter errors in a boring machine. In their work, they established an improvement in the quality of optimization of the neural networks and error compensations with the use of the PSO algorithm, which achieves a better machining precision with fewer numbers of iterations.

The PSO algorithm is proposed to get optimal the parameters of ANN. This algorithm is well used because it has convergent result and not require many iterations, so in relative calculation relative quick. PSO is a population-based approach, which uses the swarm intelligence generated by the cooperation and competition between the particles in a swarm. It has been emerged successfully to a wide variety of search and optimization problems.

For example, in [5], the authors compared the performance of the PSO technique with other EAs for both continuous and discrete optimization problems in terms of processing time, convergence speed, and quality of the results. In addition, in [6], the authors proposed a PSO learning algorithm that self-generates radial basis function neural network (RBFN) to deal with three non-linear problems. This proposed PSO allows a high accuracy within a short training time when determining RBFN with small number of radial basis functions. Then, in [7], a PSO algorithm was developed by the authors to find the optimum process parameters which satisfy given limit, tool wear and surface roughness values and maximize the productivity at the same time. Also, in [8], the authors described an evolutionary algorithm for evolving the ANN which was based on the PSO technique. Both the architecture and the weights of the ANN were adaptively adjusted according to the quality of the neural network until the best architecture or a terminating criterion was reached. Moreover, the performance of the basic PSO algorithm with the constriction PSO on some test functions of different dimensions was compared by [9] and they found that the use of constriction PSO with mutation provided significant improvement in certain cases.

Further, in [10], it is presented an improved PSO algorithm for neural network training employing a population diversity method to avoid premature convergence. Furthermore, in [11], the authors used the PSO technique to optimize the grinding process parameters such as wheel and workpiece speed, depth and lead of dressing, etc. subjected to suitable constraints with the objective of minimizing the production cost and obtaining the finest possible surface finish. As well as, by comparing the PSO algorithm results with genetic algorithms and quadratic programming techniques, the PSO algorithm gives the global optimum solution with the objective to obtain minimum cost of manufacturing, [12]. Equally, in [13], the authors applied ANN - PSO approach for selection of optimum process parameters for minimizing burr size in drilling process. Besides that, the PSO algorithm was applied for optimization of multi objective problem in tile manufacturing process [14] and also for machinery fault detection [15]. Finally, in [16] the authors used PSO to tune the radial basis function networks for modeling of MIG welding process.

The PSO is well used in control system, for instance, in [17], the parameters of PID are tuned by ANN where their weights are optimized using PSO method to avoid any local minima/maxima in its searching procedure. In [18], the authors proposed a design of decentralized load-frequency controller for interconnected power systems with ac-dc parallel using PSO algorithm. The experiment result illustrated that their method have rapid dynamic response ability. In [19], the PSO algorithm is implemented to optimize the own five parameters of ${\mathit{PI}}^{\lambda }{D}^{\delta }$controller via El-Khazali’s approach in order to minimize several error functions satisfying some step response specifications such as the set of time domain and frequency domain constraints; overshoot, rise time and settling time. In [20], a comparative analysis of PSO algorithms is carried out, where two PSO algorithms, namely PSO with linearly decreasing inertia weight (LDW-PSO), and PSO algorithm with constriction factor approach (CFA-PSO), are independently tested for different PID structures.

In this chapter, a comparison of the performance of the PSO optimized neural network with the standard back-propagation is presented for the adaptive indirect control of nonlinear time-varying discrete system.

The present chapter is organized as follows. After this introduction, section 2 reviews the problem statement. Furthermore, in section 3 the neural network optimization methods are shown. In Section 4, tuning neural network controller using classical approach is presented. However, the section 5 details tuning neural network controller using PSO approach. An example of nonlinear system is studied, in section 6, to illustrate the proposed efficiency of the method. Section 7 gives the conclusion of this chapter.

2. Problem statement

The indirect adaptive control that is used, in this chapter, is composed of two blocks: a block of neural network model and a block of control system. The proposed control system is a neural network controller. At the simulation, it is assume that the neural network controller parameter’s depending of the model parameter’s as given in Figure 1.

Figure 1.

The architecture of indirect neural control.

In this architecture of indirect control, $r\left(k\right)$is the desired value, $u\left(k\right)$is the control law from the controller, $y\left(k\right)$is the output of the nonlinear system, $\mathit{yr}\left(k\right)$is the output of the neural network model, $e\left(k\right)$is the identification error, ${\widehat{e}}_c\left(k\right)$is the estimated tracking error, $e_c\left(k\right)$is the tracking error and $k$is the discrete time.

The aim of this chapter is to find a control law $u\left(k\right)$to the nonlinear system, given by the Eq. (1), based on the tuning neural network controller’s parameters in order that the system output $y\left(k\right)$tracks, where possible, the desired value $r\left(k\right)$.

$$y\left(k+1\right)=f\left(y\left(k\right),\dots ,y\left(k-n_y\right),u\left(k\right),\dots ,u\left(k-n_u\right)\right)$$E1

$f\left(.\right)$is the nonlinear function mapping specified by the model, $n_y$and $n_u$are the number of past output and input samples respectively required for prediction.

In this structure, the neural network controller and the neural network model must be updated at the same time. But, it is a difficult task to have a neural identifier learn the system fast enough to adapt to varying parameters. Therefore, the neural controller is ineffective on variations of the system parameters. In this chapter, to solve this problem we propose a fast learning algorithm based on a particle swarm optimization approach.

3. Neural network optimization methods

3.1 Gradient back-propagation algorithm

An Artificial Neural Networks (ANN) with randomly initialized weights usually shows poor results. That’s why the most interesting characteristic of a neural network is its ability to learn, in other words, to adjust the weights of its connections according to the training data so that after training phase the faculty of generalization is obtained. This formulation turns the problem of learning into a problem of optimization.

In general, optimizing a system’s parameters for a given task requires defining a metric that captures the inadequacy of the system for that task. This measure is called the cost function. Using an optimization algorithm, it is about finding the optimal parameters of the neural model that minimizes the cost.

For this kind of problem, there are two important classes of cost minimization search algorithms. Classical or gradient algorithms in which the central concept is that of direction of descent, are based on the derivatives of the cost function and any constraints, and advantageously made use of the specific information provided by the derivatives of different orders of these functions.

The alternative to these approaches is the use of meta-heuristics or heuristics such as genetic, stochastic, or evolutionary algorithms. Despite the notable advantage of not assuming regularity and their ability to locate the global minimum, these algorithms are strongly penalized by the relatively low convergence speeds and long computation times.

In the case of algorithms using the information provided by the derivatives of the functions defining the problem, each iteration comprises two main phases: the search for a direction of descent $d_k$and the determination of a step of descent $\eta$given by the following formula:

$$x_{k+1}=x_k+\eta d_k$$E2

The difference between these algorithms is manifested in the way these two steps are performed.

There are also three classes for such algorithms according to the strategy used to calculate the direction of descent:

1. gradient algorithms such as:

   $$d_k=-\nabla f\left(x_k\right)$$E3

2. algorithms based on Newton’s method in which the direction of descent is the solution of the system

   $${\nabla }^{2}f\left(x_k\right)d_k=\nabla f\left(x_k\right)$$E4

3. the algorithms of the quasi-Newton type, in which an approximation $H_k$of the Hessian matrix evaluated in the iterates is built, the direction being then, as for the method of Newton, the solution of the linear system

   $$H_k{d}_k=\nabla f\left(x_k\right)$$E5

The gradient back-propagation algorithm is the most widely used for weight adaptation, the goal of which is to find the appropriate combination of connection weights that minimizes the error function $E$defined by:

$$E=\frac{1}{2}\sum _k{\left(y_k-{\mathit{yr}}_k\right)}^2$$E6

$y_k$and ${\mathit{yr}}_k$being, respectively, the desired output and the actual output of the $k$neuron for a given input vector.

This procedure is based on an extension of the Delta rule which involves a gradient descent and which consists in propagating an observation of the input of the neural network through the neural layer, to obtain the output values.

Compared to the desired outputs, the resulting errors allow the weights of the output neurons to be adjusted. Without the presence of the hidden layer, the knowledge of these errors allows a direct calculation of the gradient and makes the adjustment of the weights of these single neurons, easy as shown by the Delta rule. So for a network with hidden layers, ignoring the desired outputs of the hidden neurons, it thus remains impossible to know the errors of these neurons. So, as it is, this process cannot be used for weight adjustment of hidden neurons. The intuition which solves this difficulty and which gave rise to back-propagation was as follows: the activity of a neuron is linked to neurons of the preceding layer. Thus, the error of an output neuron is due to the hidden neurons of the previous layer in proportion to their influence; therefore according to their activation and the weights that connect the hidden neurons to the output neuron. Therefore, we seek to obtain the contributions of the $L$hidden neurons which gave the error of the output neuron $k$.

The back-propagation procedure consists in propagating the error gradient (error produced during the propagation of an input vector) in the network. In this phase, the propagation of an output neuron’s error starts from the output layer to the hidden neurons.

It is therefore sufficient to retrace the original activation path backwards, starting from the errors of the output neurons, to obtain the error of all the neurons in the network. Once the corresponding error for each neuron is known, the weight adaptation relationships can be obtained.

3.2 Second order optimization method

Another class of methods, more sophisticated than the previous one, is based on second order algorithms, based on Newton’s method which adapts the weights according to the following relation:

$$\mathit{\Delta w}=-H^{-1}\nabla E$$E7

where the element $H_{\mathit{ij}}$of the Hessian matrix $H$relates to the second partial derivatives of the cost function, compared to the weights. The elements of this matrix are defined by

$$H_{\mathit{ij}}=\frac{{\partial }^{2}E}{\partial w_i\partial w_j}$$E8

Like gradient-only methods, second-order methods determine the gradient by the back-propagation algorithm and generally approximate the Hessian matrix or its inverse, since the cost of its computation may quickly become prohibitive.

This type of method localizes, in a single iteration, the minimum of a quadratic empirical error criterion and requires several iterations when this criterion is not ideally quadratic.

In practice, the convergence of the corresponding algorithm towards an optimal solution is rapid so that a good number of error hyper-surfaces present a quadratic curvature in the immediate vicinity of the minima. This method nevertheless remains subject, when the error hyper-surfaces are complex, to convergence towards non-minimal solution points for which the gradient of the empirical error criterion is canceled out at the inflection points or at the saddle points. In addition, there are the possibilities of divergence of the algorithm when the Hessian matrix is not positive definite.

The evaluation and memorization of the inverse Hessian matrix, on which the second-order methods are based, is, however, a major handicap in the context of learning large networks.

But, the main drawback lies in the calculation of the second derivatives of $E$which is most often expensive and very difficult to carry out. A certain number of algorithms propose to get around this difficulty by using approximations of the Hessian matrix.

This approximation, at the basis of Gauss-Newton or Levenberg–Marquardt algorithms, is widely used in the identification of rheological parameters. This method is adapted especially for the problems of small dimensions since the computation of the Hessian matrix is easy. Whereas if the problem presents a large number of variables, it is generally advised to couple it with the conjugate gradient method or a Quasi-Newton method. Or, when the relative improvement in the objective function becomes too low, we automatically switch to the conjugate gradient method.

The Levenberg–Marquardt method, Marquardt method, another second-order method, very close to the Newton method described previously, in fact offers an interesting alternative by adjusting the weights as following:

$$\mathrm{\Delta }{w}_{\mathit{ij}}=-{\left[H+\mathit{\mu I}\right]}^{-1}\nabla E$$E9

$\mu$is the Levenberg–Marquardt parameter and $I$is the identity matrix.

This method, making a compromise between the direction of the gradient and Newton’s method, has the particularity of adapting to the shape of the error surface. Indeed, for low values of $\mu$, the Levenberg–Marquardt method approaches Newton’s method, and for large values of $\mu$, the algorithm is quite simply a function of the gradient, knowing that the parameter $\mu$is automatically updated based on the convergence of each iteration.

Stabilization is possible thanks to a reiterative process, i.e. if an iteration diverges, it can be started again by increasing the parameter $\mu$until a convergent iteration is obtained. However, the phenomenon of strong divergence when approaching the optimum, inherent in Newton’s method, is in no way suppressed here. At most, the divergence can be reduced.

Despite the interesting properties of this method, calculating the inverse of $\left(H+\mathit{\mu I}\right)$, makes its use tricky for heavy neural networks. As a result, like Newton’s method, it is advisable to automatically switch to the conjugate gradient method when this divergence phenomenon appears. Second-order methods greatly reduce the number of iterations, but increase the computation time.

3.3 Heuristic optimization methods

The advantage of heuristic optimization methods is the minimization of non-derivable cost functions, even for a large number of parameters $(1<n<{10}^5$).

Among the effective methods, we distinguish the optimization algorithm by Particle Swarms introduced by Kennedy and Eberhart and improved by Clerc is an optimization technique by agents which is essentially inspired by the behavior social in flocks of birds or schools of fish.

In addition, genetic algorithms, evolutionary algorithms, also constitute stochastic optimization techniques inspired by the theory of evolution according to Darwin, now widely used in numerical optimization, when the functions to be optimized are complex, irregular, poorly known or in question. Combinatorial optimization.

These heuristic methods of the methods presented previously (Levenberg–Marquardt, Newton, Conjugate Gradient, …) by three main aspects:

1. they do not require the gradient calculation,

2. they study a population as a whole while deterministic methods treat an individual who will evolve towards the optimum,

3. they involve random operations.

Experience has also shown that if the components as well as the evolution parameters are carefully tuned, it is possible to obtain extremely efficient and fast algorithms. However, this adjustment step can be very delicate and constitutes a drawback of the implementation of these methods.

4. Tuning neural network controller using classical approach

The architecture shown in Figure 1 assumes the role of two neural blocks. Indeed, the weights of the neural model are adjusted by the identification error $e\left(k\right)$, however the weights of the neural controller are trained by the tracking error $e_c\left(k\right)$.

The multi-layer perceptron is used in the neural model and in the neural controller. Each block consists of three layers. The sigmoid activation function $s$is used for all neurons.

Concerning the neural network model, the $j^{\mathit{th}}$output layer of the hidden layer is described as follows

$$h_j=\sum _{i=1}^{n_1}{w}_{\mathit{ji}}{x}_{i}j=1,2,\dots ,n_2$$E10

where $n_1$is the number of nodes of the input layer, $w_{\mathit{ji}}$is the hidden weight, $x_i$is the input vector of the neural model, $x={\left[u\left(k\right),u\left(k-1\right),u\left(k-2\right),\dots \right]}^T$, $u\left(k\right)$is the control input to the system and $n_2$is the number of nodes of the hidden layer given in the expression (3).

The output of the neural network model is given by the following equation

$$\mathit{yr}\left(k+1\right)=\lambda s\left(\sum _{j=1}^{n_2}{w}_{1j}s\left(h_j\right)\right)$$E11

where $w_{1j}$is the weight from the hidden layer to the output layer and $\lambda$is a scaling coefficient. The compact form of the output is given by the following equation

$$\mathit{yr}\left(k+1\right)=\lambda s\left(h_1\right)=\lambda s\left[w_1^{T}S\left(\mathit{Wx}\right)\right]$$E12

with

$$x={\left[x_i\right]}^T,i=1,\dots ,n_1,$$

$$W=\left[w_{\mathit{ji}}\right],i=1,\dots ,n_1,j=1,\dots ,n_2,$$

$$S\left(\mathit{Wx}\right)={\left[s\left(h_j\right)\right]}^T,j=1,\dots ,n_2,$$

$$w_1={\left[w_{1j}\right]}^T,j=1,\dots ,n_2.$$

The incremental change of the hidden weights $\mathrm{\Delta }{w}_{\mathit{ij}}$, $i=1,\dots ,n_1$and $j=1..n_2$, is

$$\mathrm{\Delta }{w}_{\mathit{ji}}=-\eta \frac{\partial E}{\partial w_{\mathit{ji}}}=-\eta \frac{\partial E}{\partial e}\frac{\partial e}{\partial h_1}\frac{\partial h_1}{\partial h_j}\frac{\partial h_j}{\partial w_{\mathit{ji}}}$$E13

$$\mathrm{\Delta }{w}_{\mathit{ji}}=\mathit{\eta \lambda }{s}^{\prime }\left(h_1\right)S^{\prime }\left(\mathit{Wx}\right)w_{1j}{x}^{T}e\left(k\right)$$E14

with $\eta$is the learning rate, $0\le \eta \le 1$, $S^{\prime }\left(\mathit{Wx}\right)=\mathit{diag}{\left[s^{\prime }\left(h_j\right)\right]}^T,j=1,\dots ,n_2$, $s^{\prime }\left(h_1\right)$is the derivative of $s\left(h_1\right)$defined as follows

$$s^{\prime }\left(h_1\right)=s\left(h_1\right)\left(1-s\left(h_1\right)\right)$$E15

$e\left(k\right)$is the identification error which is given by

$$e\left(k\right)=y\left(k\right)-\mathit{yr}\left(k\right)$$E16

and the function cost which is given by the following equation

$$E=\frac{1}{2}\sum _{k=1}^N{\left(e\left(k\right)\right)}^2=\frac{1}{2}\sum _{k=1}^N{\left(y\left(k\right)-\mathit{yr}\left(k\right)\right)}^2$$E17

where $N$is the number of observations.

The incremental change of the hidden weights $\mathrm{\Delta }{w}_{\mathit{ij}}$is used in the following equation

$$w_{\mathit{ji}}\left(k+1\right)=w_{\mathit{ji}}\left(k\right)+\mathrm{\Delta }{w}_{\mathit{ji}}\left(k\right)$$E18

However, the output weights are updated by the following equation

$$w_{1j}\left(k+1\right)=w_{1j}\left(k\right)+\mathrm{\Delta }{w}_{1j}\left(k\right)$$E19

where $\mathrm{\Delta }{w}_{1j}$is

$$\mathrm{\Delta }{w}_{1j}=-\eta \frac{\partial E}{\partial w_{1j}}=-\eta \frac{\partial E}{\partial e}\frac{\partial e}{\partial h_1}\frac{\partial h_1}{\partial w_{1j}}$$E20

$$\mathrm{\Delta }{w}_{1j}\left(k\right)=\mathit{\eta \lambda e}\left(k\right)s^{\prime }\left(h_1\right)S\left(\mathit{Wx}\right)$$E21

Concerning the neural network controller, the $j^{\mathit{th}}$output layer of the hidden layer is

$$h_{\mathit{cj}}=\sum _{i=1}^{n_3}{v}_{\mathit{ji}}{x}_{1i}j=1,\dots ,n_4$$E22

where $n_3$is the number of nodes of the input layer, $v_{\mathit{ji}}$is the hidden weight and $x_{1i}$is the input vector of the neural network controller $x_1={\left[r\left(k\right),r\left(k-1\right),r\left(k-2\right),\dots \right]}^T$, $r\left(k\right)$is the desired value.

The output of the neural network controller is given by the following equation

$$u\left(k\right)={\lambda }_{c}s\left(\sum _{j=1}^{n_4}{v}_{1j}s\left(h_{\mathit{cj}}\right)\right)={\lambda }_{c}s\left(\sum _{j=1}^{n_4}{v}_{1j}s\left(\sum _{i=1}^{n_3}{v}_{\mathit{ji}}{x}_{1i}\right)\right)$$E23

where $n_4$is the number of nodes of the hidden layer, ${\lambda }_c$is a scaling coefficient and $v_{1j}$is the output weight.

The compact form of the output of the neural network controller is given by the following equation

$$u\left(k\right)={\lambda }_{c}s\left(h_{c1}\right)={\lambda }_{c}s\left[v_1^{T}S\left({\mathit{Vx}}_1\right)\right]$$E24

with

$$x_1={\left[x_{1i}\right]}^T,i=1,\dots ,n_3,$$

$$V=\left[v_{\mathit{ji}}\right],i=1,\dots ,n_3,j=1,\dots ,n_4,$$

$$S\left({\mathit{Vx}}_1\right)={\left[s\left(h_j\right)\right]}^T,j=1,\dots ,n_4,$$

$$v_1={\left[v_{1j}\right]}^T,j=1,\dots ,n_4.$$

Concerning the hidden synaptic weights, they are updated by

$$v_{\mathit{ji}}\left(k+1\right)=v_{\mathit{ji}}\left(k\right)+\mathrm{\Delta }{v}_{\mathit{ji}}\left(k\right)$$E25

where $\mathrm{\Delta }{v}_{\mathit{ji}}$is given by

$$\mathrm{\Delta }{v}_{\mathit{ji}}=-{\eta }_c\frac{\partial E_c}{\partial e_c}\frac{\partial e_c}{\partial y}\frac{\partial \mathit{yr}}{\partial h_1}\frac{\partial h_1}{\partial s\left(h_j\right)}\frac{\partial s\left(h_j\right)}{\partial h_j}\frac{\partial h_j}{\partial u}\frac{\partial u}{\partial h_{c1}}\frac{\partial h_{c1}}{\partial h_{\mathit{cj}}}\frac{\partial h_{\mathit{cj}}}{\partial v_{\mathit{ji}}}$$E26

with ${\eta }_c$is the learning rate, $0\le {\eta }_c\le 1$and the function cost defined as follows

$$E_c=\frac{1}{2}\sum _{k=1}^N{\left(e_c\left(k\right)\right)}^2=\frac{1}{2}\sum _{k=1}^N{\left(y\left(k\right)-r\left(k\right)\right)}^2$$E27

where $N$is the number of observations and $e_c\left(k\right)$is the tracking error which is given by the following equation

$$e_c\left(k\right)=y\left(k\right)-r\left(k\right)$$E28

where $r\left(k\right)$is the desired output. So $\mathrm{\Delta }{v}_{\mathit{ji}}$comes

$$\mathrm{\Delta }{v}_{\mathit{ji}}={\eta }_c{\lambda }_c{e}_c\left(k\right)s^{\prime }\left(h_1\right)w_{1j}{S}^{\prime }\left(\mathit{Wx}\right)w_{\mathit{ji}}{s}^{\prime }\left(h_{c1}\right)v_{1j}{S}^{\prime }\left({\mathit{Vx}}_1\right)x_1^T$$E29

with $S^{\prime }\left({\mathit{Vx}}_1\right)=\mathit{diag}{\left[s^{\prime }\left(h_j\right)\right]}^T,j=1,\dots ,n_4$.

The output synaptic weights of the neural network controller are updated as

$$v_{1j}\left(k+1\right)=v_{1j}\left(k\right)+\mathrm{\Delta }{v}_{1j}\left(k\right)$$E30

where $\mathrm{\Delta }{v}_{1j}$is given by

$$\mathrm{\Delta }{v}_{1j}=-{\eta }_c\frac{\partial E_c}{\partial v_{1j}}=-{\eta }_c\frac{\partial E_c}{\partial e_c}\frac{\partial e_c}{\partial h_{c1}}\frac{\partial h_{c1}}{\partial v_{1j}}$$E31

So

$$\frac{\partial e_c\left(k\right)}{\partial h_{c1}}=\frac{\partial \left(y\left(k\right)-r\left(k\right)\right)}{\partial h_{c1}}=\frac{\partial y\left(k\right)}{\partial h_{c1}}=\frac{\partial y\left(k\right)}{\partial u\left(k\right)}\frac{\partial u\left(k\right)}{\partial h_{c1}}$$E32

and the Eq. (31) becomes

$$\mathrm{\Delta }{v}_{1j}=-{\eta }_c\frac{\partial E_c}{\partial e_c}\frac{\partial y\left(k\right)}{\partial u}\frac{\partial u\left(k\right)}{\partial h_{c1}}\frac{\partial h_{c1}}{\partial v_{1j}}$$E33

or in Eq. (33), $y\left(k\right)$does not depend on $h_1$, for this reason we use $\mathit{yr}\left(k\right)$instead of $y\left(k\right)$under the condition that the neural model is equal to the system behavior which gives

$$\frac{\partial y\left(k\right)}{\partial u}=\frac{\partial \mathit{yr}\left(k\right)}{\partial u}=\frac{\partial \mathit{yr}\left(k\right)}{\partial h_1}\frac{\partial h_1}{\partial s\left(h_j\right)}\frac{\partial s\left(h_j\right)}{\partial h_j}\frac{\partial h_j}{\partial u}$$E34

from where approximately

$$\mathrm{\Delta }{v}_{1j}=-{\eta }_c\frac{\partial E_c}{\partial e_c\left(k\right)}\frac{\partial e_c\left(k\right)}{\partial y\left(k\right)}\frac{\partial \mathit{yr}\left(k\right)}{\partial h_1}\frac{\partial h_1}{\partial s\left(h_j\right)}\frac{\partial s\left(h_j\right)}{\partial h_j}\frac{\partial h_j}{\partial u\left(k\right)}\frac{\partial u\left(k\right)}{\partial h_{c1}}\frac{\partial h_{c1}}{\partial v_{1j}}$$E35

the obtained incremental change $\mathrm{\Delta }{v}_{1j}$is rewritten as

$$\mathrm{\Delta }{v}_{1j}={\eta }_c{\lambda }_c{e}_c\left(k\right)s^{\prime }\left(h_1\right)w_{1j}{S}^{\prime }\left(\mathit{Wx}\right)w_{\mathit{ji}}{s}^{\prime }\left(h_{c1}\right)S\left({\mathit{Vx}}_1\right)$$E36

In this section, we used a fixed learning rate, $\eta \left(k\right)$(respectively ${\eta }_c\left(k\right)$), and a derivative of sigmoid function $s^{\prime }\left(h_1\right)=s\left(h_1\right)\left(1-s\left(h_1\right)\right)$.

This approach has two drawbacks. First, to find a suitable fixed learning rate $\eta \left(k\right)$(respectively ${\eta }_c\left(k\right)$), several tests are called which decreases the on-line operation. Second, when we use this type of derivative of sigmoid function, a large amount of error should not be spread to the weights of the output layer and the learning speed becomes very slow. In order to increase the learning speed, some of the new proposed approaches are proposed in the next section.

5. Tuning neural network controller using particle swarm optimization

An alternative technique is proposed, in this section, to optimize the neural network controller by implementing Particle Swarm Optimization algorithm. This algorithm works like animal behavior on finding foods and avoiding danger, where they will coordinate with each other to find the best position to settle. Likewise, PSO is directed by the movement of the best individual from the population, known as the social compound, and their own experience, known as the cognitive compound. The algorithm moves the set of solutions to find the best solution among them.

5.1 Mathematical formulation

In this study, the Particle Swarm Optimization Feedforward Neural Network (PSO NN) is applied to a multi-layered perceptron where the position of each particle, in a swarm, represents the set of synaptic weights of the neural network for the current iteration. The dimensionality of each particle is the number of synaptic weights.

Let us consider a search space of dimension $D$. A particle $i$of the swarm is modeled by a position vector

$$x_{\mathit{ij}}={\left[x_{i1},x_{i2},\dots ,x_{\mathit{iD}}\right]}^T$$E37

and a velocity vector denoted

$$v_{\mathit{ij}}={\left[v_{i1},v_{i2},\dots ,v_{\mathit{iD}}\right]}^T$$E38

There is no concept of backpropagation in PSO NN where the direct neural network produces the learning error, objective function of each particle, based on the set of synaptic weights and biases, the positions of the particles. Each particle moves in the weighting space trying to minimize the learning error and keeps in memory the best position through which it passed, denoted

$${\mathit{\text{Pbest}}}_{\mathit{ij}}={\left[{\mathit{\text{pbest}}}_{i1},{\mathit{\text{pbest}}}_{i2},\dots ,{\mathit{\text{pbest}}}_{\mathit{iD}}\right]}^T$$E39

whereas the best position reached by the swarm is denoted

$${\mathit{\text{Gbest}}}_{\mathit{ij}}={\left[{\mathit{\text{gbest}}}_{i1},{\mathit{\text{gbest}}}_{i2},\dots ,{\mathit{\text{gbest}}}_{\mathit{iD}}\right]}^T$$E40

Changing the position means updating the synaptic weights of the neural network controller to generate the proper control law by reducing tracking error.

In each iteration $k$, the particles update their position by calculating the new velocity and move to the new position. At the iteration $\left(k+1\right)$, the velocity vector is calculated as follows:

$$v_{\mathit{ij}}\left(k+1\right)={\mathit{wv}}_{\mathit{ij}}\left(k\right)+c_1{r}_1\left[{\mathit{\text{pbest}}}_{\mathit{ij}}\left(k\right)-x_{\mathit{ij}}\left(k\right)\right]+c_2{r}_2\left[{\mathit{\text{gbest}}}_{\mathit{ij}}\left(k\right)-x_{\mathit{ij}}\left(k\right)\right]$$E41

where:

$w$being a variable parameter making it possible to control the changing of the particle at the next iteration,

${\mathit{wv}}_{\mathit{ij}}$is a physical changing component,

$c_1{r}_1\left[{\mathit{\text{pbest}}}_{\mathit{ij}}\left(k\right)-x_{\mathit{ij}}\left(k\right)\right]$is a cognitive changing component,

$c_2{r}_2\left[{\mathit{\text{gbest}}}_{\mathit{ij}}\left(k\right)-x_{\mathit{ij}}\left(k\right)\right]$a social component of changing,

$c_1$and social $c_2$are respectively, two cognitive confidence coefficients and social which are present the degree of attraction towards the best position of a particle and that of these informants,

$r_1$and $r_2$are two random numbers drawn uniformly in the interval $\left[0,1\right]$represent the proper exploration of particles in the search space.

The smallest learning error of each particle ${\mathit{\text{Pbest}}}_i$and the smallest learning error found in the whole learning process ${\mathit{\text{Gbest}}}_i$are applied to produce a fit of the positions towards the best solution or the targeted tracking error.

The position, at the iteration $\left(k+1\right)$, of particle $i$is then defined as follows:

$$x_{\mathit{ij}}\left(k+1\right)=x_{\mathit{ij}}\left(k\right)+v_{\mathit{ij}}\left(k\right)$$E42

Once the change in positions has taken place, an update affects both ${\mathit{\text{Pbest}}}_i$and ${\mathit{\text{Gbest}}}_i$vectors. At the iteration $\left(k+1\right)$, these two vectors will be updated according to the following two formulations:

$${\mathit{\text{Pbest}}}_i\left(k+1\right)=\left\{\begin{array}{lll}{\mathit{\text{Pbest}}}_i\left(k\right)& \mathit{\text{if}}& f\left(x_i\left(k\right)\right)\ge f\left({\mathit{\text{Pbest}}}_i\left(k\right)\right)\\ x_i\left(k\right)& \mathit{\text{else}}& \end{array}\right.$$E43

and

$${\mathit{\text{Gbest}}}_i\left(k+1\right)=\mathit{Arg}\underset{i}{min}\left[{\mathit{\text{Pbest}}}_i\left(k+1\right)\right]$$E44

The algorithm is executed as long as one of the three, or all at the same time, of the following convergence criteria is verified:

• the maximum number of iterations defined has not been reached,

• the variation in particle speed is close to zero,

• the value of the objective is satisfactory, with respect to the following relation:

$$\mid f\left({\mathit{\text{gbest}}}_{\mathit{jk}}\left(k\right)\right)-f\left({\mathit{\text{gbest}}}_{\mathit{jk}}\left(k-\alpha \right)\right)\mid \le \epsilon ,\mathit{\text{with}}\alpha \in \left[1,N\right]$$E45

The parameter $\epsilon$represents a tolerance chosen, most often, of the order of ${10}^{-5}$and $N$is a number of iterations chosen of the order of $10$.

5.2 The proposed algorithm of particle swarm optimization

In this section, a summary of the proposed algorithm of the PSO neural network controller is presented.

• Random initialization of the positions and velocity of the $N$particles in the swarm,

• Fork: 1..N Do,

• Repeat,

• Forall particles i Do,

• Calculation of the control law $u_i\left(k\right)$from the controller input vector $x_c\left(k\right)$,

• Calculation of the outputs of the system $y_i\left(k+1\right)$,

• Evaluation of the positions of particles in the research space,

• Ifthe current positions of particle $i$produce the best objective function in its history Then,

• ${\mathit{\text{Pbest}}}_i$$\leftarrow$$e_{c_i}$,

• Ifthe objective function of particle i is the best overall objective function Then,

• ${\mathit{\text{Gbest}}}_i$$\leftarrow$$e_{c_i}$,

• End If,

• End If,

• End For,

• Moving of particles according to Eqs. (41) and (42),

• Evaluation of particle positions,

• Update ${\mathit{\text{Pbest}}}_i$and ${\mathit{\text{Gbest}}}_i$according to Eqs. (43) and (44),

• Untilreaching the stop criterion,

• End of PSO.

In this section, we have proposed the PSO optimization steps of an indirect neural network adaptive controller. The corresponding algorithm will be applied for discrete SISO nonlinear systems.

6. Results and discussion

In this section, a time-varying nonlinear discrete systems is used which is described by the input–output model in the following Equation [21].

$$y\left(k+1\right)=\frac{y\left(k\right)y\left(k-1\right)y\left(k-2\right)u\left(k-1\right)\left(y\left(k-2\right)-1\right)+u\left(k\right)}{a_0\left(k\right)+a_1\left(k\right)y^2\left(k-1\right)+a_2\left(k\right)y^2\left(k-2\right)}$$E46

where $y\left(k\right)$and $u\left(k\right)$are respectively the output and the input of the time-varying nonlinear system at instant $k$; $a_0\left(k\right)$, $a_1\left(k\right)$and $a_2\left(k\right)$are given by

$$\left\{\begin{array}{l}{a}_0\left(k\right)=1\\ a_1\left(k\right)=1+0.2\mathit{cos}\left(k\right)\\ a_2\left(k\right)=1+0.2\mathit{sin}\left(k\right)\end{array}\right.$$E47

The trajectory of $a_1\left(k\right)$and $a_2\left(k\right)$are given in the following Figure 2.

Figure 2.

$a_1\left(k\right)$and$a_2\left(k\right)$trajectories.

Figure 3.

The NN control system output and the desired values.

In this section, in order to examine the effectiveness of the proposed algorithm of neural network controller and the PSO neural network controller different performance criteria are used. Indeed, the mean squared tracking error (${\mathit{MSE}}_{e_c}$) and the mean absolute tracking error (${\mathit{MAE}}_{e_c}$) are, respectively, given by respectively, given by

$${\mathit{MSE}}_{e_c}=\frac{1}{N}\sum _{k=1}^N{\left(y\left(k\right)-r\left(k\right)\right)}^2$$E48

and

$${\mathit{MAE}}_{e_c}=\frac{1}{N}\sum _{k=1}^N\left(y\left(k\right)-r\left(k\right)\right)$$E49

where $y\left(k\right)$is the time-varying system output, $r\left(k\right)$is the desired value and the used number of observations $N$is $100$.

In this simulation, the desired value, $r\left(k\right)$, is given in the following

$$r\left(k\right)=\mathit{sin}\left(2\mathit{\pi k}/25\right);$$E50

6.1 Simulation system using classical NN controller

In this section, we examine the effectiveness of the used classical neural network controller in the adaptive indirect control system. Indeed, in offline phase, using a reduced number of observations $\left(M=3\right)$to find, either, the parameter initialization of the neural network parameters ($w_{1j}$, $w_{\mathit{ji}}$, $v_{1j}$, $v_{\mathit{ji}}$).

In online phase, at instant $\left(k+1\right)$, we use the input vector of the neural network controller $x_1={\left[x_r\left(k\right),x_r\left(k-1\right),x_r\left(k-2\right),x_r\left(k-3\right),x_r\left(k-4\right)\right]}^T$. The results of simulation are given by Figures 3–5.

In this case, both neural network model and neural network controller consist of single input, $1$hidden layer with $8$nodes, and a single output node, identically. The used scaling coefficient is $\lambda ={\lambda }_c=1$and ${\epsilon }_1={\epsilon }_2={10}^{-2}$.

Using a multilayered perceptron architecture, three layers: one input layer, one hidden layer and one output layer. The result of simulation are given by the following figures.

6.2 Simulation system using PSO NN controller

The PSO parameters values are respectively the number of variables (m = 50), the population size (pop = 10), the maximum of inertia weight 0.9, the minimum of the inertia weight 0.4, the first acceleration factor (c1 = 2) and the second acceleration factor (c2 = 2).

Using a multilayered perceptron architecture, three layers: one input layer, one hidden layer and one output layer. The result of simulation are given by the following figures.

Figure 6 presents the control system output and the desired values. In this case, the neural network parameters of controller are optimized by PSO technique. A concordance between the desired values and the control system output is noticed, although the time-varying parameters.

Figure 4.

The control law.

However, Figures 7 and 8 present respectively the control law and the control error. These figures reveal that the PSO NN controller has smaller errors than the other controller.

Figure 5.

The control error.

Table 1 presents the influence of the PSO technique in the control error.

	NN controller	PSO NN controller
$\eta$	variable	variable
${\mathit{MSE}}_{e_c}$	$0.0349$	$3.7730e-04$
$\mathit{max}\left(e_c\right)$	$0.3291$	$0.1372$
$\mathit{min}\left(e_c\right)$	$-0.4907$	$-9.9998e-04$
${\mathit{MAE}}_{e_c}$	$0.1305$	$0.0043$
time (s)	$323.829$	$100.926$

Table 1.

The influence of the PSO technique in the control error.

From Table 1 we observe that, using the neural network controller, the PSO neural network controller has the smallest performance criteria in the control error $e_c\left(k\right)$. These results are shown in Figures 6–8.

Figure 6.

The PSO NN control system output and the desired values.

Figure 7.

The control law.

Figure 8.

The control error.

6.3 Effect of disturbances

An added noise $v\left(k\right)$is injected to the output of the time-varying nonlinear system in order to test the effectiveness of the proposed optimization technique of the neural network controller. To measure the correspondence between the system output and the desired value, a Signal Noise Ratio $\left(\mathit{SNR}\right)$is taken from the following equation:

$$\mathit{SNR}=\frac{\sum _{k=0}^N\left(y\left(k\right)-\overline{y}\right)}{\sum _{k=0}^N\left(v\left(k\right)-\overline{v}\right)}$$E51

with $v\left(k\right)$is a noise of the measurement of symmetric terminal $\delta$, $v\left(k\right)\in \left[-\delta ,\delta \right]$, $\overline{y}$and $\overline{v}$are an output average value and a noise average value respectively. In this paper, the taken SNR is $5\%$.

Using the desired value $r\left(k\right)$, the sensitivity of the proposed neural network controller is examined in Table 2.

	NN controller	NN PSO controller
${\eta }_c$	variable	variable
${\mathit{MSE}}_{e_c}$	$0.0351$	$3.7728e-04$
${\mathit{MAE}}_{e_c}$	$00.1247$	$0.0042$
$\mathit{max}\left(e_c\right)$	$0.3123$	$0.1372$
$\mathit{min}\left(e_c\right)$	$0.5000$	$-9.9999e-04$
time (s)	$44.456972$	$337.728385$

Table 2.

The influence of the PSO optimization in the control error.

From this table, we observe that, using the PSO as a method to optimize the parameters of neural network controller, we have got the smallest performance criteria in the control error.

According to the obtained simulation results, the lowest ${\mathit{MSE}}_{e_c}$, ${\mathit{MAE}}_{e_c}$and $\mathit{max}\left(e_c\right)$are obtained using a combination between the neural network controller and the PSO technique, although the added disturbance in the system output and the time-varying parameters thanks to the PSO technique.

7. Conclusion

In this chapter, a comparative study between the neural network controller and neural network PSO controller is proposed and is applied with success in indirect adaptive control. For instance, the lowest ${\mathit{MSE}}_{e_c}$, ${\mathit{MAE}}_{e_c}$, $\mathit{min}\left(e_c\right)$and $\mathit{max}\left(e_c\right)$are obtained and it is proved that the PSO method is the best. The effectiveness of the proposed algorithm is successfully applied to single-input single-output system, with and without disturbances, and it proved its robustness to reject disturbances and to accelerate the speed of the learning phase of the neural model and neural controller.