Robust Adaptive Cooperative Control for Formation-Tracking Problem in a Network of Non-Affine Nonlinear Agents

In this chapter, a decentralized cooperative control protocol is proposed with application to any network of agents with non-affine nonlinear multi-input-multi-output (MIMO) dynamics. Here, the main purpose of cooperative control protocol is to track a time-variant reference trajectory while maintaining a desired formation. The reference trajectory is defined to a leader, which has at least one information connection with one of the agents in the network. The design procedure includes a robust adaptive law for estimating the unknown nonlinear terms of each agent’s dynamics in a model-free format, that is, without the use of any regressors. Moreover, an observer is designed to have an approximation on the values of control parameters for the leader at the agents without connection to the leader. The entire design procedure is analysed successfully for the stability using Lyapunov stability theorem. Finally, the simulation results for the application of the proposed method on a network of nonholonomic wheeled mobile robots (WMR) are presented. Desirable leader-following tracking and geometric formation control performance have been successfully demonstrated through simulated group of wheeled mobile robots.


Introduction
Great attention has been paid to the problems of the multi-agent network ranging from consensus, collective behaviours of flocks and swarms, formation control of multi-robot systems, leaderfollowing, algebraic connectivity of complex network, rendezvous, containment and so on [1][2][3][4][5][6].
The formation control problem is an interesting issue in biology, automatic control, robotics, artificial intelligence and so on, which requires each agent to move according to the prescribed trajectory. Various control strategies have been formulated to achieve the group control objectives.
The systems are usually in nonlinear form due to unpredictable environmental disturbances, unmodelled dynamics or other uncertainties. A class of nonlinear first-order multi-agent systems with external disturbances consensus problem was discussed in Ref. [7], whereas other works that involve second-order and higher order nonlinear multi-agent systems are reported in Refs. [8] and [9], respectively. Wang et al. [10] reported the design of distributed state/output feedback cooperative control approaches for uncertain multi-agents in undirected communication graphs. This is later extended to a condition of directed graphs containing a spanning tree [11]. To remedy the problem of a non-affine system for a general class, several reported works such as Ref. [12] employ a direct adaptive approach using an artificial neural network (ANN) to approximate an ideal controller. By employing a system transformation, a non-affine system can be transformed into an affine system as demonstrated in Ref. [11]. However, the transformation technique to convert a multi-agent non-affine system to a multi-agent affine system is still new and open to further studies which are to be discussed in this chapter.
Hou et al. [13] illustrate the method of dealing with non-affine multi-agent system by incorporating dynamic surface control or DSC but it is limited to a single-input-single-output (SISO) type of system, that is, with one control input. A similar approach is reported in Ref. [14] where the distributed dynamic surface design approach is used to design local consensus controllers using the transformation to convert the system to an affine strict-feedback multiagent system. The work is also limited to a single control input per agent.
In this chapter, several novel contributions can be highlighted, that is, the introduction of transformation techniques from a non-affine multi-agent system to an affine multi-agent system for a network of generic nonlinear multi-input-multi-output (MIMO) systems, that is, a single agent may have more than one control input and more than one output. The second contribution to be highlighted in the chapter is the estimation of nonlinear terms in the dynamics without requiring the linear-in-parameter condition (LIP), that is, the dependence on any model regressor is elevated. The lumped nonlinear function existing in the model agent can be estimated online despite time-varying characteristics. This implies that the estimation is model free. By virtue of a sigmamodified adaptive law with projection algorithm that drives the estimation using the cooperative consensus error, the unknown nonlinear function can be reconstructed. The proposed cooperative control scheme requires a robust adaptive observer which can reconstruct the control signal from all agents to be used in the consensus formation control. Owing to the robustification term in the observer, the control signals can be estimated in finite time. The proposed robust adaptive formation control is to be exemplified in a form of simulation of multi nonholonomic mobile robots with differential drive configurations. They are commissioned to follow the leader trajectory while at the same time required to maintain predefined geometric formation guaranteeing safe inter-agent separation.
The chapter is organized into preliminaries, problem definition, design procedure of the proposed robust adaptive formation control algorithm, simulated results and lastly the conclusion of the chapter.

Mean value theorem
Suppose that the function F is continuous on the closed interval ½a, b and differentiable on the open interval ða, bÞ (i.e. F is Lipschitz). Then, there is a point X 0 in the open interval ða, bÞ at which [15] In physical terms, the mean value theorem says that the average velocity of a moving object during an interval of time is equal to the instantaneous velocity at some moment in the interval [15].

Schur complement lemma
For any constant symmetric matrix S ¼ " , the following statements are equivalent [17] -

Graph theory preliminaries
Consider a network consisting of N agents. Let GðV, E, AÞ be a graph with the set of N nodes An edge e ij in G is a link between a pair of nodes (ν j , ν i Þ, representing the flow of information from ν j (as parent) to ν i (as child). The e ij is in existence if and only if a ij > 0. The graph is undirected, that is, the e ij and e ij in G are considered to be the same. We name ν i and ν j as neighbors if e ij ∈ E . A path is defined as a sequence of connected edges in a graph. A graph is connected if there is a path between every pair of the nodes. The degree matrix where each d i is the input degree to each node, which is equal to the number of all edges through it (i.e. d i ¼ X j¼1:N a ij ). Hence, we can define Laplacian Matrix (L) as below [16,18,19] Furthermore, we can define an adjacency matrix for the leader as follows where each b i indicates the existence of a communication link between the leader and each agent [16,18,19]. Besides, we would have,

Problem definition
Consider a network of N agents with general non-affine nonlinear dynamics for each of them. The problem is to design a set of decentralized control protocols for all agents to enhance a desired formation in the state space and also track a reference trajectory on state variables. Here, a virtual node is considered as the leader, which knows the desired trajectory and has at least one communication link with the agents in the network. It means that some agents are unaware about the leader states and also their control inputs. The whole problem in a general format can be considered as a platform for any possible state space in diverse applications. For a MIMO system, one can define the following general nonlinear formulation where n is the number of states for the system, t is the total number of nonlinear terms in the system (which t ≤ n), x i ∈ R n is the states vector, u i ∈ R m is the input (or control parameters) vector, m is the number of control parameters, h j for j ¼ ½1, n is any linear combination on x i , R j for j ¼ ½1, n is any Lipschitz continuous nonlinear function on x i and f j for j ¼ ½1, n is any Lipschitz continuous nonlinear function on both x i and u i . The last term defines the non-affine property of the system which represents the completely coupled inter-relation between states and control parameters. Each agent dynamic can be represented in matrix form as follows where C ∈ R nÂn is a constant matrix including the multipliers for each state. The elements of C define the dependence of each state's derivative to the other states.
For a network of N of similar agents (or systems), dynamics for each agent i can be represented by Eq. (9). Also, the dynamic of the leader node can be proposed by this format. The difference is that the control parameters for the leader are defined with respect to a time-varying reference trajectory, that is where h 0 j for j ¼ ½1, n is any linear combination on the leader control parameters (i.e. reference trajectory u 0 ). Actually, the reference trajectory is a set of inputs which provide certain dynamics in state space for the leader agent. The leader dynamics can be represented in the matrix form as the following: where D ∈ R nÂm is a constant matrix including the multipliers for each control parameters.
Moreover, the desired formation among the agents in a network can be presented by a set of constant values F ∈ ðR N Â R n Þ, which determines the relative distance between agents in the state space.
The problem is to enhance F among the network agents and track the reference trajectory defined by (x 0 , u 0 ) at the leader node with inter-agent communication topology defined by the communication graph.

Design procedure for robust adaptive cooperative control protocol
This section is dedicated to presenting the design process for cooperative control protocol, an observer to estimate the control parameters of the leader at each agent and a robust adaptive law to estimate the nonlinear terms at each agent. The design process is initiated by dealing with the non-affinity property of the agents.

Dealing with non-affinity property
Using the mean-value theorem presented in Section 1, for the nonlinear functions f j , which has a coupled terms of x i and u i , we have [19] ∂f and without any loss of generality we can consider μ = 1 and u i is any constant value.
where q j ðx i Þ is an unknown nonlinear function depending only on x i . As can be seen, the nonaffine nonlinear function f j ðx i , u i Þ is converted to an affine form. Now, the dynamics of each agent can be modified as Considering where g j ðx i Þ is an unknown nonlinear function depending on x i , the matrix format for each agent dynamics can be presented as where D ∈ R nÂm is a constant matrix including the multipliers for each control parameter. Actually, the elements of D define the dependence of each state's derivative to each control parameters. Moreover, D 1 ∈ R nÂt is a diagonal matrix defining the existence of nonlinear functions in the equation for derivative of each state. Elements of D 1 can only be one or zero.
It should be noted that since t ≤ n, we may have some states' derivatives which do not include any nonlinear terms.
In the following subsections, the elements of G i , which define the unknown nonlinear functions on each state's derivative, would be estimated (adapted) online using consensus error of the network.

Cooperative protocol for formation and tracking problem
For a network of N agents with the dynamics described by Eq. (16), we can have a lumped formulation for the dynamics of all agents using the Kronecker product, For this network, we can define the combined formation and tracking errors in a single formulation in relation to the neighbouring information available to each agent i via the communication graph [16] where Δ ∈ R nÂ1 is the vector of desired values for states of agents and also the leader. We can consider e i as the consensus error for agent i. Hence By changing the variables, we have Trying to lump the consensus errors of all agents in an N-array format, we have Besides, considering Eq. (17), we can have an N-array form for dynamics of agents in the changed variables space If the consensus errors of all agents converge to zero, then both formation and tracking objectives are reached, that is Here, the cooperative protocol U is designed using the Lyapunov stability theorem to ensure Eq. (23) is reached. Consider the following Lyapunov function Then, Considering Eq. (3), we have Besides, using Eqs. (3) and (21), we have Then, Eq. (25) leads to, Forcing _ V < 0 and referring to Eq. (11), we have Hence, Based on Lyapunov stability theorem, using U ∈ R NmÂ1 in Eq. (30) as the cooperative control protocol will ensure that _ V < 0 and that E reaches zero asymptotically. Hence, the objectives in formation problem and tracking problem have been accomplished. Expressing the control signal at agent level for agent i and then Finally, the control parameter for agent i can be presented as the following Here, a pseudo-inverse method is employed on D.
There are two required conditions on achieving this goal, which are explained in the following assumptions.
Assumption 1. The communication graph should be undirected and connected. It means sufficient information can be available on agents.
Assumption 2. The dynamics of each agent should be completely controllable, that is D matrix should be full rank. It leads us to a state transformation in some applications.
Looking at the proposed cooperative control protocol in Eq. (33), there are two terms, which are not totally available to all agents: i. u j (fourth term in the prentices in Eq. (33)), which is the control parameter for the neighbouring agent at the current moment.
ii. G j (third term in the prentices in Eq. (33)), which includes the unknown nonlinear terms for dynamics of neighbouring agents.
By reaching consensus on the states of agents, we can conclude that the control parameters of each agent has converged to the values of leader control parameters [20] lim Hence, the control parameters for the neighbouring agent (u j ) are approximated by the control parameter of the leader, which in turn will be observed locally at each agent. It means that each agent has its own estimation on u 0 and sends it to the neighbouring agents as its control parameter. The observed data will be transmitted to the neighbouring agents via communication graph to compute the control protocols.
The unknown nonlinear terms (G j ) also will be estimated using the consensus error of each agent. Similarly, the adapted data are shared with neighbouring agents through the communication graph.

Observer design for leader control parameters
Here, the objective is to have consensus on the value of u 0 among the all agents in the network. For this objective, we can define the following consensus error for each agent whereT i ∈ R mÂ1 is the observed vector at agent i for the leader control parameter, and again the a ij and b i are the elements of adjacency matrix for the communication graph in the network. Eq. (35) can be represented in a lumped format as the following If the equation is satisfied, we can say that the observation objective is achieved. Considering the following Lyapunov function, we have Then, Since the summation of all elements in each row of the Laplacian matrix is zero, we can say that and recalling Eq. (7), Eq. (39) can be written as following, where since ðH ⊗ I m Þ is the positive definite recalling the Schur Complement Lemma, the first term is surely negative. To achieve _ V 1 < 0, we should show that Recalling Eq. (3), we have Hence, the Eq. (43) is, where _ U 0M is the upper band or maximum absolute value for _ u 0 . This value should be available beforehand. Now, we should only show that Hence, where sign Δ T c ðH ⊗ I m Þ ∈ R NmÂNm is a diagonal matrix whose diagonal elements are the signs of each element in Δ T c ðH ⊗ I m Þ ∈ R 1ÂNm . Finally, since we have the second term in and recalling Eq. (36), the rate for the observed parameter is By using _ T from Eq. (50), we can have V 1 ≤ 0, which in turn shows that the consensus error on observation (i.e. Δ c ) is stable in accordance to the Lyapunov stability theorem. It is obvious that the observed values for u 0 (i.e.T ) at each agent are computed iteratively using the rate value proposed in Eq. (50).
The lumped format for rate of observer parameter in Eq. (50) can be presented for each agent as the following where Δ ci is defined as in Eq. (35).

Adaptive law design for unknown nonlinear terms in each agent dynamics
In this subsection, the objective is to estimate the values of unknown nonlinear terms in each agent dynamics (i.e. G in Eq. (30)). Since, there is not any data available on exact values of G, the estimation error for adaptation process is not available. Hence, the adaptation should be handled using the output error which in this problem is the consensus error (i.e. E in Eq. (21)).
Considering the consensus error in Eq. (21) and the agent dynamics according to Eq. (22), the derivative for consensus error is where G here is the exact value for nonlinear terms. If we put the designed cooperative control protocol (from Eq. (30)) withĜ is the adapted value for the unknown nonlinear terms, into Eq. (52), we have Using the following positive definite Lyapunov function where Γ ∈ R NnÂNn is a positive definite matrix, we have where the first term in the last equation is the negative definite. To show _ V 2 < 0, we have Then,G which in turn leads to this adaptive law Considering the Lyapunov stability theorem for the function in Eq. (55), ifĜ is updated using the rate value proposed in Eq. (59) iteratively,G converges to zeros asymptotically. It means that the adapted parameterĜ will converge to the actual value of the nonlinear terms in agent dynamics. One of the important issues of the proposed adaptive law in Eq. (59) is that it is not required to include any set of nonlinear basis functions as regressors in the adaptive law. It is only based on the consensus error of the network, which may have sufficient information to tune the adaptive parameter.
Since the adapted signals are always vulnerable for being distracted and diverged by unknown terms, two robusting methods are provided to make the designed adaptive law robust against the divergence [21].
i. Parameter projection method where M 0i is chosen so that M 0i ≥ jg i j. The value for M 0 should be defined beforehand. The algorithm is named as parameter projection in the literature [21].
ii. σ-modification or leakage method; Hence, the complete robust adaptive control for estimating the nonlinear terms in each agent's dynamics is presented as the following The lumped format for the rate of adaptive parameter in Eq. (60) can be presented for agent i as the following

Application: wheeled mobile robot
In this section, application of the proposed cooperative control protocol on a team including three nonholonomic wheeled mobile robots (WMRs) is presented. The robots are moving on a smooth planar surface with a constraint on the speed (Figure 1). They can only move in the direction of their attitudes and speed in the perpendicular direction is zero. This is a nonholonomic constraint. Few number of researches can be found in literatures, which deal with the cooperative control of the multi-agent of WMRs taking account of each agent's WMR dynamics [22,23].

Problem definition
Here, the kinematics and dynamics for motion of ith WMR are considered as the following where x i and y i represent the position of a single WMR in the inertial coordinate system, θ i is the orientation of the WMR, υ i is the translational speed in the WMR's pose direction and ω i is the angular speed of WMR about the Z axis. Also, m and J are the mass and moment of inertia for WMR. Moreover, F i and T i are the force and torque generated by the electric motors disclosed in each wheel of WMR. The last parameters are the control parameters for motion of each WMR. By transforming the kinematics of WMR to a local coordinate system fixed to the WMR, [24] x i1 x i2 x i3 Then by considering where u i1 ¼ 1 m F i and u i2 ¼ 1 J T i . The state-space system can be represented in matrix form similar to Eq. (16), as the following As can be seen, D is not full rank. According to assumption 2, we need a change of variables to have D in the full-rank form. Recalling the idea of the back-stepping method [25] we have Applying the back-stepping method we have Then, the state-space representation of a single WMR can be represented in following format which has a full rank D matrix. Hence, assumption 2 is satisfied and the proposed cooperative controller can be implemented. Hence, we have five state variables, four control parameters and two nonlinear terms for each WMR. At each agent within the network, the nonlinear terms will be adapted using Eq. (63) and the control parameters of the leader will be observed using Eq. (51).
Here, the desired formation is a rectangle with four agents and four equal edges. The length of each edge is equal and is r. The virtual leader is positioned at the centroid of the geometry ( Figure 2). Moreover, the communication graph for this network is shown in Figure 2. The leader information is only available to agent 1. Hence, the adjacency matrices are defined as the following There is a well-known reference trajectory for this problem in the literature [20], which is presented as the following, where υ r and ω r can be any known time-varying functions. Usually, these functions are considered as constant values. In Eq. (73), t is time.

Simulation results
The simulation for the problem defined in Section 5.1 is performed by MATLAB/Simulink. The constant values for running the simulation are presented in Table 1.
Moreover, the values of P i as the gain values for cooperative control protocol at each agent (see Eq. (33)) are as follows P 1 ¼ diagf10, 10, 100, 10, 10g, P 2 ¼ diag{10, 10, 12, 10, 10} P 3 ¼ diagf10, 10, 30, 10, 10g, P 4 ¼ diag{10, 10, 55, 10, 10} The values in P i are determined in a way to ensure that the whole matrix P is positive definite and the sufficient transient performance of the whole network is achieved. The leakage factor (ρ) 100 The maximum value for rate of u 0 ( _ U 0M Þ ones (4,1) The maximum value for adapted signal (M 0 ) 1 0 Â ones (2, 1) The simulation results for this problem are presented in the following figures. The position of all agents in the X-Y plane is shown in Figure 3. The consensus on both reference trajectory and the desired formation can be seen. Actually, the desired formation is achieved gradually. In addition, the position of the centroid of all agents is compared with the reference trajectory in Figure 4. Moreover, the signals for translational and angular speeds of agent 4 are presented in Figure 5. Finally, the observed data for control parameters of the leader and also the adapted nonlinear terms at agent 4 are shown in Figures 6 and 7. Appropriate performance of proposed algorithms can be inferred by these figures.

Conclusion
This chapter is dedicated to the design procedure of a cooperative control protocol for any network consisting of agents with non-affine nonlinear dynamics and multi-input multi-output structure. The main goal is to satisfy a tracking problem for the whole network while maintaining a predefined formation topology in the state space of the agents' dynamics. The proposed design procedure is including an adaptive law incorporated with a robustification method to estimate the unknown nonlinear terms in the agents' dynamics. In addition, an observer is designed using the consensus-type error for estimating the leader's control parameters at each agent. Since there are no complete information links between the leader and all agents, the observed control parameters of the leader are required at each agent to construct the cooperative control protocol. The entire design procedure is analysed successfully for the stability using Lyapunov stability theorem. The presented simulation results for a team of wheeled mobile robots show the appropriate performance of the proposed method.