A Game Theoretic Approach Based Adaptive Control Design for Sequentially Interconnected SISO Linear Systems

Adaptive control has attracted a lot of research attention in control theory for many decades. In the certainty equivalence based adaptive controller design [4, 5], the unknown parameters of the uncertainty system are substituted by their online estimates, which are generated through a variety of identifiers, as long as the estimates satisfy certain properties independent of the controller. This approach leads to structurally simple adaptive controllers and has been demonstrated its effectiveness for linear systemswith or without stochastic disturbance inputs [10] when long term asymptotic performance is considered. Yet, the certainty equivalence approach is unsuccessful to generalize to systems with severe nonlinearities. Also, early designs based on this approach were shown to be nonrobust [13] when the system is subject to exogenous disturbance inputs and unmodeled dynamics. Then, the stability and the performance of the closed-loop system becomes an important issue. This has motivated the study of robust adaptive control in the 1980s and 1990s, and the study of nonlinear adaptive control in the 1990s.


Introduction
Adaptive control has attracted a lot of research attention in control theory for many decades. In the certainty equivalence based adaptive controller design [4,5], the unknown parameters of the uncertainty system are substituted by their online estimates, which are generated through a variety of identifiers, as long as the estimates satisfy certain properties independent of the controller. This approach leads to structurally simple adaptive controllers and has been demonstrated its effectiveness for linear systems with or without stochastic disturbance inputs [10] when long term asymptotic performance is considered. Yet, the certainty equivalence approach is unsuccessful to generalize to systems with severe nonlinearities. Also, early designs based on this approach were shown to be nonrobust [13] when the system is subject to exogenous disturbance inputs and unmodeled dynamics. Then, the stability and the performance of the closed-loop system becomes an important issue. This has motivated the study of robust adaptive control in the 1980s and 1990s, and the study of nonlinear adaptive control in the 1990s.
The topic of adaptive control design for nonlinear systems was studied intensely in the last decade after the celebrated characterization of feedback linearizable or partially feedback linearizable systems [7]. A breakthrough is achieved when the integrator backstepping methodology [8] was introduced to design adaptive controllers for parametric strict-feedback and parametric pure-feedback nonlinear systems systematically. Since then, a lot of important contributions were motivated by this approach, and a complete list of references can be found in the book [9]. Moreover, this nonlinear design approach has been applied to linear systems to compare performance with the certainty equivalence approach. However, simple designs using this approach without taking into consideration the effect of exogenous disturbance inputs have also been shown to be nonrobust when the system is subject to exogenous disturbance inputs.
The robustness of closed-loop adaptive systems has been an important research topic in late 1980s and early 1990s. Various adaptive controllers were modified to render the closed-loop systems robust [6]. Despite their successes, they still fell short of directly addressing the disturbance attenuation property of the closed-loop system.
The objectives of robust adaptive control are to improve transient response, to accommodate unmodeled dynamics, and to reject exogenous disturbance inputs, which are the same as the objectives to motivate the study of the H ∞ -optimal control problem. H ∞ -optimal control was proposed as a solution to the robust control problem, where these objectives are achieved by studying only the disturbance attenuation property for the closed-loop system. The game-theoretic approach to H ∞ -optimal control developed for the linear quadratic problems, offers the most promising tool to generalize the results to nonlinear systems [3]. Worst-case analysis based adaptive control design was proposed in late 1990s to address the disturbance attenuation property directly, and it is motivated by the success of the game-theoretic approach to H ∞ -optimal control problems [2]. In this approach, the robust adaptive control problem is formulated as a nonlinear H ∞ control problem under imperfect state measurements. By cost-to-come function analysis, it is converted into an H ∞ control problem with full information measurements. This full information measurements problem is then solved using nonlinear design tools for a suboptimal solution. This design scheme has been applied to worst-case parameter identification problems [11], which has led to new classes of parametrized identifiers for linear and nonlinear systems. It has also been applied to adaptive control problems [1, 12,14,15,18,19], and the convergence properties is studied in [20]. In [14], adaptive control for a strict-feedback nonlinear systems was considered with noiseless output measurements, and more general class of nonlinear systems was studied in [1]. In [12], single-input and single output (SISO) linear systems were considered with noisy output measurements. SISO linear systems with partly measured disturbance was studied in [18], which leads to a disturbance feed-forward structure in the adaptive controller. [19] generalizes the results of [12] to the adaptive control design for SISO linear systems with zero relative degree under noisy output measurements. In [17], adaptive control for a sequentially interconnected SISO linear system was considered, and a special class of unobservable systems was also studied using the proposed approach. More recently, [16] generalized the result of [17] to adaptive control design for a linear system under simultaneous driver, plant and actuation uncertainties.
In this Chapter, we study the adaptive control design for sequentially interconnected SISO linear systems, S 1 and S 2 (see Figure 1), under noisy output measurements and partly measured disturbance using the similar approaches as [12] and [17]. We assume that the linear systems satisfy the same assumption as [17], and the adaptive control design follows the same design method discussed above. The robust adaptive controller achieves asymptotic tracking of the reference trajectories when disturbance inputs are of finite energy. The closed-loop system is totally stable with respect to the disturbance inputs and the initial conditions. Furthermore, the closed-loop system admits a guaranteed disturbance attenuation level with respect to the exogenous disturbance inputs, where ultimate lower bound for the achievable attenuation performance level is equal to the noise intensity in the measurement channel of S 1 . The results are as same as those in [17]. In addition, the controller achieves arbitrary positive distance attenuation level with respect to the measured disturbances by proper scaling. Moreover, if the measured disturbances satisfy the assumption 2 forw 1,b andw 2,b , the proposed controller achieves disturbance attenuation level zero with respect to the measured disturbances, which further leads to a stronger asymptotic tracking property, namely, the tracking error converges to zero when the unmeasured disturbances are L 2 ∩ L ∞ , and the measured disturbances are L ∞ only.
The balance of this Chapter is organized as follows. In Section 2, we list the notations used in the Chapter. In Section 3, we present the formulation of the adaptive control problem and discuss the general solution methodology. In Section 4, we first obtain parameter identifier and state estimator using the cost-to-come function analysis in Subsection 4.1, then we derive the adaptive control law in Subsection 4.2. We present the main results on the robustness of the system in Section 5, and the example in Section 6. The Chapter ends with some concluding remarks in Section 7.

Notations
We denote IR to be the real line; IR e to be the extended real line; IN to be the set of natural numbers; C to be the set of complex numbers. For a function f , we say that it belongs to C if it is continuous; we say that it belongs to C k if it is k-times continuously (partial) differentiable.

For any matrix A, A denotes its transpose. For any
For n ∈ IN, the set of n × n-dimensional positive definite matrices is denoted by S +n . For n ∈ IN ∪ {0}, I n denotes the n × n-dimensional identity matrix. For any matrix M, M p denotes its p-induced norm, 1 ≤ p ≤ ∞. L 2 denotes the set of square integrable functions and L ∞ denotes the set of bounded functions. For any n, m ∈ IN ∪ {0}, 0 n×m denotes the n × m-dimensional matrix whose elements are zeros. For any n ∈ IN and k ∈ {1, · · · , n}, e n,k denotes 0 1×(k−1) 1 0 1×(n−k) .

Problem Formulation
We consider the robust adaptive control problem for the system which is described by the block diagram in Figure 1. We assume that the system dynamics for S 1 and S 2 are given by,x 1 =À 1x1 +B 1 y 2 +D 1ẁ1 +D 1w1 ; (1a) ( 1 c ) , and E i are of the appropriate dimensions, generally unknown or partially unknown, i = 1, 2. For subsystem S 1 , the transfer function from y 2 to y 1 is H 1 (s) =C 1 (sI n 1 −À 1 ) −1B 1 , for subsystem S 2 , the transfer function from u to y 2 is H 2 (s) =C 2 (sI n 2 −À 2 ) −1B 2 . All signals in the system are assumed to be continuous.
The subsystems S 1 and S 2 satisfy the following assumptions, Assumption 1. For i = 1, 2, the pair (À i ,C i ) is observable; the transfer function H i (s) is known to have relative degree r i ∈ IN, and is strictly minimum phase. The uncontrollable part of S 1 (with respect to y 2 ) is stable in the sense of Lyapunov; any uncontrollable mode corresponding to an eigenvalue of the matrixÀ 1 on the jω-axis is uncontrollable from ẁ 1w 1 . The uncontrollable part of S 2 (with respect to u) is stable in the sense of Lyapunov; any uncontrollable mode corresponding to an eigenvalue of the matrixÀ 2 on the jω-axis is uncontrollable from ẁ 2ý 2w 2 .
Based on Assumption 1, for i = 1, 2, there exists a state diffeomorphism: x i =T ixi , and a disturbance transformation: w i =M iẁi , such that S i can be transformed into the following state space representation, and E i are known and have the following structures, A i = (a i,jk ) n i ×n i ; a i,j (j+1) = 1, a i,jk = 0, for 1 ≤ j ≤ r i − 1 We denote the elements of x 1 and x 2 by x 1,1 · · · x 1,n 1 and x 2,1 · · · x 2,n 2 , with initial conditions x 1,0 and x 2,0 , respectively. Assumption 2. The measured disturbancew 1 can be partitioned as:w 1 = w 1,aw 1,b wherew 1,a iš q 1,a dimensional,q 1,a ∈ IN ∪ {0}, and the transfer function from each element ofw 1,a to y 1 has relative degree less than r 1 + r 2 ; the measured disturbancew 2 can be partitioned as:w 2 = w 2,aw 2,b wherě w 2,a isq 2,a dimensional,q 2,a ∈ IN ∪ {0}, and the transfer function from each element ofw 2,a to y 2 has relative degree less than r 2 .
Based on Assumption 2, the matrixĎ i can be partitioned into Ď i,aĎi,b , whereĎ i,a andĎ i,b have n i ×q i,a -and n i ×q i,b -dimensional, respectively; andĎ i,b ,Ā i,213 (q i,a +1) , · · · ,Ā i,213q i have the following structurě Since we will base our design of adaptive controllers using the model (2), we call (2) the design model, and make the following two assumptions. Define Due to the structures of A i ,Ā i,212 and B i , the high frequency gain of the transfer function To guarantee the stability of the identified system, we make the following assumption on the parameter vectors θ 1 and θ 2 .
Assumption 4. The sign of b i,0 is known; there exists a known smooth nonnegative radially-unbounded strictly convex function P i : Assumption 4 delineates a priori convex compact sets where the parameter vectors θ 1 and θ 2 lie in, respectively. This will guarantee the stability of the closed-loop system and the boundedness of the estimate of θ 1 and θ 2 .
We make the following assumption about the reference signal, y d .
For design purposes, instead of attenuating the effect of ẁ 1w 1ẁ 2w 2 we design the adaptive controller to attenuate the effect of w 1w 1 w 2w 2 . This is done to allow our design paradigm to be carried out. This will result in a guaranteed attenuation level with respect toὼ 1 andὼ 2 . To simplify the notation, we take the uncertainty ω 1 : We state the control objective precisely as follows, Definition 1. A controller μ ∈ M μ is said to achieve disturbance attenuation level γ with respect to w 1w 1,a w 2w 2,a , and disturbance attenuation level zero with respect to w 1,bw 2,b , if there exists functions l 1 (t, θ 1 , , and a known nonnegative constant l 0 (x 1,0 ,x 2,0 ,θ 1,0 ,θ 2,0 ), such that and l 1 ≥ 0 and l 2 ≥ 0 along the closed-loop trajectory, where and n i × n i -dimensional positive definite matrices, respectively, i = 1, 2.
Clearly, when the inequality (4) is achieved, the squared L 2 norm of the output tracking error C 1 x 1 − y d is bounded by γ 2 times the squared L 2 norm of the transformed disturbance input w 1w 1,a w 2w 2,a , plus some constant. When the L 2 norm ofẁ 1 ,ẁ 2 ,w 1 , andw 2 are finite, the squared L 2 norm of C 1 x 1 − y d is also finite, which implies lim under additional assumptions.
Let ξ i denote the expanded state vector ξ i = [θ i , x i ] , i = 1, 2, and note thatθ i = 0, we have the following expanded dynamics for system (2), The worst-case optimization of the cost function (4) can be carried out in two steps as depicted in the following equations. sup where ω m is the measured signals of the system, and defined as The inner supremum operators will be carried out first. We maximize over ω i given that the measurement ω m is available for estimator design, i = 1, 2. In this step, the control input, u, is a function only depended on ω m , then u is an open-loop time function and available for the optimization. Using cost-to-come function analysis, we derive the dynamics of the estimators for subsystem S 1 and S 2 independently.
The outer supremum operator will be carried out second. In this step, we use a backstepping procedure to design the controller μ.
This completes the formulation of the robust adaptive control problem.

Adaptive control design
In this section, we present the adaptive control design, which involves estimation design and control design. First, we discuss estimation design.

Estimation design
In this subsection, we present the estimation design for the adaptive control problem formulated. First, we will derive the identifier of subsystem S 1 . In this step, the measurement waveform y 1 , y 2 and measured disturbancew 1 are assumed to be known. Then we can obtain the identifier of subsystem S 1 from a game-theoretic solution methodology -cost-to-come function analysis.
We first set function l 1 in the definition to be |ξ 1 −ξ 1 | 2Q 1 + 2(ξ 1 − l 1,1 ) l 1,2 +ľ 1 , wherê ) are three design functions to be introduced later, the cost function of subsystem S 1 is then of the a linear quadratic structure.
The robust adaptive problem for S 1 becomes an H ∞ control of affine quadratic problem, and admits a finite dimensional solution. By cost-to-come function analysis, we obtain the dynamics of worst-case covariance matrixΣ 1 , and state estimatorξ 1 , which are given bẏΣ whereL 1 is defined asL 1 = 0 1×σ 1 L 1 .
We summarize the equations for subsystem S 1 as follows, This completes the estimation design of S 1 .
Next, we will derive the estimator for subsystem S 2 . In this step, the measurements waveform ω m is assumed to be known. Since the control input, u, is a causal function of ω m , then it is known. Again, we will apply the cost-to-come function methodology to derive the estimator. We briefly summarize the estimation design for S 2 as follows.
Set function l 2 in definition to be |ξ 2 is the worst-case estimate for the expanded state ξ 2 ,ξ 2 is the estimate of ξ 2 ,Q 2 is a matrix-valued weighting function, l 2,2 andľ 2 are two design functions to be introduced later, the cost function of subsystem S 2 is then of a linear quadratic structure. By cost-to-come function analysis, we obtain the dynamics of worst-case covariance matrixΣ 2 , and state estimatorξ 2 . We partition , then the weighting matrixΣ 2 is positive definite if and only if Σ 2 and Π 2 are positive definite. To guarantee the boundedness of Σ 2 , we choose weighing matrixQ 2 as follows, where Δ 2 (t) = γ −2 β 2,Δ Π 2 (t) + Δ 2,1 , with β 2,Δ ≥ 0 being a constant and Δ 2,1 being an n 2 × n 2 -dimensional positive-definite matrix, and 2 is a scalar function defined by 2 Then the dynamics of Σ 2 , Φ 2 , Π 2 are given as follows, Hurwitz. By Lemma [12], we have the covariance matrix Σ 2 upper and lower bounded as follows, To avoid the calculation of Σ −1 2 online, we define s 2,Σ = Tr(Σ −1 2 ). To guarantee the estimates parameter to be bounded and the estimate of high frequency gain to be bounded away from zero without persistently exciting signals, we introduce the following soft projection design on the parameter estimate.
Associated with the above identifier and estimator of subsystem S i , i = 1, 2, we introduce the value function W i : IR n i +σ i × IR n i +σ i × S +(n i +σ i ) → IR and the time derivative are as follows where w i, * is the worst-case disturbance, given by w i, * : We note that (18) holds when Σ i > 0 and θ i ∈ Θ i,0 , and the last term inẆ i is nonpositive, zero on the set Θ i and approaches −∞ asθ i approaches the boundary of the set Θ i,o , which guarantees the boundedness ofθ i , i = 1, 2.
Then (5) can be equivalently written as, i = 1, 2: This completes the identification design step.

Control design
In this section, we describe the controller design for the uncertain system under consideration. Note that, we ignored some terms in the cost function (5) in the identification step, since they are constant when y 1 , y 2 ,w 1 ,w 2 andý 2 are given. In the control design step, we will include such terms. Then, based on the cost function (5), the controller design is to guarantee that the following supremum is less than or equal to zero for all measurement waveforms, ,w 2 ) to be designed, which are constants in the identifier design step and are therefore neglected.
By equation (21), we observe that the cost function is expressed in term of the states of the estimator we derived, whose dynamics are driven by the measurement y 1 , y 2 ,w 1 ,w 2 , y 2 , the reference trajectory y d , the input u, and the worst-case estimate for the expanded state vectorξ 1 andξ 2 , which are signals we either measure or can construct. This is then a nonlinear H ∞ -optimal control problem under full information measurements. Sinceý 2 = y 1 in the adaptive system under consideration, we can equivalently deal with the following transformed variables instead of considering y 1 , y 2 ,w 1 ,w 2 , andý 2 as the maximizing variable, By the special structure of the system, we define v i,a = ζ i (y i − C ixi )w i,a , i = 1, 2, v a = v 1,a v 2,a , and we will attenuate disturbance v a , and cancel the disturbancew 1,b andw 2,b . In view of y 2 = ζ −1 2 e q a +2,q 1,a +2 v a +x 2,1 , we will treatx 2,1 as the virtual control input of subsystem S 1 , whereq a =q 1,a +q 2,a .
Then the optimal choice for the variable ξ i,c andξ i , i = 1, 2, are: ξ 1,c * = − 1 2 ς 1,r 1 +r 2 ⇐⇒ξ 1, * =ξ 1 − 1 2 ς 1,r 1 +r 2 ; which yields that the closed-loop system is dissipative with storage function U and supply rate with optimal choice forξ i , i = 1, 2: This completes the adaptive controller design step. We will discuss the robustness and tracking properties of the proposed adaptive control laws.

Main result
In this Section, we present the main result by stating two theorems.
For the adaptive control law, with the optimal choice of ξ i,c * , the closed-loop system dynamics areẊ where F, G and G M are smooth mapping of D × IR, D and D, respectively; and the initial condition X 0 ∈ D 0 : where Q : D × IR → IR is smooth and given by The closed-loop adaptive system possesses a strong stability property, which will be stated precisely in the following theorem.
Based on the dynamics of η 1,d , we have η 1,d is uniformly bounded. Sinceη 1 = η 1 − η 1,d is uniformly bounded, then η 1 is also uniformly bounded. Furthermore, there is a particular linear combination of the components of η 1 , denoted by η 1,L , η 1 = A 1, f η 1 + p 1,n 1 y 1 η 1,L = T 1,L η 1 which is strictly minimum phase and has relative degree 1 with respect to y 1 . Then the signal η 1,L has relative degree r 1 + 1 with respect to the input y 2 , and is uniformly bounded. The composite system of η 1 andx 1 with inputẁ 1 and y 2 and output η 1,L may serve as a reference system in the application of bounding Lemma [12].
Note Φ 1 = Φ 1,y + Φ 1,u and Φ 1,y is uniformly bounded. To prove Φ 1 is bounded, we need to prove Φ 1,u is uniformly bounded. Define the following equations to separate Φ 1,u into two part: We observe that the relative degree for each element of Φ 1,u s 1 is at least r 1 + 1 with respect to the input y 2 , and is the output of a stable linear system. Take η 1,L and y 2 as output and input of the reference system, we conclude Φ 1,u s 1 is uniformly bounded by bounding Lemma. Because the first row element ofx 1 − Φ 1θ1 is: we can conclude thatx 1,1 − λ 1,b1Ā1,212 0θ1 is uniformly bounded in view of the boundedness ofx 1 − Φ 1θ1 ,θ 1 , Φ 1,u s 1 , and η 1 . Since z 1,1 =x 1,1 − y d , and z 1,1 , y d are both uniformly bounded, we have thatx 1,1 is also uniformly bounded.
To derive the uniformly boundedness of the closed-loop system states, we separate the relative degree, r 1 , into two cases: r 1 = 1, and r 1 ≥ 2. First, we consider the case 1: r 1 = 1.
Taking x 1,1 and y 2 as the output and input of the reference system, we note that x 1,1 is strictly minimum phase and has relative degree r 1 with respect to input y 2 . Since the state x 1 can be viewed as stably filtered output signals of y 2 and y 1 , it is uniformly bounded. Since λ 1 is also some stably filtered signals of y 1 and y 2 , it is uniformly bounded. It further implies Φ is uniformly bounded. Then we can concludex 1 is uniformly bounded from the boundedness ofx 1 − Φ 1θ1 . This further implies that the inputsx 2,1 andξ 1 are uniformly bounded.
From equation (23c), we note that every element of Φ 1,u s k+1 has relative degree of at least r 1 − k + 1 with respect to y 2 , and is the output of a stable linear system. Since the boundedness ofx 11 , · · · ,x 1k , we conclude Φ 1,u s k+1 is uniformly bounded by Lemma 11 in [12], where the reference system has input y 2 and output y 1 .
Then, we have the boundedness ofx 1 k+1 . Thus, we can conclude the boundedness of Φ 1,u s i , Since the state x 1 can be viewed as stably filtered output signals of y 2 and y 1 , it is uniformly bounded. Also, η 1 , λ 1 are some stably filtered signals of y 2 and y 1 , they are uniformly bounded. It further implies Φ 1 is uniformly bounded. Then we can concludex 1 is uniformly bounded from the boundedness ofx 1 − Φ 1θ1 . This further implies that the control inpuť x 2,1 is uniformly bounded. Therefore, it follows T f = ∞ and the complete system states are uniformly bounded on [0, ∞).
then, we establish the second statement.
For the third statement, we consider the following inequality, By the first statement, we notice that Then, we have lim For the last statement, it's easy to establish by Section 4.
This complete the proof of the theorem.

Example
In this section, we present one example to illustrate the main results of this Chapter. The designs were carried out using MATLAB symbolic computation tools, and the closed-loop systems were simulated using SIMULINK.
For the adaptive control design, we set the desired disturbance attenuation level γ = 10. We select the true value of the parameters in subsystem S 1 and subsystem S 2 are zeros, and belong to the interval [−1, 1]. The projection function P 1 (θ 1 ) and P 2 (θ 2 ) are chosen as P 1 (θ 1 ) = 0.5(θ 2 1 + θ 2 2 ), P 2 (θ 2 ) = θ 2 3 . The reference trajectory, y d , is generated by the following linear systemẋ d,1 = −x d,2 ,ẋ d,2 = x d,1 − x d,2 + d, y d = x d,1 with zeros initial condition, where d is the command input signal. The objective is to achieve asymptotic tracking ofx 1 to the reference trajectory y d .

Conclusions
In this Chapter, we present the game-theoretical approach based adaptive control design for a special class of MIMO linear systems, which is composed of two sequentially interconnected SISO linear systems, S 1 and S 2 . We assume the subsystem under studied subject to noisy output measurements, unknown initial state conditions, linear unknown parametric uncertainties, measured and unmeasured additive exogenous disturbance input uncertainties. Our design objective is to address the asymptotical tracking, the transient response and robustness of the closed-loop system, which are the same as the objectives to motivate the study of the H ∞ -optimal control problem. In view of the similar solution between H ∞ optimal control design and zero sum differential game, we convert the original adaptive control design problem into a zero-sum game with soft constraints on the disturbance input uncertainties and the unknown initial state uncertainties, which incorporates the measures of transient response, disturbance attenuation, and asymptotic tracking into a single game-theoretic cost function and formulates the design problem as a nonlinear H ∞ control problem under imperfect state measurements. A game-theoretical approach, cost-to-come function analysis, is then applied to obtain the finite dimensional estimators of S 1 and S 2 independently, which is also converted the control design as an H ∞ control problem with full information measurements. The integrator backstepping methodology is finally applied on this full information measurements problem to obtain a suboptimal solution. The controller achieves the same result as [17], namely the total stability of the closed-loop system, the desired disturbance attenuation level, and asymptotic tracking of the reference trajectory when the disturbance is of finite energy and uniformly bounded. In addition, the proposed controller may achieve arbitrary positive disturbance attenuation level with respect to the measured disturbances by proper scaling. The contribution of the measurements of part of the disturbance inputs is that we can design an adaptive controller with disturbance feedforward structure with respect tow 1,b andw 2,b to eliminate their effect on the squared L 2 norm of the tracking error. Moreover, the asymptotic tracking is achieved even if the measured disturbances are only uniformly bounded without requiring them to be of finite energy.  (11): [1971][1972][1973][1974][1975][1976][1977][1978][1979][1980][1981][1982][1983][1984][1985][1986][1987][1988][1989][1990].