Self-Healing Control Framework Against Actuator Fault of Single-Rotor Unmanned Helicopters

Unmanned helicopters (UHs) develop quickly because of their ability to hover and low speed flight. Facing different work conditions, UHs require the ability to safely operate under both external environment constraints, such as obstacles, and their own dynamic limits, especially after faults occurrence. To guarantee the postfault UH system safety and maximum ability, a self‐healing control (SHC) framework is presented in this chapter which is composed of fault detection and diagnosis (FDD), fault‐tolerant control (FTC), trajectory (re‐)planning, and evaluation strategy. More specifically, actuator faults and saturation constraints are considered at the same time. Because of the existence of actuator constraints, usable actuator efficiency would be reduced after actuator fault occurrence. Thus, the performance of the postfault UH system should be evaluated to judge whether the original trajectory and reference is reachable, and the SHC would plan a new trajectory to guarantee the safety of the postfault system under environment constraints. At last, the effectiveness of proposed SHC framework is illustrated by numerical simulations.


Introduction
Safe operation and reliability are important performances of unmanned aerial vehicles (UAVs). Unmanned helicopters (UHs) as a kind of UAVs develop quickly because of their ability to hover and low speed flight. Nevertheless, traditional control and planning strategies cannot guarantee their safety and reliability in the face of malfunctions such as sensor faults, actuator faults, and component faults. Furthermore, taking physical limits such as actuator constraints and state constraints into account, the situation would be more serious. In order to ensure the reliability and safety of UHs, fault detection and diagnosis (FDD) and faulttolerant control (FTC) approaches become focus of research and many related research results have been presented [1]. However, physical limits are not well considered in conventional FTC approaches which may affect their efficiency. On the other hand, besides UH's dynamic limits, external environment constraints, such as obstacles, also affect UH's safety. However, traditional planning methods cannot consider environment constraints and vehicles' limits at the same time which makes these methods helpless in the face of faults. In this chapter, a self-healing control (SHC) framework is proposed against actuator faults and constraints of single-rotor UHs under external environment constraints. The SHC framework is shown in Figure 1 which involves FDD module, reconfigurable controller, trajectory (re-)planning module, and evaluator module. The tasks of the above modules are as follows: 1. FDD module: Estimate the actuator healthy coefficient (AHC) [2] in real time. AHCs can indicate healthy conditions of actuators and actuator fault information is the basis of controller reconfiguration.

2.
Reconfigurable controller: This module realizes the same functions as conventional FTC approaches and considers actuator constraints at the same time. After fault occurrence, the fault-free controller will be configured as a postfault controller according to AHCs provided by FDD module to guarantee the stability of the postfault UH system.

3.
Trajectory (re-)planning module: Compute realizable trajectory based on desired path and UH dynamic model under environment constraints. This module calculates a feasible trajectory under both fault-free and postfault conditions. Furthermore, controller references are computed directly in this module which can be used by reconfigurable controller.

4.
Evaluator module: Evaluate system performance of UHs according to AHCs, UH dynamic model, and controller and trajectory information. If the original controller and trajectory are not feasible, this module will ask related modules to reconfigure controller and replan trajectory.
The remaining part of this chapter is organized as follows: single-rotor UH and AHC models are simply introduced in Section 2; Section 3 investigates FDD approach against actuator faults based on extended Kalman filter (EKF) and linear neural network (LNN). In Section 4, the postfault controller is designed to guarantee the stability of UH system under both actuator faults and constraints; Section 5 presents invariant-set based planning (ISBP) approach that can compute controller reference according to desired path and UH dynamic model. Evaluation strategy is introduced in Section 6 and numerical simulations are shown in Section 7. Finally, Section 8 summarizes the chapter with conclusions.

UH model
The 6-DOF dynamics of UH is given by the following Newton-Euler equations: where V b = u v w is the velocity vector, ω b = p q r is the angular velocity vector, F b is the aerodynamic force vector, F g is the gravity force vector, I b is the moment of inertia matrix, M b is the aerodynamic moment vector, and m is the UH mass.
Aerodynamic forces and moments can be calculated by where δ lon , δ lat , δ col , and δ ped are the longitude, latitude, main rotor collective pitch, and tail rotor collective pitch control inputs, respectively. ω r is control input of main rotor rotating speed.
Typically, f f and f m are nonlinear functions and their details can be found in [3,4].
Note that δ lon , δ lat , δ col are the nominal control inputs and they do not represent the actual actuator outputs. The relationship between the nominal and actual inputs is determined by the structure of swashplate [5] and their values can be converted to each other.
In order to guarantee the accuracy of AHC estimation, nonlinear model is used to design FDD approach. However, the above nonlinear model is too complex to design EKF filter so that a simplified nonlinear discrete-time model is used [6]: where x ∈ R n is the system state vector, y ∈ R p is the system output vector, u ∈ R m is the system control input vector, ω and ν are the process noise and measurement noise that are assumed to be Gaussian white noise with zero mean and uncorrelated from each other. Considering swashplate configuration [5], the system control input vector is u = θ l , θ r , θ b , θ t , ω r where θ l , θ r , θ b are left, right, back swashplate actuator outputs, and θ t = δ ped is the tail actuator output.
On the other hand, the linear dynamic model is used to design controller and trajectory planning which is shown as follows:

AHC model
Suppose v is the control signal given by controller called control variable and u is the actual actuator output called manipulated variable, their relationship is represented by where Λ is a diagonal matrix with Λ = diag(λ 1 , ..., λ m ), λ i ∈ 0, 1 , and ū ∈ R m is a constant vector. In addition, λ i and ū i are the proportional effectiveness and fault bias of the ith actuator's AHC. λ i = 1 and ū i = 0 represent that the ith actuator is fault-free; otherwise, the ith actuator has fault. Thus, the fault condition of actuators can be represented by AHCs.

EKF-and LNN-based FDD approach
Because of the serious nonlinear coupling between all control channels, we assume that an actuator has fault whereas others are still well, and the actuator for rotate speed control, ω r , is always well. Here, the multiplicative fault of an actuator as a parameter that is indicated by the AHC for this kind of fault is more likely to happen. The following will introduce the realization of fault diagnosis in the order of fault identification module, fault isolation module, and fault detection module. In this section, x(k ) is written as x k for simplicity and x k |k −1 represents the pre-estimation value of x k based on the information of k − 1 step.

Fault identification module
The task of fault identification is to determine the size of the fault and the fault time after fault isolation. The size of a fault is indicated by the proportional effectiveness of AHC, so we should estimate the actual manipulated variable vector u according to state estimations from EKF and actual control signal v to calculate and obtain the proportional effectiveness.
As shown in Eq. (1), in a typical nonlinear discrete-time system, the input vector is obtained from actuators, and the current state vector x k is calculated based on the state vector of the last step. The estimations of x k |k and x k −1|k −1 can be obtained by EKF [7]. Therefore, the state equation can be transformed into the equation as follows: For the calculation, processes of forces and torques of the simplified nonlinear model have been linearized, Eq. Typically, Eq. (3) is an overdetermined linear equation set and could not be solved directly [8].
After knowing which actuator is faulty, many output estimations of this actuator can be obtained by the faultless manipulated variables of other actuators according to Eq. (3). Multiply the sample column vectors formed by these estimations by a weight matrix and correct this result with an offset. Then, the manipulated variable estimation of this faulty actuator can be obtained.
To obtain the above weight matrices and offsets, consider a linear neuron as a simple LNN, and its input-output relation is expressed as follows: where a is the output vector, p is the input sample vector, W is the weight matrix, and b is the offset. The weight matrices and offsets can be obtained by training. Assume that an actuator has a fault, and let the output estimations of this actuator be a set of input of the linear neuron. After simulation, multiple sets of input samples can be obtained, and let the actual manipulated variable be the target value to train the weight matrix and offset of this actuator. Determine the weight matrix and offset of every faulty actuator, and make use of them in the online experience to calculate the estimation of every manipulated variable. The above process is closely related to the estimation process of EKF; more certainly, EKF and LNN complete the joint estimation of system states and AHCs together.

Fault isolation module
The task of fault isolation is to determine the location and type of the fault after fault detection. The structure of this module is shown in Figure 2 to record the completion time and pass the serial number of the faulty actuator to fault identification module.
Four fault identification modules are in parallel in this module, and we assume that there are different actuator faults in different identification modules. Therefore, every fault identification module can obtain the proportional effectiveness of AHC called isolation AHC. Make use of the isolation AHC of the ith actuator and let these values of other actuators be both 1 to calculate the manipulated variables called isolation-manipulated variables according to control variable vector v. Based on these variables, we can obtain the system state vector and define it as x iso . In this way, we define the isolation residual of the ith actuator as follows: where x est is state estimation. If the isolation residual of an actuator is less than the isolation residuals of others, there will be fault in this actuator. For example, if res i is less than other isolation residuals, the ith actuator can be considered to be faulty.

Fault detection module
The basis of the above two modules is that the fault is detected quickly and accurately. Known from the EKF process, the filter obtains the state vector pre-estimation x k |k −1 by the estimations of the last step and updates x k |k −1 by the measurement information to obtain the current estimation of the state vector x k |k . If there is a fault in the actuator, there will be difference between the pre-estimations of states and the actual states with fault. However, the estimations of states would track the actual states after the update. Define a filter residual as follows: Perform the weighted sum of squared residuals (WSSR) operation [9]:

Reconfigurable controller design
In this section, AHC-based antiwindup controller design method is introduced which can be achieved by solving a set of linear matrix inequalities (LMIs). At the same time, the related invariant-set-based safety region is also calculated. The definition of invariant set and the controller design under actuator constraints are given as follows.

Considering a linear system
Ax k + = the (positively) invariant set can be given by the following definition.

Definition 1 [10]:
The set S ⊂ R n is said invariant for the above system if for all x(0) ∈ S the solution x(k) ∈ S. If x(0) ∈ S implies x(k) ∈ S for k > 0, then we say that S is positively invariant.
Based on Definition 1, the definition of robustly controlled invariant set for the following system is provided: The set S ⊂ R n is said robustly controlled invariant for the above system if there exists a continuous feedback control law u(k ) = Kx(k ), which ensures the existence and uniqueness of the solution on R + such that S is positively invariant for the closed loop system.
An invariant set S can be constructed by the following theorem.
For convenience, ρ is chosen equal to 1 and simplify S ρ as S = {V (x(k )) ≤ 1}. Hence, the construction of S finds a function V , which implies ΔV (x(k)) ≤ 0 when V (x(k)) ≤ 1. Obviously, if we assume V (x(k)) ≥ 0, V would be a candidate of the Lyapunov function. More specially, if the Lyapunov candidate function is in quadratic form such that V (x(k )) = x T (k)Px(k), the invariant set will be an ellipsoid and this nature will be used later.

Controller design under actuator constraints
Consider linear discrete UH model, Eq. (2), with actuator constraints: Where σ(u h (k )) represents constrained control inputs such that − 1 ≤ u(k ) ≤ 1 and the saturation feature can be defined by σ(u i (k)) = sgn(u i (k))min{1, | u i (k) | }, where sgn( ⋅ ) is a sign function and i = 1, ..., m. Then, considering AHC model, the above system can be rewritten as In the following discussion, assume the number of λ i = 0 is m f , which implies m f actuators cannot respond to the control signal and m 0 = m − m f is the number of fault-free/partial-fault actuators that can respond to the control signal. In other words, m 0 represents the number of nonzero columns of matrix B h 0 . In this way, the postfault system with different actuator faults can be represented.
In order to achieve set-point tracking offset free and compensate actuator saturation, an integrator with a saturation compensator is introduced as follows: where e(k) is the integrator state vector, E c is the pending compensator matrix, Based on linear quadratic regulator (LQR) theory, state-feedback controller v(k) = Kx(k ) can be calculated. Thus, the closed-loop postfault system can be expressed by . In order to guarantee the closed-loop system stability and to track offsetfree under external input ω(k) and actuator constraints, the following theorem is proposed. As a basis, first a lemma is given which defines a set of system states related to actuator saturation.

Lemma 1 ([12]):
Define the following polyhedral set with matrix G ∈ R m 0 ×(n+ p) : where K i and G i represent the ith row of matrices K and G. If x(k) ∈ ℰ, then the relation Based on this lemma, saturation compensator can be designed in the following theorem.
Theorem 2: Given LQR parameter matrices Q u , Q x , if there exist a symmetric positive-definite  . The proof can be found in [13].
According to Theorem 2, the safety region S is an invariant set which means that if initial states and steady states of a system are inside the set, the state trajectory from initial states to steady states will also be inside the set. In this way, a postfault system with a fault-tolerant controller can be seen as a new closed-loop system with the initial state x(k f ), where k f is instant when the fault is detected Clearly, a postfault system will be safe if initial state x(k f ) is inside the safety region S.
On the other hand, the related admissible set of reference Ω ref is also achieved by Theorem 2 and it has a closed relationship with the controller. In the next section, the controller design and the reference design will be composed together.

ISBP approach
This section proposes an ISBP approach that can plan a feasible trajectory based on the desired path nodes under both external environment constraints and their own dynamic limits [14]. In order to consider environment constraints and dynamic constraints at the same time, invariant set is used as a bridge to connect the two kinds of constraints because the invariant set is calculated based on the Lyapunov function that is linked to dynamic model, and on the other hand, the invariant set has certain geometrical shapes, such as ellipsoid, so that it can easily be represented in work space with obstacles.
Note that, in this section, we have assumed that the heading of UH is kept to be 0, pitch angle and roll angle of UH are kept small so that the position of UH in the world coordinate system can be considered as the integration of velocities in the body coordinate system.

Reachable set
Considering System (4) with controller v(k) = Kx(k), the definition of reachable set is given.

ISBP method
For the convenience of discussion, a 2D work space is used in the following whereas the proposed ISBP approach can be expanded in 3D condition directly. Consider a preknown environment as shown in Figure 3, where the initial location is L 0 and the goal location is L 5 . Based on the preknown environment, a lot of path planning approaches can be used to find a feasible path against external environment constraints such as obstacles. Suppose this path is represented by a series of nodes such as L 0 ∼ L 5 , as shown in Figure 3.
The target of the proposed ISBP approach is to calculate feasible controller reference inputs to move the UH from the initial location to the goal location based on the feasible path and satisfy both external environment constraints and UH's limits. Consider a simple example in the preknown environment as shown in Figure 4(a) where the initial location is L 0 and the goal location is L 1 . Suppose a feasible path is represented by two nodes L 0 and L 1 and their connection is shown in the same figure.
Clearly, the work space is state space; thus, the location in work space is equality to the state such as the initial state, which is equal to L0. Thus, an invariant set S can be constructed according to plant dynamic model (4) under controller v(k) = Kx(k ), which also guarantees obstacle avoidance. Suppose the invariant set is S 0 and the related reachable set is S r 0 as shown in Figure 4(b). Clearly, because of the existence of constraints such as obstacles and UH's limits, the invariant set S 0 may not contain the goal location L 1 as shown in Figure 4(b). Closely, the goal location L 1 is also outside the reachable set S r 0 , which means L 1 is unreachable under this condition. Thus, the system states should be continued moving.
Suppose the intersection of reachable set S r 0 and feasible path L 0 L 1 is c 1 as shown in Figure 4(b). In order to move state from c 1 to L 1 , a new invariant set, whose center is c 1 , is required.
Suppose the state of the new system is , where x c1 is a constant vector whose position is equal to c 1 and other elements are 0. Based on the new system, we can construct the second invariant set S 1 whose center is c 1 as shown in Figure 4(c). The related reachable set S r 1 is also shown. Obviously, the goal location L 1 is inside the reachable set S r 1 now, which means L 1 is reachable under this condition. In this way, we can choose the center of S 1 , c 1 , and the goal location L 1 as a sequence of the controller reference input. Finally, Figure 4(d) shows the practical trajectory that is obtained by the UH actual operation.

Calculation of the invariant set and the reachable set
According to Theorem  should not intersect with all of obstacle spheres. Suppose S 0 ⊂ R n+ p is a sphere that is defined , d min is the distance between the center of the nearest obstacle and the current system equilibrium point.
According to the above analysis, the following theorem is proposed to calculate the maximum invariant set S and the reachable set S r of system (4) under both external environment constraints and UH's limits.
Theorem 3: Given η > 0, LQR parameter matrices Q u , Q x , symmetric positive-definite matrices W 0 and R, if there exist a symmetric positive-definite matrix W ∈ R (n+ p)×(n+ p) , matrices , Z ∈ R p×m 0 , diagonal positive-definite matrices S ∈ R m 0 ×m 0 , W r ∈ R p× p , and a positive scale λ satisfying , , , , , The relationship between the related sets and the obstacles are shown in Figure 5 assuming that all sets are projected to a 2D plane. In other words, based on the calculated invariant set S and the reachable set S r , a new center c 1 can be computed which is the intersection point of S r and the feasible path. The next step is to calculate the second invariant set and the reachable set whose center is c 1 until the last reachable set covers the final path point.

Evaluation strategy
The open-loop model (2) of UH is unstable and the actuator outputs are limited. Thus, the only regional stability of UH can be guaranteed and the stability region is determined by the system structure and actuator constraints [16]. In other words, if the states of UH are inside the stability region, the UH would be of safety; otherwise, the UH may be in danger. Obviously, after actuator fault occurrence, actuator efficiency will be reduced. Hence, the safety region of the postfault system will be different with the fault-free case as shown in Figure 6(a). Suppose that the safety region of the fault-free system with a fault-free controller is Ω ff and the postfault system with a FTC controller is Ω pf as shown in Figure 6(b); furthermore, the initial state of the fault-free system is x 0 ∈ Ω ff and an actuator fault is detected at k f 1 . Clearly, the state x(k f 1 ) is outside the safety region of the postfault system Ω pf so that the postfault system may be unstable (as shown by state trajectory x(k f 1 )x ′ (k f 1 )) at last. That is to say, the actuator fault cannot be compensated. In contrast, suppose that the other fault is detected at k f 2 and x(k f 2 ) ∈ Ω pf is valid, which represents that the fault can be compensated. However, the states in steady case are also determined by references such as x f 1 which may be outside the safety region Ω pf . Obviously, this reference is unreachable and tracking such reference may lead UH unsafe (as shown by the state trajectory x(k f 2 )x ′ f 1 ). Hence, reference reachability should be analyzed before system motion and a new reachable reference is necessary. Compared with x f 1 , x f 2 may be more reasonable which is inside Ω pf . According to Theorem 2, the safety region S is an invariant set. Clearly, a postfault system will be safe if initial states are inside the safety region S f of the postfault system. In this way, the initial states of the postfault system can be evaluated. Second, the steady states will be analyzed. In steady-state case, actuators are not expected to be saturated so that the remaining efficiency of actuators can be used for disturbance defence. Hence, the original reference should be inside the reachable set of the postfault system such as ref ∈ S rf , where S rf is the reachable set of the postfault system; otherwise, the original reference ref is not reachable and tracking the original one may lead UH unsafe. The reason is that the actuator efficiency is reduced in the postfault system and tracking unreachable reference will lead fault-free actuator saturated which implies that UH cannot respond to control signal correctly. Under this condition, a new optimal reference is required which can be calculated by the trajectory replanning approach. In other words, if the original reference is not reachable after detecting the actuator fault, the ISBP approach should be called to calculate new trajectory and controller reference based on the postfault dynamic model of UH.

Numerical simulation
The parameters of nonlinear UH dynamic model used in numerical simulations is obtained from reference [6]. This model is used as the simulation model in this section. The step size of this simulation is 0.02s, which is the same as the control period of the real UH flight platform.

FDD simulation
Based on the simplified nonlinear model, the EKF is designed and LNN is used to train weight matrices and offsets of faulty actuators. The parameters of the EKF filter and the training results of the left swashplate actuator by LNN are as follows: Q = diag(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), (40, 40, 40, 1, 1, 1, 1, 1, 1, 1, 1, 1 where Q and R are covariance matrices of the process noise and the measurement noise, respectively. Then, we introduce a multiplicative fault on the left swashplate actuator as follows:   After fault detection, isolation residuals are calculated to isolate this fault. The curves of four isolation residuals are shown in Figure 8. According to this figure, the isolation residual of the left swashplate actuator is less than other residuals. That is to say, the left actuator of swashplate is faulty. The fault isolation module cannot provide the right isolation results as soon as the completion of fault detection because of the vibration caused by the measurement noise. There may be misdiagnosis of the fault location within a very short time. The curve of the proportional effectiveness of the left actuator under the fault is shown in Figure 9. After fault isolation, the isolation module passes the serial number of the faulty actuator to fault identification module to identify the AHC of this faulty actuator, and the simulation results indicate the size of this multiplicative fault. Because the weight matrices and offsets are trained by the LNN under the faults whose AHCs' parameters are all 0.5 for the online simulations, there may be error between the identification results and the actual size of this fault. By the calculation of average, the fault identification result is equal to the actual fault approximately.
The simulation results show that fault detection and isolation modules could detect and isolate a fault quickly, and the identification result is accurate enough; moreover, this method is with good real-time performance.

Controller and trajectory (re-)planning simulation
A linear UH model (2) with added three-axis positions is used in this part and related parameters can be found in [13]. The added positions are considered integration of velocities in the body coordinate. Thus, the system states, control inputs, and outputs are The preknown work space is shown in Figure 10 where the start point is (0,0,0) and the target point is (20,20,20). The external environment constraint considered here is a sphere obstacle whose position is (10,10,10) and radius is 1. The feasible path is given by path nodes, such as (0,0,0), (8,8,13), and (20,20,20), which can guide UH from the start point to the target point with obstacle avoidance. According to Theorem 3, a series of controllers K , invariant sets S, and reachable sets S r satisfying obstacle avoidance and UH dynamic limits can be computed as shown in Figure 11 where the green ellipsoids are the reachable sets in 3D and the centers of ellipsoids, blue stars, are controller references. According to these references, the UH can run from the start point to the target point and the actual trajectory is shown by the blue curve. Clearly, the trajectory is kept inside the green ellipsoids so that it is also inside the invariant sets. Compared with fault-free case as shown in Figure 11, Figure 12 shows the results of the postfault case without the SHC framework. The actuator fault is detected at 6.4s and the related state is marked by red points in the figure. According to the postfault dynamic model, a new reachable set, red ellipsoid, is calculated. It is easy to see that the original reference, black point, is outside the reachable set of the postfault system which implies that the original reference is unreachable. Under this condition, if it does nothing, the UH system may be in danger as shown by the blue curve. The related manipulated variables are shown in Figure 13 where the blue curves are the actuator outputs and the red dashed lines are the actuator constraints. Clearly, the actuator outputs are saturated at last which leads the UH system out of order.    Figure 14 shows the results of the postfault system with a SHC framework. After fault detection, Theorem 3 is recalled to calculate the new controllers, invariant sets, and reachable sets to evaluate the performance and guarantee the postfault UH system to be stable. At last, the UH can reach the target point with obstacle avoidance as shown by the actual trajectory.

Conclusions
In this chapter, a self-healing control framework is proposed for UH systems. The SHC framework aims at providing a solution to guarantee UHs safety and maximum ability to achieve the desired missions under both fault-free and postfault conditions. The EKF-and LNN-based FDD approach is used to detect and diagnosis actuator faults modeled by AHCs. Then, the AHC-based reconfigurable controller design method is proposed to calculate the fault-tolerant controller and the related safety region against both actuator faults and constraints by solving a set of LMIs. Third, the ISBP approach is presented for planning a feasible trajectory and computing the related controller reference under both external environment constraints and UH dynamic limits at the same time. After fault occurrence, based on the calculated safety region and controller reference, the performance of the postfault UH system can be evaluated, which can provide information whether the fault can be compensated and the original reference can be reached. If the original reference is not reachable, the ISBP approach will be recalled to calculate the new trajectory and reference again according to the postfault dynamic model. Finally, numerical simulations illustrate the effectiveness of the proposed SHC framework.