Hypersonic Vehicles Profile-Following Based on LQR Design Using Time-Varying Weighting Matrices

In the process of applying linear quadratic regulator (LQR) to solve aerial vehicle reentry reference trajectory guidance, to obtain better profile-following performance, the parameters of the aerial vehicle system can be used to calculate weighting matrices according to the Bryson principle. However, the traditional method is not applicable to various disturbances in hypersonic vehicles (HSV) which have particular dynamic characteristics. By calculating the weighting matrices constructed based on Bryson principle using time-varying parameters, a novel time-varying LQR design method is proposed to deal with the various disturbances in HSV reentry profile-following. Different from the previous approaches, the current states of the flight system are employed to calculate the parameters in weighting matrices. Simulation results are given to demonstrate that using the proposed approach in this chapter, performance of HSV profile-following can be improved significantly, and stronger robustness against different disturbances can be obtained.


Introduction
Hypersonic vehicles (HSV) possess great meaning for aerospace applications, and have important potential values in various fields [1][2][3][4]. Reentry guidance of HSV is a critical technology for assuring the vehicle's arrival at a desired destination. The reentry guidance concepts can be described in detail under two general categories [5]; that is, one uses predictive capabilities, and the principal disadvantage is the stringent onboard computer requirements for the fasttime computation; the other one uses a reference trajectory, which provides a simple onboard guidance computation and high reliability of guidance accuracy. The latter has a strong engineering and application value, and has been employed in Apollo entry guidance [6] and shuttle entry guidance [7]. Nevertheless, in reference trajectory reentry guidance, since HSV owns particular characteristics such as strong nonlinear, large flight envelope, complex entry environment, and precise terminal guidance accuracy requirement, it is difficult for HSV to track the nominal profile properly. Many scholars have done continuous research on it [8][9][10][11][12].
For following nominal profile problem in reentry reference trajectory guidance, namely trajectory tracking law, traditional PID control law of shuttle was given in [7]. Roenneke et al. [13] derived a linear control law tracking the drag reference in drag state space to achieve guidance command. In [14], a feedback linearization method for shuttle entry guidance trajectory tracking law to extend its application range was presented. In [15,16], a feedback tracking law is designed by taking advantage of the linear structure of system dynamics in the energy space to achieve bounded tracking of the flat outputs. These foregoing approaches improved performances of trajectory tracking laws for aerial vehicles. However, they are not applied to HSV which owns particular characteristics. Dukeman [17] proposed a linear trajectory tracking law based on linear quadratic regulator (LQR) by constructing weighting matrices with Bryson principle [18], and the tracking law was very robust with respect to varying initial conditions and worked satisfactorily even for entries from widely different orbits than that of the reference profile. The capacity of the approach in [17] against other reentry process interferences such as aerodynamic parameter error, nevertheless, was relatively poor, and simulations demonstrated that performances for HSV tracking nominal profile under various disturbances depended directly on weighting matrices in LQR. In this study, one focuses on constructing LQR weighting matrices to strengthen robustness of HSV trajectory tracking law.
The weighting matrices Q and R are the most important parameters in LQR optimization and determine the output performances of systems [19]. Trial-and-error method has been employed to construct these matrices, which is simple, but primarily depends on people's experience and intuitive adjustment. In trial-and-error method, elements of weighting matrices must be repeatedly experimented to get a proper value, and is not feasible for application in large scale system. In [18,20], certain general guidelines were followed to construct weighting matrices simply and normally, but might not lead to satisfactory responses. Connecting closedloop poles to feedback gains for LQR were presented in [21][22][23][24] using pole-assignment approach, which resulted in more accurate responses. However, it was difficult to balance state and control variables and to account for control effectiveness using the approaches in [21][22][23][24]. A trade-off between penalties on the state and control inputs for optimization of the cost function was considered in [25], where specified closed-loop eigenvalues were obtained, but the computation normally needed more iterations. Genetic algorithm (GA) can be applied to find a global optimal solution [26][27][28], and the differential evolution algorithms inspired from GA are efficient evolution strategies for fast optimization technique [29][30][31]. However, the approaches in [26][27][28][29][30][31] have little improvement for HSV profile-following performances under different disturbances.
In this study, a novel method to construct weighting matrices with time-varying parameters on the basis of Bryson principle is proposed. This idea employs current flight states to provide flexible and accurate feedback gains in HSV trajectory tracking law under various interferences and errors. Simulations indicate that this approach effectively improves profile-following performance and strengthens the robustness of LQR under different internal and external disturbances.
The rest of this chapter is organized as follows. Section 2 introduces reentry dynamics, LQR, Bryson principle, and their applications in hypersonic vehicle trajectory tracking law. Section 3 analyzes the problem of hypersonic vehicles profile-following. The novel LQR design method with time-varying weighting matrices is presented in Section 4. A numerical simulation is given in Section 5. Finally, Section 6 concludes the whole work.

Preliminaries
In this section, the concepts and basic results on reentry dynamics, LQR, Bryson principle, and their applications in hypersonic vehicle trajectory tracking law are introduced, which are the research foundation of the following sections.

Reentry dynamics
For a lifting reentry vehicle, the common control variables are the bank angle σ, and the angle of attack α. The state variables include the radial distance from the Earth center to the vehicle r, the longitude θ, the latitude ϕ, the Earth-relative velocity v, the flight path angle γ, and the heading angle ψ. The three-degree-of-freedom point-mass dynamics for the vehicle over a sphere rotating Earth are expressed as [32]: where ω is the Earth's self-rotation rate, and g is the gravitational acceleration. L and D are the aerodynamic lift and drag accelerations defined by where m is the mass of the vehicle, S is the reference area, C L is the lift coefficient, and C D is the drag coefficient. ρ is the atmospheric density expressed as an exponential model [33].
where ρ 0 is the atmospheric density at sea level, h is the altitude of the vehicle, and β is a constant.
To guide the vehicle from the initial point to the terminal interface with multiple constraints, a reference trajectory is usually optimized offline, and a profile-following law is utilized to track the reference trajectory onboard. In the longitudinal profile-following, the linear quadratic regulator (LQR) law is a good choice [10].

Linear quadratic regulator
This subsection introduces LQR and Bryson principle. For a linear system, the dynamics can be described by where A(t), B(t) and C(t) are system matrices, The quadratic performance index required to be minimized can be written as where the weighting matrix Q(t) is symmetrical positive semi-definite and weighting matrix R(t) is symmetrical positive definite. The specific procedure of LQR minimizing quadratic performance index is as follows.
The Riccati equation is given as After getting the solution P(t) corresponding to each time instant t by solving Eq. (11), the feedback gain matrix can be obtained as Based on Eq. (12), the control input can be designed as In order to obtain a proper quadratic performance index, elements of weighting matrices must be chosen properly, and Bryson principle can solve this problem effectively.
The basic principle of Bryson principle is to normalize the contributions, and then the states and the control terms may behave effectively within the definition of the quadratic cost function. The normalization is accomplished by using the anticipated maximum values of the individual states and control quantities. The method can be explained as follows.
First, define the weighting matrices Q(t) and R(t) as diagonal matrices, namely: Then, develop the quadratic index in the following expression.
Determine each maximum value of all the states and control terms.
Normalize all the contributions to 1 with the help of all the maximum values.
Then, the elements to construct the weighting matrices can be obtained as time-invariant parameters.
Through simple calculation, Bryson principle can generate better results in a short time, which minimizes the quadratic index value in a proper scope. Because of that, Bryson principle is widely applied to the selection of weighting matrices in LQR.

Reentry reference trajectory guidance based on LQR
In reentry reference trajectory guidance, the chief challenge of following the nominal profile lies in generating a proper compensatory signal, and LQR can solve this problem effectively. After generating a feasible reference entry profile containing altitude z, velocity v, flight path angle γ as reference parameters, and bank angle σ ref as guidance reference signal, one can download this profile into the onboard computer. After the vehicle enters the atmosphere, the deviations between nominal profile and the actual real-time data can be obtained by navigation facilities. The deviations contain altitude error z δ , velocity error v δ and flight path angle error γ δ .
Denote the compensatory bank angle by σ δ . In order to minimize the deviations and keep the aerial vehicle tracking nominal profile properly, the optimal feedback gain of guidance compensatory signal σ δ can be calculated by LQR in the following algorithm.
Algorithm 1. The actual guidance signal in trajectory tracking law based on LQR can be determined in the following procedure.
Step 1. Based on perturbation theory, one establishes the linear equations of motion by taking the deviations as state parameters.
Step 2. Construct the quadratic performance index as follows: Step 3. The weighting matrices Q and R in Eq. (20) are determined by Bryson principle. Since altitude z and velocity v are main factors in profile-following, q 3 , which is the weighting element of path angle γ, can be ignored. Using Eq. (18), the other elements of weighting matrices can be obtained as where z δmax and v δmax are anticipated maximum deviations between the actual profile and the nominal profile, and σ δmax is the maximum allowable modification of guidance signal σ. Based on Eq. (21), one can get the weighting matrices: Step 4. In order to minimize the index J in Eq. (20), one calculates the Riccati Eqs. (11) and (12) to obtain the optimal feedback gain K(t). Then, the compensatory signal can be obtained as Step 5. The actual guidance signal σ(t) which consists of guidance reference signal u(t) and guidance compensatory signal δu(t) can be obtained. It can be shown that It has been verified in [17,34] that using Algorithm 1, the aerial vehicle performs well in tracking reference trajectory.

Problem statement
In this section, the problem of HSV profile-following using trajectory tracking law based on LQR in Section 2 is presented.
The traditional reference guidance is not suitable for hypersonic vehicles because of its particular characteristics, including strong nonlinear, large flight envelope and complex entry environment. In the process of entry flight, it is difficult to constrain the deviation between real profile and nominal profile into a proper scope. Furthermore, strict terminal accuracy requirement demands that hypersonic vehicles track nominal profile precisely, that is, deviations in the terminal stage must be smaller.
Consequently, in the reference profile-following of HSV based on LQR, new problems occur in the selection of weighting matrices. In the initial flight stage, it is assumed that the deviations of altitude and velocity are z δ0 and v δ0 , respectively, which are chosen to be Let z δ1 and v δ1 be the anticipated maximum deviations accuracy in the terminal stage, expressed as Substituting z δ0 , v δ0 , z δ1 and v δ1 into Eq. (21), one can get the weighting matrix Q 0 and Q 1 as From Eqs. (25) and (26), one sees that z δ0 and v δ0 are bigger than z δ1 and v δ1 , respectively. The weighting matrix Q 0 , which is determined by z δ0 and v δ0 , can effectively eliminate the large initial stage deviations between the real and nominal profiles. Nevertheless, the capacity of Q 0 for resisting disturbance in the process of flight is not strong enough to satisfy the terminal accuracy requirement. On the contrary, the weighting matrix Q 1 constructed by z δ1 and v δ1 can eliminate the disturbance in the process of flight effectively. However, facing the existence of large deviations in the initial entry flight, it is difficult to keep HSV tracking the nominal profile properly, which will further influence the terminal accuracy.
Therefore, compared with Q 0 , the weighting matrix Q 1 is not applicable to initial deviations, and has good robustness to deal with disturbance in the process of entry flight. The   simulations for hypersonic vehicles profile-following with Q 0 and Q 1 under different disturbances are shown in Figures 1-3. The flight profiles are expressed in altitude and velocity plane.
The reference profile described in this study is similar to the shuttle entry reference profile. The initial altitude of simulation is 55 km, and the initial velocity is 6 km/s. Figure 1 shows that hypersonic vehicle tracks nominal profile with initial altitude deviation of positive 3 km, where circle line, solid line, dashed line indicate nominal profile, actual profile with Q 0 , actual profile with Q 1 , respectively. From Figure 1, one sees that the performance of Q 0 tracking nominal profile is better than Q 1 . Figures 2 and 3 are in respect to HSV profile-following with initial deviation of path angle and process disturbance of positive 20% aerodynamic parameter error. It can be seen that the performance of Q 0 tracking nominal profile is better than Q 1 in Figure 2, and Q 1 is better than Q 0 in Figure 3. Based on Figures 1-3, it can be obtained that the LQR with weighting matrices constructed by Bryson principle hasn't strong robustness to different disturbances in HSV profile-following.
In order to solve above problem, it is required that LQR cannot only minimize the initial deviations, but also enhance the capability that resists the process disturbance effectually. Therefore, it is necessary to develop an algorithm to determine a proper weighting matrix in LQR. With the help of Bryson principle, an approach to determine the weighting matrix in LQR with current flight states is proposed in the following section.

LQR with time-varying weighting matrices
In this section, first, the flow chart of HSV profile-following is presented. Then, LQR design method using time-varying weighting matrix for HSV reentry trajectory tracking law is derived.
Based on LQR, here is the flow chart of HSV tracking reference profile shown as the solid lines in Figure 4.
The work flow of HSV profile-following is explained as follows: Comparing the actual flight profile with the reference profile, one can get the state deviations containing z δ and v δ . With these deviations, the compensatory signal u δ can be calculated by multiplying feedback gain K. Then one can input the compensatory signal and the reference guidance signal into HSV guidance loop. In this way, actual flight profile of the next step is obtained. The calculation of the feedback gain K by LQR involves four matrices. As shown in the Figure 4, the construction of system matrices A and B needs actual state parameters. Weighting matrices Q and R need to be determined and downloaded into the onboard computer before starting entry guidance of HSV.
Instead of obtaining the specific elements in traditional method, the LQR design method using time-varying weighting matrix substitutes the flight state deviations z δ and v δ into the calculation of Q. The main idea of this method can be explained as the dashed line in Figure 4. With the help of Bryson principle, the calculation of elements in weighting matrix Q involves two parameters z δmax and v δmax . These two parameters represent maximal allowable deviations in altitude and velocity between actual and reference profiles, respectively. In the time-varying optimization method, one can make a comparison between the actual real-time profile and the relevant reference profile, and get the current deviations z δ (t) and v δ (t). Then substitute them into z δmax and v δmax , that is, Substituting z δmax and v δmax into Eq. (21), the weighting matrix Q can be obtained. The following algorithm is proposed to determine the actual guidance signal σ(t) with timevarying weighting matrix in LQR.
Algorithm 2. The actual guidance signal in trajectory tracking law based on LQR using timevarying weighting matrix can be designed in the following procedure.
Step 1 Measure the actual current flight profile which contains altitude z and velocity v.
Compare them with the relevant reference altitude z ref and velocity v ref , the current deviations z δ and v δ can be obtained, respectively.
Step 2 Substitute z, v, and γ into system, and calculate the linear system matrices A(t) and B(t).
Step 5 The compensatory guidance signal δu can be calculated by K(t) in Eq. (23). Then one can get the actual guidance signal in Eq. (24), namely bank angle σ(t).

Simulation results
In this section, a numerical simulation is given to demonstrate the effectiveness of the proposed method in the previous section.
Q 0 and Q 1 are defined in Section 3, and the time-varying weighting matrix is denoted by Q.
The simulations for hypersonic vehicle following reference profile with Q are shown in Figures 5-7.   For initial deviations of altitude and path angle, it's clear that the profile-following performances of Q 0 are better than Q 1 from Figures 1 and 2. Consequently, for the same deviations, Figures 5 and 6 choose Q 0 to compare with Q. Since Q 1 performs better than Q 0 in Figure 3 under the aerodynamic parameter error, one can choose Q 1 to compare with Q under the same disturbance in Figure 7.
From Figures 5-7, it can be shown that the time-varying matrix Q has better performance than Q 0 and Q 1 in the application of LQR on HSV following reference profile.

Conclusions
Because of complex entry environment and particular dynamic characteristics, such as strong nonlinear and large flight envelope, LQR with weighting matrices constructed by traditional Bryson principle was not suitable for HSV profile-following in reentry reference trajectory guidance. On the basis of Bryson principle, this chapter proposed time-varying matrices constructed by current flight states. The capability of HSV tracking nominal profile using LQR with time-varying weighting matrices was significantly improved. From simulations, it could be clearly shown that the LQR designed by presented method was more robust than traditional way to different initial deviations and disturbances in the process of reentry flight.