Development of a Genetic Fuzzy Controller and Its Application to a Noisy Inverted Double Pendulum Development of a Genetic Fuzzy Controller and Its Application to a Noisy Inverted Double Pendulum

Fuzzy logic is used in a variety of applications due to its universal approximator attribute and non-linear characteristics. The tuning of the parameters of a fuzzy logic system, viz. the membership functions and the rulebase, requires a lot of trial and error. This process could be simplified by using a heuristic search algorithm like genetic algorithm (GA). In this chapter, we discuss the design of such a genetic fuzzy controller that can control an inverted double pendulum. GA improves the fuzzy logic controller (FLC) with each gen eration during the training process to obtain an FLC that can bring the pendulum to its inverted position. After training, the effectiveness of the FLC is tested for different scenarios by varying the initial conditions. We also show the effectiveness of the FLC even when sub - jected to noise and how the performance improves when the controller is tuned with noise.


Introduction
Intelligent control techniques are gaining traction and increased focus and are being used in a wide variety of engineering applications. Fuzzy logic control is one such intelligent non-linear control technique that provides significant benefits in terms of design flexibility, universal approximator attribute and the ability to couple with optimization algorithms such as genetic algorithm (GA) for tuning its parameters. When coupled with the ability to capture expert or heuristic knowledge, and the ability to tune behavior in local envelopes of the operating space, fuzzy logic can be an indispensable control design tool in many applications. Fuzzy logic control also possesses inherent robustness due to having knowledge-based properties, making them good candidates for stochastic systems. One of the main challenges facing fuzzy logic control designers is the tuning of the membership functions and the heuristics involved. GA is used in this study to provide an autonomous guided search of the design space to develop a more optimized solution in accordance with the design requirements.

Literature review
Double pendulum is an example of a dynamic system that exhibits chaotic behavior. The inverted double pendulum is an archetype for thrust vector controlled multi-staged rocket or missile or even multi-rotor UAV flight control. The problem of controlling an inverted double pendulum has been studied for decades using different types of controllers. One of the approaches that has been used is a self-tuned neuro-PID controller to control the inverted pendulum on a cart [1]. The controller was realized by summing up two controllers for position and angle controls. They proved the effectiveness and robustness of this technique. An angular momentum based controller was able to control the double pendulum at any unstable position [2].
With the increase in computational capability and the advent of new and improved machine learning algorithms over the last decade, there has been an increase in the development of intelligent systems for various engineering applications such as path planning, target tracking, satellite attitude control systems [3], collaborative control of a swarm of UAVs [4], etc. Such intelligent systems provide various advantages, a few of which include adaptability, robustness to uncertainties and improved efficiency.
Fuzzy logic is one such intelligent system that can provide robustness and adaptability to controllers. Fuzzy logic was used in combination with optimal control theory to design a highly effective controller [5]. In a related research [6], fuzzy logic was used for stabilizing a parallel type double inverted pendulum. This is different from the inverted double pendulum in that it involves two separate pendulums being controlled simultaneously on a cart. Simulation results showed that the controller was able to stabilize completely the parallel-type double inverted pendulum system within 10s for a wide range of the initial angles of the two pendulums. The performance of fuzzy logic was compared to a PID controller in controlling an inverted pendulum [7]. Simulation results showed that the fuzzy logic controllers (FLCs) are far superior compared to PID controllers in terms of overshoot, settling time and response to parameter changes.
The performance of fuzzy controller tuned with noisy data was compared to that of a controller tuned without noise [8]. Optimizing the fuzzy system for a higher noise level results in good performance at lower noise levels. Lee presented three fuzzy system architectures and methods for automatically designing them for high dimensional problems [9]. The results indicate that the real coded algorithms consistently outperformed the binary coded algorithms in both the final performance of the system and the performance of the search algorithm. The Asymmetric-Triangular fuzzy systems consistently improved faster than the hyper-ellipsoidal and shared triangular representations in all cases.
Designing an FLS includes tuning the membership functions and the rulebase. This process can be automated by coupling GA with FLS to obtain the methodology known as Genetic Fuzzy System (GFS). In a GFS, GA tunes the parameters of the FLS to minimize a cost function that is carefully chosen such that minimizing it provides the desired behavior of the system. Such GFSs have been developed with much success for clustering and task planning [10], aircraft conflict resolution [11], simulated air-to-air combat [12], collaborative control of UAVs, etc. Since fuzzy logic systems are made up of a set of membership functions that define the inputs and a set of linguistic rules that define the relationship between the inputs and the outputs, it is more interpretable compared to other machine learning techniques like neural networks and support vector machines. Since it is trained using GA, differentiable cost functions such as integral squared error is not required. So, as long as the mission requirement can be defined as a mathematical cost function, we do not need to have ground truth data available. GA will traverse the search space looking for the optimal set of membership functions and rulebase that minimizes the cost function, which makes it a form of reinforcement learning. Reinforcement learning is a branch of machine learning where an agent is trained to take the optimal control action to maximize a reward.

Problem formulation
The objective is to design two fuzzy logic controllers that control the torques at the two joints to bring the double pendulum to its inverted position from any initial condition as shown in Figure 1. T 1 and T 2 are the torques applied by the controller at the joints. GA is used to tune the fuzzy membership functions as well as the rule base to come up with the best possible solution, which settles at θ 1 = 0, θ 2 = 0, in minimum time. The position of the masses m 1 and m 2 are given by x 2 = l 1 sin θ 1 + l 2 sin θ 2 (3) The equations of motion of the double pendulum when torques T 1 and T 2 act at the joints are given below.
The masses m 1 and m 2 are assumed to be concentrated at the joints.

2.
The masses m 1 and m 2 are considered as particles.
3. The rods are assumed to be massless.

4.
Non-conservative forces like friction, air resistance, etc., are not considered.

5.
There is no delay in the transmission of torque from the controller to the joint.

6.
The motion is constrained to two dimensions. Hence z = 0.

Methodology
The flow chart showing the process of genetic fuzzy logic design as applied to the case of the inverted double pendulum is shown in Figure 2. Fuzzy logic is used to determine the torques acting on the two joints, T 1 and T 2 . In order to reduce the computational complexity, each of the inputs and outputs are defined by just three membership functions as shown in Figure 3.
GA is used to tune a 15-element vector R. GA is a search heuristic inspired from the process of natural selection that can perform an extensive search of a complicated n-D space, where n is the number of variables, to find a near optimal solution. For this research, n = 15. GA starts off with an initial set of random solutions also called a population of individuals, the size of which is pre-defined. There are two main operators in GA viz., crossover and mutation.
The crossover requires two individuals and some portion of the two solution vectors are interchanged to obtain two child individuals. In a mutation, some values in a chosen individual are arbitrarily changed to obtain a new individual. The cost functions of all the resulting individuals are evaluated and the ones with lower cost values have higher probability of moving onto the next generation.
For this research, we use centroid defuzzification to obtain the crisp output values. The rulebase is assumed to be same for both the controllers T1 and T2. AND operator connects the inputs θ and θ ̇ . For example, Figure 2. Flowchart showing the genetic fuzzy logic methodology used for designing controllers for the inverted double pendulum problem [13].
The cost function that needs to be minimized by GA is given in Eq. (7). Since the cost function is an integral over time, it ensures that the system settles to the inverted position quickly. In our simulations, the simulation is performed for 5 s and hence T = 5. ) dt (7)

Numerical results and discussion
The system response is simulated for the following parameter values: The FISs are tuned for 2 cases: (1) Without noise and (2) With 5% measurement noise applied to the inputs.

FIS tuned without noise
The rulebase obtained after tuning is shown in

With zero initial angular velocity
The system response, with zero initial angular velocity, is shown in Figure 4. The system was tested for different starting positions and in each case, the response settles within 5 s. Settling time is the time it takes the response to settle within an absolute value of 0.01 rad. The genetic fuzzy controller works well even when it is subjected to noise in the angle measurements. The responses for 5 and 10% noise are shown in Figure 5. For the 5% noise scenario shown in Figure 5(a), the controller brings the system to settle in a very smooth manner. Both θ 1 and θ 2 settle within 5 s. In the case of 10% noise shown in Figure 5(b), θ 1 settles within 5 s, but θ 2 takes close to 10 s to settle. The response for θ 2 also shows a slight oscillation before settling down.

With non-zero initial angular velocities
In this case, initial angular velocities of 2 rad/s are considered ( θ ̇ 1 = 2 rad / s, θ ̇ 2 = 2 rad / s ). The system response is shown in Figure 6. The system was tested for different starting positions and in each case, the response settles within 10 s. Just like the previous case with no initial angular velocity, the  genetic fuzzy controller works well even when it is subjected to noise in the angle measurements. The responses for 5 and 10% noise are shown in Figure 7. For the 5% noise scenario shown in Figure 7(a), the controller brings the system to settle within 10 s. Some oscillation can be observed, although insignificant. In the case of 10% noise shown in Figure 7(b), θ 1 settles within 10 s, but θ 2 takes close to 15 s to settle. There is significant oscillation before it settles to θ 1 = 0; θ 2 = 0.

FIS tuned with 5% noise
In this case, the FIS is tuned with 5% noise and the response of the resulting controller is examined. The rulebase obtained is same as the one shown in

With no initial angular velocity
The system response, with zero initial angular velocity, is shown in Figure 8. The system was tested for different starting positions and in each case, the response settles within 5 s. The genetic fuzzy controller works well even when it is subjected to noise in the angle measurements. The responses for 5 and 10% noise are shown in Figure 9. The controller is able to bring the system to settle within 5 s even in the presence of noise. Although there is a small undershoot, θ 1 settles much faster compared to the responses shown in Figures 4 and 5. There are no significant oscillations in the system response. It can be observed from Figures 8 and 9 that the noise has a minuscule effect on the system response thus proving that the controller is very robust. Thus, when there is no initial angular velocity, the controller tuned for 5% noise performs much better, as expected.

With non-zero initial angular velocities
In this case, initial angular velocities of 2 rad/s are considered ( θ ̇ 1 = 2 rad / s, θ ̇ 2 = 2 rad / s ). The system response is shown in Figure 10. The system was tested for different starting positions and in each case, the response settles within 5 s. The response of the controller when subjected to 5 and 10% noise are shown in Figure 11. For both cases, the controller brings the system to settle within 5 s. Even in the case of non-zero initial angular velocity, the controller is very resilient to noise. The settling time is less compared to the controller tuned without noise.
where e(t) is the error obtained by subtracting the actual response and the desired response which in our case is zero.
It can be seen from Table 3 that the settling time and ISE are better in the case of the controller tuned with 5% noise. Optimizing the controller with 5% noise helps with the system response for larger window of uncertainty. As can be seen from Table 3, the ISE values for 10% noise Figure 11. Controllers trained with 5% noise: plots showing θ 1 (t) and θ 2 (t) under (a) 5% noise and (b) 10% noise with initial angular velocities of 2 rad/s [13].  Table 3. Comparison of settling times (Ts in seconds) and the ISE for the scenarios discussed before [13].

Controllers trained without noise
Fuzzy Logic Based in Optimization Methods and Control Systems and Its Applications scenario in the case of the controller tuned without noise is on the higher side. But, this gets reduced when the controller is tuned for 5% noise. Thus, the overall performance of the system increases by tuning the controller with 5% measurement noise.

Conclusions
This chapter discussed the design of genetic fuzzy controllers to control an inverted double pendulum. While fuzzy logic by itself works well, tuning the parameters involved to satisfy a specific requirement might need a lot of trial and error to be done by the researchers. Incorporating GA to tune these parameters solves this problem. In this chapter, the objective was to bring the system to its inverted position. The time-integral cost function ensured that the FISs are trained to reduce the settling time.