Development of a Genetic Fuzzy Controller and Its Application to a Noisy Inverted Double Pendulum

Abstract

Fuzzy logic is used in a variety of applications due to its universal approximator attribute and non-linear characteristics. The tuning of the parameters of a fuzzy logic system, viz. the membership functions and the rulebase, requires a lot of trial and error. This process could be simplified by using a heuristic search algorithm like genetic algorithm (GA). In this chapter, we discuss the design of such a genetic fuzzy controller that can control an inverted double pendulum. GA improves the fuzzy logic controller (FLC) with each generation during the training process to obtain an FLC that can bring the pendulum to its inverted position. After training, the effectiveness of the FLC is tested for different scenarios by varying the initial conditions. We also show the effectiveness of the FLC even when subjected to noise and how the performance improves when the controller is tuned with noise.

1. Introduction

Intelligent control techniques are gaining traction and increased focus and are being used in a wide variety of engineering applications. Fuzzy logic control is one such intelligent non-linear control technique that provides significant benefits in terms of design flexibility, universal approximator attribute and the ability to couple with optimization algorithms such as genetic algorithm (GA) for tuning its parameters. When coupled with the ability to capture expert or heuristic knowledge, and the ability to tune behavior in local envelopes of the operating space, fuzzy logic can be an indispensable control design tool in many applications. Fuzzy logic control also possesses inherent robustness due to having knowledge-based properties, making them good candidates for stochastic systems. One of the main challenges facing fuzzy logic control designers is the tuning of the membership functions and the heuristics involved. GA is used in this study to provide an autonomous guided search of the design space to develop a more optimized solution in accordance with the design requirements.

2. Literature review

Double pendulum is an example of a dynamic system that exhibits chaotic behavior. The inverted double pendulum is an archetype for thrust vector controlled multi-staged rocket or missile or even multi-rotor UAV flight control. The problem of controlling an inverted double pendulum has been studied for decades using different types of controllers. One of the approaches that has been used is a self-tuned neuro-PID controller to control the inverted pendulum on a cart [1]. The controller was realized by summing up two controllers for position and angle controls. They proved the effectiveness and robustness of this technique. An angular momentum based controller was able to control the double pendulum at any unstable position [2].

With the increase in computational capability and the advent of new and improved machine learning algorithms over the last decade, there has been an increase in the development of intelligent systems for various engineering applications such as path planning, target tracking, satellite attitude control systems [3], collaborative control of a swarm of UAVs [4], etc. Such intelligent systems provide various advantages, a few of which include adaptability, robustness to uncertainties and improved efficiency.

Fuzzy logic is one such intelligent system that can provide robustness and adaptability to controllers. Fuzzy logic was used in combination with optimal control theory to design a highly effective controller [5]. In a related research [6], fuzzy logic was used for stabilizing a parallel type double inverted pendulum. This is different from the inverted double pendulum in that it involves two separate pendulums being controlled simultaneously on a cart. Simulation results showed that the controller was able to stabilize completely the parallel-type double inverted pendulum system within 10s for a wide range of the initial angles of the two pendulums. The performance of fuzzy logic was compared to a PID controller in controlling an inverted pendulum [7]. Simulation results showed that the fuzzy logic controllers (FLCs) are far superior compared to PID controllers in terms of overshoot, settling time and response to parameter changes.

The performance of fuzzy controller tuned with noisy data was compared to that of a controller tuned without noise [8]. Optimizing the fuzzy system for a higher noise level results in good performance at lower noise levels. Lee presented three fuzzy system architectures and methods for automatically designing them for high dimensional problems [9]. The results indicate that the real coded algorithms consistently outperformed the binary coded algorithms in both the final performance of the system and the performance of the search algorithm. The Asymmetric-Triangular fuzzy systems consistently improved faster than the hyper-ellipsoidal and shared triangular representations in all cases.

Designing an FLS includes tuning the membership functions and the rulebase. This process can be automated by coupling GA with FLS to obtain the methodology known as Genetic Fuzzy System (GFS). In a GFS, GA tunes the parameters of the FLS to minimize a cost function that is carefully chosen such that minimizing it provides the desired behavior of the system. Such GFSs have been developed with much success for clustering and task planning [10], aircraft conflict resolution [11], simulated air-to-air combat [12], collaborative control of UAVs, etc. Since fuzzy logic systems are made up of a set of membership functions that define the inputs and a set of linguistic rules that define the relationship between the inputs and the outputs, it is more interpretable compared to other machine learning techniques like neural networks and support vector machines. Since it is trained using GA, differentiable cost functions such as integral squared error is not required. So, as long as the mission requirement can be defined as a mathematical cost function, we do not need to have ground truth data available. GA will traverse the search space looking for the optimal set of membership functions and rulebase that minimizes the cost function, which makes it a form of reinforcement learning. Reinforcement learning is a branch of machine learning where an agent is trained to take the optimal control action to maximize a reward.

3. Problem formulation

The objective is to design two fuzzy logic controllers that control the torques at the two joints to bring the double pendulum to its inverted position from any initial condition as shown in Figure 1. T₁ and T₂ are the torques applied by the controller at the joints. GA is used to tune the fuzzy membership functions as well as the rule base to come up with the best possible solution, which settles at θ₁ = 0, θ₂ = 0, in minimum time. The position of the masses m₁ and m₂ are given by

$$x_1=l_1\mathit{sin}{\theta }_1$$ E1

$$y_1=l_1\mathit{cos}{\theta }_1$$ E2

$$x_2=l_1\mathit{sin}{\theta }_1+l_2\mathit{sin}{\theta }_2$$ E3

$$y_2=l_1\mathit{cos}{\theta }_1+l_2\mathit{cos}{\theta }_2$$ E4

Figure 1.

Double pendulum schematic with controllers at the joints.

The equations of motion of the double pendulum when torques T₁ and T₂ act at the joints are given below.

$$\begin{array}{l}{T}_1+\left(m_1+m_2\right)L_1^2{\ddot{\theta }}_1+m_2{L}_1{L}_2{\ddot{\theta }}_2\text{cos}\left({\theta }_1-{\theta }_2\right)+m_2{L}_1{L}_2{\dot{\theta }}_2^2\text{sin}\left({\theta }_1-{\theta }_2\right)\\ +\left(m_1+m_2\right){\mathit{gL}}_1\mathit{sin}{\theta }_1=0\end{array}$$ E5

$$\begin{array}{l}{T}_2+m_2{L}_2^2{\ddot{\theta }}_2+m_2{L}_1{L}_2{\ddot{\theta }}_1\text{cos}\left({\theta }_1-{\theta }_2\right)-m_2{L}_1{L}_2{\dot{\theta }}_1^2\text{sin}\left({\theta }_1-{\theta }_2\right)\\ +m_2{\mathit{gL}}_2\mathit{sin}{\theta }_2=0\end{array}$$ E6

Assumptions

1. The masses m₁ and m₂ are assumed to be concentrated at the joints.

2. The masses m₁ and m₂ are considered as particles.

3. The rods are assumed to be massless.

4. Non-conservative forces like friction, air resistance, etc., are not considered.

5. There is no delay in the transmission of torque from the controller to the joint.

6. The motion is constrained to two dimensions. Hence z = 0.

7. The rods do not undergo expansion or compression. Thus, Eqs. (1)–(4) are satisfied.

4. Methodology

The flow chart showing the process of genetic fuzzy logic design as applied to the case of the inverted double pendulum is shown in Figure 2. Fuzzy logic is used to determine the torques acting on the two joints, T₁ and T₂. In order to reduce the computational complexity, each of the inputs and outputs are defined by just three membership functions as shown in Figure 3.

Figure 2.

Flowchart showing the genetic fuzzy logic methodology used for designing controllers for the inverted double pendulum problem [13].

Figure 3.

GA string values assigned to the input and output membership function boundaries for FIS1 [13].

GA is used to tune a 15-element vector R. GA is a search heuristic inspired from the process of natural selection that can perform an extensive search of a complicated n-D space, where n is the number of variables, to find a near optimal solution. For this research, n = 15. GA starts off with an initial set of random solutions also called a population of individuals, the size of which is pre-defined. There are two main operators in GA viz., crossover and mutation. The crossover requires two individuals and some portion of the two solution vectors are interchanged to obtain two child individuals. In a mutation, some values in a chosen individual are arbitrarily changed to obtain a new individual. The cost functions of all the resulting individuals are evaluated and the ones with lower cost values have higher probability of moving onto the next generation.

The first 9 elements in the vector, R(1–9), represent the rules as shown in Table 1 and R(10–15) represent the boundaries of the membership functions as shown in Figure 3. The membership functions are assumed to be symmetric around zero. R(1–9) are integers with values from one to three. One, two and three represent negative (N), zero (ZO) and positive (P) membership functions for the inputs, respectively.

$\dot{\mathit{\theta }}$; θ	N	ZO	P
N	R(1)	R(2)	R(3)
ZO	R(4)	R(5)	R(6)
P	R(7)	R(8)	R(9)

Table 1.

GA string values assigned to the rulebase during training.

For this research, we use centroid defuzzification to obtain the crisp output values. The rule-base is assumed to be same for both the controllers T1 and T2. AND operator connects the inputs θ and $\dot{\theta }$ . For example,

$$\mathit{\text{If}}{\theta }_1\mathit{\text{is}}N\mathit{\text{AND}}\mathit{\text{is}}P,\mathit{\text{thenT}}1\mathit{is}\mathit{R}\left(3\right).$$

The cost function that needs to be minimized by GA is given in Eq. (7). Since the cost function is an integral over time, it ensures that the system settles to the inverted position quickly. In our simulations, the simulation is performed for 5 s and hence T = 5.

$$f=\underset{0}{\overset{T}{\int }}\left({\theta }_1^2+{\theta }_2^2+{\dot{\theta }}_1^2+{\dot{\theta }}_2^2\right)\mathit{dt}$$ E7

5. Numerical results and discussion

The system response is simulated for the following parameter values:

$$m_1=0.1\mathit{kg}$$

$$m_2=0.1\mathit{kg}$$

$$l_1=1m$$

$$l_2=1m$$

The FISs are tuned for 2 cases: (1) Without noise and (2) With 5% measurement noise applied to the inputs.

5.1. FIS tuned without noise

The rulebase obtained after tuning is shown in Table 2. The membership function boundaries are obtained as R(10,15) = [0.5642 8.3738 7.1121 1.0264 4.2160 3.1641].

$\dot{\mathit{\theta }}$; θ	N	ZO	P
N	P	P	ZO
ZO	P	ZO	N
P	ZO	N	N

Table 2.

Rulebase obtained after training [13].

5.1.1. With zero initial angular velocity

The system response, with zero initial angular velocity, is shown in Figure 4. The system was tested for different starting positions and in each case, the response settles within 5 s. Settling time is the time it takes the response to settle within an absolute value of 0.01 rad. The genetic fuzzy controller works well even when it is subjected to noise in the angle measurements. The responses for 5 and 10% noise are shown in Figure 5. For the 5% noise scenario shown in Figure 5(a), the controller brings the system to settle in a very smooth manner. Both θ₁ and θ₂ settle within 5 s. In the case of 10% noise shown in Figure 5(b), θ₁ settles within 5 s, but θ₂ takes close to 10 s to settle. The response for θ₂ also shows a slight oscillation before settling down.

Figure 4.

Controllers trained without noise: Plots showing θ₁(t) and θ₂(t) under no noise with zero initial angular velocities [13].

Figure 5.

Controllers trained without noise: Plots showing θ₁(t) and θ₂(t) under (a) 5% noise and (b) 10% noise with zero initial angular velocities [13].

5.1.2. With non-zero initial angular velocities

In this case, initial angular velocities of 2 rad/s are considered (${\stackrel{\mathrm{̇}}{\theta }}_1=2\mathit{rad}/s,{\stackrel{\mathrm{̇}}{\theta }}_2=2\mathit{rad}/s$ ). The system response is shown in Figure 6. The system was tested for different starting positions and in each case, the response settles within 10 s. Just like the previous case with no initial angular velocity, the genetic fuzzy controller works well even when it is subjected to noise in the angle measurements. The responses for 5 and 10% noise are shown in Figure 7. For the 5% noise scenario shown in Figure 7(a), the controller brings the system to settle within 10 s. Some oscillation can be observed, although insignificant. In the case of 10% noise shown in Figure 7(b), θ₁ settles within 10 s, but θ₂ takes close to 15 s to settle. There is significant oscillation before it settles to θ₁ = 0; θ₂ = 0.

Figure 6.

Controllers trained without noise: Plots showing θ₁(t) and θ₂(t) under no noise with initial angular velocities of 2 rad/s [13].

Figure 7.

Controllers trained without noise: Plots showing θ₁(t) and θ₂(t) under (a) 5% noise and (b) 10% noise with initial angular velocities of 2 rad/s [13].

5.2. FIS tuned with 5% noise

In this case, the FIS is tuned with 5% noise and the response of the resulting controller is examined. The rulebase obtained is same as the one shown in Table 2. The membership function boundaries are obtained as R(10:15) = [0.3100 11.9261 7.4158 0.8904 3.8477 3.5883].

5.2.1. With no initial angular velocity

The system response, with zero initial angular velocity, is shown in Figure 8. The system was tested for different starting positions and in each case, the response settles within 5 s. The genetic fuzzy controller works well even when it is subjected to noise in the angle measurements. The responses for 5 and 10% noise are shown in Figure 9. The controller is able to bring the system to settle within 5 s even in the presence of noise. Although there is a small undershoot, θ₁ settles much faster compared to the responses shown in Figures 4 and 5. There are no significant oscillations in the system response. It can be observed from Figures 8 and 9 that the noise has a minuscule effect on the system response thus proving that the controller is very robust. Thus, when there is no initial angular velocity, the controller tuned for 5% noise performs much better, as expected.

Figure 8.

Controllers trained with 5% noise: plots showing θ₁(t) and θ₂(t) under no noise with zero initial angular velocities [13].

Figure 9.

Controllers trained with 5% noise: plots showing θ₁(t) and θ₂(t) under (a) 5% noise and (b) 10% noise with zero initial angular velocities [13].

5.2.2. With non-zero initial angular velocities

In this case, initial angular velocities of 2 rad/s are considered ( ${\stackrel{\mathrm{̇}}{\theta }}_1=2\mathit{rad}/s,{\stackrel{\mathrm{̇}}{\theta }}_2=2\mathit{rad}/s$ ). The system response is shown in Figure 10. The system was tested for different starting positions and in each case, the response settles within 5 s. The response of the controller when subjected to 5 and 10% noise are shown in Figure 11. For both cases, the controller brings the system to settle within 5 s. Even in the case of non-zero initial angular velocity, the controller is very resilient to noise. The settling time is less compared to the controller tuned without noise.

Figure 10.

Controllers trained with 5% noise: plots showing θ₁(t) and θ₂(t) under no noise with initial angular velocities of 2 rad/s [13].

Figure 11.

Controllers trained with 5% noise: plots showing θ₁(t) and θ₂(t) under (a) 5% noise and (b) 10% noise with initial angular velocities of 2 rad/s [13].

Table 3 compiles the results obtained for the double pendulum system. The table compares the settling time and the integral square error (ISE) for the two different controllers. The ISE is given by,

$$\mathit{ISE}=\underset{0}{\overset{\infty }{\int }}{e}^2\left(t\right)\mathit{dt}$$ E8

where e(t) is the error obtained by subtracting the actual response and the desired response which in our case is zero.

		Controllers trained without noise			Controllers trained with 5% noise
		No noise	5% Noise	10% Noise	No noise	5% Noise	10% Noise
${\dot{\theta }}_1={\dot{\theta }}_2=0$	Ts(θ1)	1.632	1.636	1.708	0.3895	0.3906	0.3911
	Ts(θ2)	3.871	3.874	5.723	2.1532	2.1545	2.1560
	ISE(θ1)	0.8825	0.8853	0.8961	0.3875	0.3943	0.4012
	ISE(θ2)	0.2412	0.2450	0.3323	0.0985	0.0988	0.0990
${\dot{\theta }}_1={\dot{\theta }}_2=2$	Ts(θ1)	4.431	4.487	5.191	2.347	2.368	2.449
	Ts(θ2)	5.208	5.271	13.538	1.475	1.487	1.531
	ISE(θ1)	3.4389	3.5963	5.9624	2.2963	2.3440	2.4009
	ISE(θ2)	3.2142	3.4433	11.0016	0.6271	0.6363	0.6674

Table 3.

Comparison of settling times (Ts in seconds) and the ISE for the scenarios discussed before [13].

It can be seen from Table 3 that the settling time and ISE are better in the case of the controller tuned with 5% noise. Optimizing the controller with 5% noise helps with the system response for larger window of uncertainty. As can be seen from Table 3, the ISE values for 10% noise scenario in the case of the controller tuned without noise is on the higher side. But, this gets reduced when the controller is tuned for 5% noise. Thus, the overall performance of the system increases by tuning the controller with 5% measurement noise.

6. Conclusions

This chapter discussed the design of genetic fuzzy controllers to control an inverted double pendulum. While fuzzy logic by itself works well, tuning the parameters involved to satisfy a specific requirement might need a lot of trial and error to be done by the researchers. Incorporating GA to tune these parameters solves this problem. In this chapter, the objective was to bring the system to its inverted position. The time-integral cost function ensured that the FISs are trained to reduce the settling time.

The genetic fuzzy controller was able to stabilize the double pendulum at the inverted position starting from any initial position. The controller was tuned for two cases: (1) when there is no noise, and (2) when subjected to 5% noise. For each of the two cases, the results were shown for two sub-cases: (a) with zero initial angular velocities and (b) with non-zero initial angular velocities. The controller tuned for 5% has a better performance than the one tuned without noise. Tuning the controller with 5% noise improves the robustness of the system for a larger window of uncertainty. Since a lot of real-life systems suffer from measurement noise, it is important to develop robust controllers that can make decisions even when the inputs are noisy.