1. Introduction
Fuzzy systems (FSs) have been extensively applied to automatic control, pattern recognition, and decision analysis. However, a common bottleneck is encountered in the derivation of fuzzy rules, which is often difficult, time consuming, and relies on expert knowledge. To automate the design of FSs, many metaheuristic learning algorithms have been proposed. One major optimization category uses Swarm Intelligence (SI) model (Kennedy et al., 2001). The SI technique studies collective behavior in decentralized systems. Its development was based on mimicking the social behavior of animals or insects in an effort to find the optima in the problem space. SI models are initialized with a population of random solutions. One well-known SI model is particle swarm optimization (PSO) (Kennedy & Eberhart, 1995). Many modified PSO models have been proposed and successfully applied to different optimization problems (Clerc & Kennedy, 2002; Bergh & Engelbrecht, 2004; Ratnaweera et al., 2004; Juang, 2004; Kennedy & Mendes, 2006; Parrott & Li, 2006; Chen & Li, 2007). FS design using PSO has also been proposed in several studies (Juang, 2004; Chatterjee et al., 2005; Juang et al., 2007 ; Araujo & Coelho, 2008; Sharma et al., 2009).
Another well-known SI is ant colony optimization (ACO) (Dorigo & Stutzle, 2004). The ACO technique is inspired by real ant colony observations. It is a multi-agent approach that was originally proposed to solve difficult discrete combinatorial optimization problems, such as the traveling salesman problem (TSP) (Dorigo et al., 1996; Dorigo & Gambardella, 1997). In the original ACO meta-heuristic, artificial ant colonies cooperate to find good solutions for difficult discrete optimization problems. Different ACO models have been applied to FS design problems (Cassillas et al., 2000; Cassillas et al., 2005; Mucientes & Casillas; 2007; Juang & Lo, 2007 ; Juang et al., 2008; Juang & Lu; 2009 ). In (Cassillas et al.,2000; Mucientes & Casillas; 2007; Juang et al., 2008 ; Juang & Lu; 2009), the FS input space was partitioned in grid type with antecedent part parameters of an FS manually assigned in advance. In ( Juang & Lo, 2007 ), the FS input space was flexibly partitioned using a fuzzy clustering-like algorithm in order to reduce the total number of rules. For all of these studies, the consequent part parameters were optimized in discrete space using ACO. Since only the consequent part parameters are optimized, and the optimization space is restricted to be discrete, the designed FSs are unsuitable for problems where high accuracy is a major concern.
Several studies on the combination of PSO or ACO with other optimization algorithms have been proposed in order to improve the performance of the original optimization model. In (Juang, 2004), a hybrid of GA and PSO, called HGAPSO, was proposed. In HGAPSO, new individuals are created not only by PSO but also by the crossover and mutation operations of a GA. In (Fan et al., 2004; Juang & Hsu, 2005), the simplex method was introduced into PSO. In (Ling et al., 2008), a hybrid of PSO with wavelet mutation was proposed. To apply the ACO technique to solve continuous optimization problems, several studies on the combination of ACO with other continuous optimization methods have been performed (Feng & Feng, 2004; Ge et al., 2004). An ACO followed by immune operation for optimization in a continuous space was proposed in (Feng & Feng, 2004). The incorporation of a deterministic searching algorithm (the Powell method) into ACO for continuous optimization was proposed in (Ge et al., 2004).
This chapter studies the combination of PSO and ACO for FSs design. One problem of PSO in FS design is that its performance is affected by initial particle positions, which are usually randomly generated in a continuous search space. A poor initialization may result in poor performance. Searching in the discrete-space domain by ACO helps to find good solutions. However, the search constraint in a discrete-space domain restricts learning accuracy. The motivation on the combination of ACO and PSO is to compensate the aforementioned weakness of each method in FS design problems. Two combination approaches, sequential and parallel, for PSO and ACO proposed in ( Juang & Lo, 2008 ; Juang & Wang, 2009) are described and discussed in this Chapter.
This chapter is organized as follows. Section 2 describes the FS to be designed. The rule generation algorithm and rule initialization are also described in this section. Section 3 describes PSO and how to apply it to FS design. Section 4 describes ACO and how to apply it to FS design. Section 5 describes the sequential combination of PSO and ACO for FS design. Section 6 describes the parallel combination of PSO and ACO for FS design. Finally, Section 7 draws conclusions.
2. Fuzzy systems
2.1. Fuzzy system functions
This subsection describes the FS to be designed. The FS is of zero-order Takagi-Sugeno-Kang (TSK) type. That is, the
where
where
If there are
where
2.2. Rule generation and initialization
Most studies on SI-based FS design algorithms determine the number of rules by trial and errors and assign the initial FS parameters randomly and uniformly in the domain of each free parameter. The subsection describes one promising rule generation and initialization algorithm based on the fuzzy clustering-like approach that has been used in an SI algorithm (
Juang et al., 2007
). It is assumed that there are initially no rules in the designed FS. The rule generation method generates fuzzy rules online upon receiving training data. Rules are generated in order to ensure that at least one rule is activated with a firing strength larger than a pre-defined threshold
Geometrically, as Fig. 1 shows, this threshold ensures that each input data is properly covered by a rule in the input space. According to this concept, the firing strength
where
where
3. Particle swarm optimization (PSO) for FS design
This section first describes the basic concept of PSO. The application of PSO to optimize the generated FS in Section 2 is then is then described. The swarm in PSO is initialized with a population of random solutions (Kennedy & Eberhart, 1995). Each potential solution is called a particle. Each particle has a position, which is represented by a position vector
where
Based on (7) and (8), the particle population tends to cluster around the best.
The use of PSO for FS design, i.e., optimization of all free parameters in an FS, is described as follows. For the FS in (1) that consists of
After the rule generation and initialization process described in Section 2, the initial antecedent part parameters are determined. Based on the solution vector representation in (9) and the antecedent part parameter initialization in (6), the
where
4. Ant colony optimization (ACO) for FS design
ACO is a meta-heuristic algorithm inspired by the behavior of real ants, and in particular how they forage for food (Dorigo & Caro, 1999; Dorigo & Stutzle, 2004). It was first applied to the traveling salesman problem (TSP). In ACO, a finite size colony of artificial ants is created. Each ant then builds a solution to the problem. While building its own solution, each ant collects information based on the problem characteristics and on its own performance. The performance measure is based on a quality function
Let
where
where
ACO can be applied to design the consequent part parameters in an FS. The general design approach is described as follows. Consider the FS whose structure and antecedent part parameters are determined according to (5) and (6) in Section 2. Suppose the consequent part is selected from a discrete set
That is, we have to decide one from a total of
In an FS, different fuzzy rules may share the same consequent value. That is, when value
The ACO works without the use of heuristic values, and the consequent part can be simply selected by using (14). The use of heuristic values can be further employed for learning performance improvement. However, determination of the heuristic value
5. Sequential combination of ACO and PSO
This section describes the sequential combination approach of ACO and PSO proposed in ( Juang & Lo, 2008 ). In this sequential combination approach, the rule consequent of each on-line generated rule described in Section 2 is first learned by ACO. The advantage of using ACO for rule consequent learning is that it can help determine a good fuzzy rule base for subsequent learning. However, the search constraint in a discrete-space domain restricts learning accuracy, and the ants do not optimize antecedent part parameters. Therefore, after ACO learning, PSO is then employed to further optimize both the antecedent and consequent parameters, where initial particles in PSO are generated according to learning results from ACO.
Figure 3 shows the flow of the sequential combination of ACO and PSO. The formulation of consequent part learning by ACO is described in Section 4. Detailed function evaluation and update of pheromone levels are described as follows. The pheromone trails,
where 0 <
The ACO learning iteration above repeats until the criterion for switching is met. The switching point from ACO to PSO learning is determined by the learning error convergence property of the global best ant. Let
then ACO learning terminates and learning switches to PSO.
Using PSO releases the discrete space constraint imposed on consequent parameters when ACO is used, and searches the best consequent parameters in continuous space. In addition to the consequent parameters, PSO also searches the optimal antecedent part parameters. Like ACO, population size in the PSO is equal to
where
6. Parallel combination of ACO and PSO
This section describes the parallel combination approach of ACO and PSO for FS design (
Juang & Wang, 2009
). Like the sequential combination approach, the parallel combination approach uses a constant population size and is denoted as
In the first iteration, the rule generation algorithm in Section 2 and
The second and subsequent iterations generate a new population with 2
The performance of these
For ant individual update by ACO at the end of each iteration, as in Section 4, a new ant path generates a new FS (individual) according to the pheromone levels and transition probability in (14). The pheromone levels are updated using (15) and (16). For particle individual update in iteration
In ( Juang & Wang, 2009 ), the parallel combination learning approach has been applied to two control examples, nonlinear plant tracking control and reversing a truck following a circular path. These examples generate training data only when fuzzy control starts. Simulation results show that the proposed method achieves a smaller control error than ACO, PSO, and other different SI learning algorithms used for comparison in each example.
7. Conclusion
This chapter describes the design of FSs using PSO, ACO, and their sequential and parallel combination approaches. The use of on-line rule generation not only helps to determine the number of fuzzy rules, but also helps to locate the initial antecedent parameters for subsequent parameter learning using PSO. For PSO, the incorporation of ACO helps to locate a good initial fuzzy rule base for further PSO learning. For ACO, the incorporation of PSO helps to find the parameters in a continuous space. The cooperative search of ACO and PSO compensates for the searching disadvantage of each optimization method. Reported results show that the two combination approaches outperform different advanced PSO and ACO algorithms for FS design problems. Performance of these two combination approaches on other optimization problems will be studied in the future. Other different combination approaches of ACO and PSO may also be studied for further performance improvement.
References
- 1.
Araujo E. Coelho L. d. S. 2008 Particle swarm approaches using Lozi map chaotic sequences to fuzzy modeling of an experimental thermal-vacuum system, ,8 4 1354 1364 . - 2.
Blum C. Dorigo M. 2004 The hyper-cube framework for ant colony optimization, ,34 2 1161 72 . - 3.
Bergh F. van den Engelbrecht A. P. 2004 A cooperative approach to particle swarm optimization , ,8 3 225 239 . - 4.
Cassillas J. Cordon O. Herrera F. 2000 Learning fuzzy rules using ant colony optimization algorithms,13 21 , Brussels, Belgium. - 5.
Clerc M. Kennedy J. 2002 The particle swarm- explosion, stability, and convergence in a multidimentional complex space, ,6 1 58-73. - 6.
Cassillas J. Cordon O. Viana I. F. Herrera F. 2005 Learning cooperative linguistic rules using the best-worst ant system algorithm, ,20 433-452. - 7.
Chatterjee A. Pulasinghe K. Watanabe K. Izumi K. 2005 A particle-swarm-optimized fuzzy-neural network for voice-controlled robot systems , ,52 6 1478 1489 . - 8.
Chen X. Li Y. 2007 A modified PSO structure resulting in high exploration ability with convergence guaranteed, ,37 5 1271 89 . - 9.
Dorigo M. Maniezzo V. Colorni A. 1996 Ant system: optimization by a colony of cooperating agents , ,26 1 29-41. - 10.
Dorigo M. Gambardella L. M. 1997 Ant colony system: A cooperative learning approach to the traveling salesman problem , ,1 1 53 66 . - 11.
Dorigo M. Caro G. D. 1999 The ant colony optimization meta-heuristic , , D. Corne, M. Dorigo, & F. Glover Eds.,11 32 , London, McGraw-Hill. - 12.
Dorigo M. Stutzle T. 2004 Ant Colony Optimization, MIT. - 13.
Feng Y. J. Feng Z. R. 2004 An immunity-based ant system for continuous space multi-modal function optimization, ,2 1050 1054 . - 14.
Fan S. K. S. Liang Y. C. Zahara E. 2004 Hybrid simplex search and particle swarm optimization for the global optimization of multimodal functions, ,36 4 401-418. - 15.
Ge Y. Meng Q. C. Yan C. J. Xu J. 2004 A hybrid ant colony algorithm for global optimization of continuous multi-extreme functions, ,4 2427 2432 . - 16.
Juang C. F. 2004 A hybrid of genetic algorithm and particle swarm optimization for recurrent network design, ,34 2 997 1006 . - 17.
Juang C. F. Hsu C. H. 2005 Temperature control by chip-implemented adaptive recurrent fuzzy controller designed by evolutionary algorithm , ,52 11 2376 2384 . - 18.
Juang C. F. Chung I. F. Hsu C. H. 2007 Automatic construction of feedforward /recurrent fuzzy systems by clustering-aided simplex particle swarm optimization , ,158 18 1979 1996 . - 19.
Juang C. F. Lo C. 2007 Fuzzy systems design by clustering-aided ant colony optimization for plant control , ,36 6 623 641 . - 20.
Juang C. F. Lu C. M. Lo C. Wang C. Y. 2008 Ant colony optimization algorithm for fuzzy controller design and its FPGA implementation , ,55 3 1453 1462 . - 21.
Juang C. F. Lo C. 2008 Zero-order TSK-type fuzzy system learning using a two-phase swarm intelligence, ,159 21 2910 2926 . - 22.
Juang C. F. Lu C. M. 2009 Ant colony optimization incorporated with fuzzy Q-learning for reinforcement fuzzy control , ,39 3 597 608 . - 23.
Juang C. F. Wang C. Y. 2009 A self-generating fuzzy system with ant and particle swarm cooperative optimization , ,36 3P1 , 5362-5370. - 24.
Kennedy J. Eberhart R. 1995 Particle swarm optimization , , Perth, Australia,1942 1948 . - 25.
Kennedy J. Eberhart R. Shi Y. 2001 Swarm Intelligence, Morgan Kaufmann Publisher. - 26.
Kennedy J. Mendes R. 2006 Neighborhood topologies in fully informed and best-of-neighborhood particle swarms, ,36 4 515- 519. - 27.
Ling S. H. Iu H. H. C. Chan K. Y. Lam H. K. Yeung B. C. W. Leung F. H. 2008 Hybrid particle swarm optimization with wavelet mutation and its industrial applications, ,38 3 743-763. - 28.
Mucientes M. Casillas J. 2007 Quick design of fuzzy controllers with good interpretability in mobile robotics , ,15 4 636 651 . - 29.
Parrott D. Li X. 2006 Locating and tracking multiple dynamic optima by a particle swarm model using speciation , ,10 4 440 458 . - 30.
Ratnaweera A. Halgamuge S. K. Watson H. C. 2004 Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , ,8 3 240 255 . - 31.
Stutzle T. Hoos H. H. 2000 MAX MIN ant system, ,8 16 889-914. - 32.
Sharma K. D. Chatterjee A. Rakshit A. 2009 A hybrid approach for design of stable adaptive fuzzy controllers employing Lyapunov theory and particle swarm optimization, ,17 2 329-342.