Introductory Chapter: Swarm Intelligence and Particle Swarm Optimization

1. Introduction

In order to survive, the main objective of all creatures is foraging. Foraging behavior is cooperative in the same species. Each agent in the swarm communicates with others in such a way to find the food in the shortest time and way. This capability of all lively beings gives inspiration to the human being in order to find solutions to the optimization problems. Collective foraging behaviors of the lively beings are called swarm intelligence.

Most of the animals live as social groups in order to find foods easily and protect from the enemies to survive. Each individual lives in their habitat. Looking for food, they use their own experiences called cognitive movements as well as the experience of their leaders called social movements.

Optimization is to find the best solution to a given problem under some constraints. All disciplines use optimization for finding the best solution for their problems. Optimization is the first and foremost objective for engineers too. So especially in the future engineering applications, optimization will be an indispensable part of the product.

Optimization is everywhere. In the production of a new device, in a new artificial intelligence technique, in a big data application or in a deep learning network, optimization is the most important part of the application. To design a device with optimum sizes using minimum energy, to train a network, to minimize the error between the desired output and real output values, optimization is required.

Because of the difficulties of classical optimization algorithms, scientists have started to find an easy way to solve their problems in the last 1960s. The development of the computers made the efforts of the scientists easy, and completely new problem solution techniques are studied. These techniques using heuristic information were derivative free, easy to implement, and shorten the solution time. The first product of these studies is genetic algorithm (GA) developed by Holland [1]. The evolutionary idea has been applied to the solution of the optimization problems. Instead of the evolving only one solution, a group of solutions called population has been used in the algorithm. Each solution is called individual. By this way, running such algorithms with multiple processors could be possible. After GA, simulated annealing [2] has been generally accepted as the second algorithm, inspired from the annealing process of physical materials. In high temperatures, particles move randomly in order to explore the solution space. While temperature is decreasing, particles try to create a perfect crystalline structure, only with local movements.

2. Particle swarm optimization

Particle swarm optimization (PSO) is accepted as the second population-based algorithm inspired from animals. Since James Kennedy (a social psychologist) and Russell C. Eberhart simulated the bird flocking and fish schooling foraging behaviors, they have used this simulation to the solution of an optimization problem and published their idea in a conference in 1995 [3] for the optimization of continuous nonlinear functions. There are two main concepts in the algorithm: velocity and coordinate for each particle. Each particle has a coordinate and an initial velocity in a solution space. As the algorithm progresses, the particles converge toward the best solution coordinates. Since PSO is quite simple to implement, it requires less memory and has no operator. Due to this simplicity, PSO is also a fast algorithm. Different versions of PSO have been developed, using some operators since the first version of PSO was published.

In the first versions of PSO, the velocity was calculated with a basic formula using current velocity, personal best and local best values in the formula, multiplying stochastic variables. The current particle updates its previous velocity, not only its previous best but also the global best. The total probability was distributed between local and global best using stochastic variables.

In the next versions, in order to control the velocity, an inertia weight was introduced by Shi and Eberhart in 1998 [4]. Inertia weight balances the local and global search ability of algorithm. Inertia weight specifies the rate of contribution of previous velocity to its current velocity. Researchers made different contributions to the inertia weight concept. Linearly, exponential or randomly decreasing or adaptive inertia weight was introduced by different researchers [5]. In the next version of PSO, a new parameter called constriction factor was introduced by Clerc and Kenedy [6, 7]. Constriction factor (K) was introduced in the studies on stability and convergence of PSO. Clerc indicates that the use of a constriction factor insured convergence of the PSO. A comparison between inertia weight and constriction factor was published by Shi and Eberhart [8].

Nearly all engineering discipline and science problems have been solved with PSO. Some of the most studied problems solved with PSO are from Electrical Engineering, Computer Sciences, Industrial Engineering, Biomedical Engineering, Mechanical Engineering and Robotics. In Electrical Engineering, power distribution problem [9] is solved with PSO. Another most studied problem in Electrical Engineering is economic dispatch problem [10, 11]. In Computer Sciences, face localization [12], edge detection [13], image segmentation [14], image denoising [15], image filtering [16] problems are solved with PSO. In Industrial Engineering, examination timetabling problems [17], traveling salesman problem [18], and job-shop scheduling problems [19] are solved with PSO. In Robotics, particle swarm optimization in coconut tree plucking robot is introduced [20], and path planning problem is solved with PSO [21]. In these studies, it has been proven PSO success in point of both performance and speed in most of the studies.

After the main PSO algorithm was studied and evolved with some parameters, hybrid algorithms were designed and developed by researchers. The most prominent ones are hybrid PSO with genetic algorithm (GA) [22] and particle swarm optimization with chaos [23, 24, 25] and quantum chaotic PSO [26]. In the next section, some of the recent hybrid PSO algorithms are presented.

3. Hybrid PSO algorithms using swarm intelligence

Hybrid algorithms are quite successful since they combine both algorithms’ powerful sides. Since PSO is quite fast algorithm, nearly all newly developed algorithm combined with PSO. Some of the recent studies using swarm intelligence are crow search algorithm (CSA) [27], ant lion optimizer (ALO) [28], the whale optimization algorithm (WOA) [29], grey wolf optimizer (GWO) [30], monarch butterfly optimization (MBO) [31], moth flame optimization [32], selfish herd optimization (SHO) [33], and salp swarm optimization (SSO) [34]. Since the algorithms stated in the following paragraph are quite new, according to the Web of Science records, there is not any hybrid study combining PSO with them.

Firefly algorithm (FFA) [35], bacterial foraging optimization algorithm (BFOA) [36], ant colony optimization (ACO) [37], artificial bee colony (ABC) [38], and cuckoo search (CS) [39] are some of the algorithms using swarm intelligence improved in the last decade. There are hybrid versions of these algorithms with PSO.

3.1. Firefly algorithm

FFA is improved by mimicking the flashing activity of fireflies. FFA is similar most of the swarm intelligence algorithm. Fireflies are located in a position in the solution space randomly initially. The fitness of the fireflies is calculated according to the light intensity. The next location of each firefly is calculated according to the current position, randomness and attractiveness. The hybrid algorithm combined PSO with firefly optimization [40] proposes a technique for the detection of Bundle Branch Block (BBB), one of the abnormal cardiac beat, using hybrid firefly and particle swarm optimization (FFPSO) technique in combination with Levenberg Marquardt Neural Network (LMNN) classifier.

3.2. Bacterial foraging optimization algorithm

BFOA is inspired by the social foraging behavior of Escherichia coli. BFOA is an efficient algorithm in solving real-world optimization problems. Chemotaxis process simulates the movement of an E. coli cell through swimming and tumbling via flagella. E. coli cells like particles move in solution space and change their location with a formula-dependent previous location, randomness and chemotactic step size. The big difference between PSO is that not only global best, but also global worst is also evaluated in the algorithm. Among E. coli cells, the least healthy one, eventually die. Each of the healthier bacteria splits into two bacteria, which are then placed in the same location. This keeps the swarm size constant. The hybrid algorithm using PSO and BFOA optimized PI controller, for multiobjective load frequency [41]. The authors developed a hybrid PSO with firefly, also developed a hybrid PSO with BFOA, for the detection of BBB. The same classifier is used in the study [42].

3.3. Ant colony optimization

ACO is inspired from the pheromone trails of ants. The first version improved by Dorigo is called ant system [43]. Although most of the ant species are blind, they can find the shortest path from their nest to food source using swarm intelligence. Ants are located in random positions in the solution space and moves with pheromone trail and randomness. In the first iterations, ants move stochastically. Eventually, pheromone increases in the path used most because ants prefer the path that contains more pheromone. This means that “Trace me.” In order not to get trapped to the local convergence, pheromone evaporates related to time. ACO is generally used for the solution of combinatorial optimization problem solution such as travelling salesman problem (TSP) and some network problems. Hybrid PSO with ACO [44] solves routing problem.

3.4. Artificial bee colony

ABC optimization is developed by Karaboga in 2005. ABC is also a swarm intelligence algorithm based on the foraging behavior of honey bee swarms. The artificial bee colonies in the ABC algorithm consists of three groups: employed bees, onlookers and scouts. Employed bees search for food source and sharing this information to recruit onlooker bees. Onlooker bees select better food sources from those employed bees and further search around the selected food source. If a food source is not improved by some iteration, this employed bee will become a scout bee to search randomly for new food sources. In [45], a hybrid algorithm is developed combining PSO and ABC. Since PSO is fast convergent algorithm and ABC is slow convergent algorithm, hybrid algorithm uses the powerful sides of each algorithm.

3.5. Cuckoo search

CS is also another swarm intelligence algorithm inspired from cuckoos. CS is based on the interesting breeding behavior such as brood parasitism of certain species of cuckoos in combination with L’evy flight behavior of some birds. CS is successful for finding optimum values of multimodal functions. A hybrid algorithm is proposed finding for optimal design of multiband stop filters [46].

4. Conclusion

After GA, PSO is the first and foremost algorithm which is the most successful algorithm especially for the solution of the continuous optimization problems. Subsequent algorithms using swarm intelligence nearly have the same ideas. All of them are population based, and all particles are distributed in the solution space. Particles have initial locations and velocities. They converge to the optimum solution using their swarm intelligence.

Although there have been improvements in the optimization algorithms using swarm intelligence, there is not a unique algorithm which is successful in all types of optimization problems. So the efforts trying to simulate the animal behaviors and swarm intelligence will continue. At the same time, developing hybrid algorithms will also continue until the best combination of the algorithms would be found.