Particle Swarm Optimization

Abstract

The procedure is to obtain the best solution for the certain parameters in the given network to satisfy every requirement for design by considering the smallest affordable cost will be considered as an optimization. Optimization traditional approach will have some constraints like, outcome of single-based, local optima convergence, problems in unknown search space. To overcome the above constraints, many research organizations have established various metaheuristics to search optimization solutions for unsolved issues. The main intend to explain the Particle Swarm Optimization algorithm (PSOA) is to explain the stochastic optimization approach basics. Motivation of this Particle Swarm Optimization algorithm (PSOA) is to develop a strong metaheuristic optimization solution which is inspired natural swarm behavior like schooling of birds and fishes. PSOA is a simplified social network simulation. The final intent of this PSOA is a graphical representation and graphical simulate smoothly but undefined bird or fish flock’s directions. Every bird’s vicinity of observability is restricted to some area. Though having many birds permits every bird in the swarm fitness function to be bigger surface concerned. Mathematically every Particle Swarm Optimization algorithm (PSOA) has associated with fitness value, velocity, and position. Memory of maintaining global fitness, best position, and global fitness value.

1. Introduction

The optimization is having great impact on nature and human law of affairs. The characteristic of this optimization is to obtain the most superior (global minima or maxima). The optimization structure should maintain the generality. Hence the optimization design study should be obtaining the best of activity of human aspects and analyzing, understanding, solution and evaluating of mathematical issues of the system. This optimization issues are including some real problems constraints and these constraints must be satisfied to get the most feasible global solution. Many constraints are required in designing this optimization solution in real world practice.

The main objective this optimization is to obtain the best with the small effort. For example, in economic operation of electrical power system, with the minimum cost of operation obtaining the maximum efficiency. This is also applicable for many more fields in the real world.

2. Mathematical formulation

Generally, all the optimization design having the following steps:

1. Input data

2. Initialization

3. Constraints

4. Evaluation

5. Updating

6. Storing Result

The design of optimization will be expressed in a standard form as follows:

$$\mathit{\text{Objective Function}}=f\left(X\right)=\mathit{\text{maximizing or minimizing}}\left(\mathit{\text{cost}}\right)$$E1

Subjected to

$$\begin{array}{c}\mathit{\text{Inequality Constraints for}}\\ \mathit{\text{minimizing objective function}}\left(\mathit{\text{pnumber}}\right)a_z\left(X\right)\le 0,\mathit{\text{where z}}=1,2,3,\dots ,p\end{array}$$E2

$$\begin{array}{c}\mathit{\text{Inequality Constraints for}}\\ \mathit{\text{maximizing objective function}}\left(\mathit{\text{p number}}\right)a_z\left(X\right)\ge 0,\mathit{\text{where z}}=1,2,3,\dots ,p\end{array}$$E3

$$\mathit{\text{Equality Constraints}}\left(\mathit{\text{q number}}\right)b_z\left(X\right)=0,\mathit{\text{where z}}=1,2,3,\dots ,q$$E4

$$\mathit{\text{Input or design varibales defined as}}{x}_i,\mathit{\text{where i}}=1,2,3,\dots ,n$$E5

$$\text{Input or design variable expressed}\mathrm{as}X=\left\{\begin{array}{c}{x}_1\\ x_2\\ x_3\\ .\\ .\\ .\\ x_n\end{array}\right\}$$E6

The design variables can also be expressed as $x_i^l=x_i=x_i^u$. Here $x_i^l$ is the lower limit and $x_i^u$ is the upper limit.

3. Present best and global best optimization

If design or Input variables that give the optimization like maximization or minimization of objective function f(X) can be represented with X^* (Figure 1).

Figure 1.

Local minima and global minima [1].

Pints P1, P2, P3, P4, P5, P6 are known as Present best or Local best solutions and the P7 is known as the Global best solution for the given system.

Objective of this given system has a present or local best solution at X^* point, then

$$f\left(X^{*}\right)\le f\left[X\right]\text{for every}\mathrm{X}\text{value and in single variable of}\mathrm{x}\text{for}\mathrm{a}\text{closed region of}a\le x\le b.$$

4. Various optimization techniques

In the literature we have different types of mathematical approaches for optimization are found like Simplex approach, gradient approach, dynamic programming, branch and bound and integer programming etc. for all these approaches the size will be limited to solve. These approaches are more efficient in solving linear issues. With the increase in variables number and constraints number, the time of computation is also increases to obtain the solution. This increase in time increases exponentially. With the increase in complexity in solving complex issues with the help of mathematical approaches is also become more and more complex.

In last few years number of meta-heuristic approaches has developed to sole such complex issues with limited flexibility. Many numbers of bio and nature inspired optimization approaches has been developed like Genetic Algorithm (GA) and Swarm Intelligence (SI) approaches.

Here GA will work on the Darwinian principle for survival of the best solution in the population. GA uses the selection, cross-over and the mutation etc., as the operators to evaluate the population. Proved that, GA will give the closest solution for the global optimization with the introducing required improvements into it. Furthermore, the Differential Evaluation (DE) approach has explored to reach the global optimal solution.

Taking an Inspiration from living organism social behavior like fishes, birds, inspects etc. these can be communicate with each other might be directly or indirectly with a set of patterns used for the individual optimization. These approaches will operate on the cooperation of organisms instead of competing among them. All these exchange of information from one to other.

Here the Particle Swarm Optimization (PSO) is from the nature inspired behavior of food searching behavior of fishes and birds flocking behavior. To obtain the present best or local optimal solution and the global best or global optimal solution the fishes and the birds will be considered as particles.

As said previously PSO is the stochastic optimization method on swarm based. The finding the food is with the cooperative way in these swarms. Every particle (member) in this swarm ready to modify their pattern for search based on experience from learning from other particles (members) or by own.

Velocity and the position of particles will update in PSO based on the change in environment to reach proximity and the quality requirement. Here the proximity is the swarms must carry time and space calculations and the quality is swarms have to find the appropriate environmental change and have to respond.

According to cornfield model proposed by Heppner on the birds or fish flock behavior, let us assume a food location in the plane. Birds will randomly move or search for the food initially. Now (x₀, y₀) are the coordinates of position of cornfield. The individual birds coordinate for position and the velocity can be considered as (x, y) and (v_x, v_y) respectively. Distance from cornfield and the current position is used to determine the performance of the speed and the current position. If each particle has its own memory and can be able to memorize the best position it reached.

This present best position is represented with P_best (local best). Adjustment of velocity denoted with a constant ‘a’, random numbers between [0, 1] denoted with ‘rand’. The below mathematical equations used to change the velocity as.

If P_best x < x, then

V_x = V_x – (a _* rand)

else

V_x = V_x + (a _* rand)

end

If P_best y < y

V_y = V_y – (a _* rand)

else

V_y = V_y + (a _* rand)

end

Now if there is a communication between swarms in some other way, every individual able to memorize their best location, considering global best position (gbest) of entire swarm.

Now if the constant for the velocity adjustment is denoted with ‘b’, by using above mathematical equations if the velocity adjusted, Gbest is also changed (or) updated as.

If gbest < x,

V_x = V_x - (b _* rand)

else

V_x = V_x + (b _* rand)

end.

If gbest < y

V_y = V_y - (b _* rand)

else

V_y = V_y + (b _* rand)

end

The final fixed velocity and position updating after some trial and errors as.

Velocity updating:

$$V_x=V_x+2\ast \left(\mathit{\text{pbestx}}-x\right)\ast \mathit{rand}+2\ast \left(\mathit{\text{gbestx}}-x\right)\ast \mathit{rand}$$E7

Position updating:

$$x=x+V_x$$E8

Here every individual is considered as a particle and these particles are not having any mass and volume. But these particles are having position as well as velocity. Hence this optimization is known as ‘Particle Swarm Optimization Algorithm’.

This PSO algorithm can be explained using the following flowchart (Figure 2).

Figure 2.

The Flowchart of the PSOA [2].

From the flowchart we assume that ‘N’ is the size of swarm. Position vector of every individual particle in D - dimensional vector space is

$$X_i=\left(X_{i1},X_{i2},X_{i3},,\dots ,X_{\mathit{id}},\dots ,X_{\mathit{iD}}\right)$$E9

Velocity vector,

$$V_i=\left(V_{i1},V_{i2},V_{i3},,\dots ,V_{\mathit{id}},\dots ,V_{\mathit{iD}}\right)$$E10

Present best (or) local best position of individual particle is,

$$P_i=\left(P_{i1},P_{i2},P_{i3},,\dots ,P_{\mathit{id}},\dots ,P_{\mathit{iD}}\right)$$E11

Optimal (or) the best global position of individual particle

$$P_g=\left(P_{g1},P_{g2},P_{g3},,\dots ,P_{\mathit{gd}},\dots ,P_{\mathit{gD}}\right)$$E12

Now without generality loss, by considering minimization is an objective function, the mathematical equation for updating of individual particle optimal position as,

$$\mathit{\text{if}}f\left(P_i\right)>f\left(x_i\left(t+\mathit{1}\right)\right)P_i^d,\left(t+1\right)=x_i^d\mathit{\text{else}}$$

$$P_i^d\left(t+1\right)=P_i^d$$

Now the velocity and position formula are given as,

$$V_i^d\left(t+1\right)=V_i^d\left(t\right)+C_1\ast \left(P_i^d\left(t\right)-x_i^d\left(t\right)\right)\ast \mathit{rand}+C_1\ast \left(P_i^d\left(t\right)-x_i^d\left(t\right)\right)\ast \mathit{rand}$$E13

$$x_i^d\left(t+1\right)=x_i^d\left(t\right)+V_i^d\left(t+1\right)$$E14

This proposed PSO was good enough for optimization. But later this PSO was modified to improve the effectiveness by adding Inertia weight. With Inertia weight introduction, the velocity update equation modified as,

$$V_i^d\left(t+1\right)=\omega \ast V_i^d\left(t\right)+C_1\ast \left(P_i^d\left(t\right)-x_i^d\left(t\right)\right)\ast \mathit{rand}+C_1\ast \left(P_i^d\left(t\right)-x_i^d\left(t\right)\right)\ast \mathit{rand}$$E15

Now by taking the convergence rate into an account, the construction factor (x) is introduced into PSO. Now the new update equation for velocity is became,

$$V_i^d\left(t+1\right)=x\left(V|i^d\left(t\right)+C_1\ast \left(P_i^d\left(t\right)-x_i^d\left(t\right)\right)\ast \mathit{rand}+C_1\ast \left(P_i^d\left(t\right)-x_i^d\left(t\right)\right)\ast \mathit{rand}\right)$$E16

Here the distance from the current position of the particle to its own best position called ‘Cognitive’ i.e., own thinking. Hence ‘C₁’ is called cognitive acceleration factor (or) cognitive learning factor.

Next the distance from current position of the particle to the global best position It is known as ‘Social’ factor. This indicates that there is a coordination and sharing of information between particles. Good particles movement through the cognition. Hence ‘C₂’ is the social acceleration factor (or) social learning factor.

Theory and practical application of this PSO algorithm have found great progress. This PSO used in various domains by every researcher with the understanding principle of operation and application.

PSO algorithm does not required any continuous derivative and differential optimized functions. This has fast rate of convergence. This PSO is very easy to understand, execute using programming.

Now coming to disadvantages: multiple local extreme functions, PSO may local minima and cannot produce optimal result because premature convergence of particle. PSO may not produce good results because of lack of best search techniques because of not using information sufficiently in procedure of calculations. In every Iteration PSO using only local optima Information which may not produce correct results. This PSO can be able to provide global search possibility but cannot guarantee for convergence to the global test. This PSO more suitable for optimization Issues s class with high dimensional and getting accurate outcome is not required because PSO is a meta-heuristic optimization algorithm. It never gives the correct explanation from the principle why it is efficient and not specified any of application range.

5. Various types of research about PSO

1. PSO algorithm can be analyzed theoretically and understanding of its working mechanism.

2. Structure change and obtain better performance.

3. Different parameters influence on PSO.

4. Different topology influence on PSO.

5. Parallel study of PSO algorithm.

6. Discrete study of PSO algorithm.

7. Multi-objective optimization study of PSO algorithm.

8. Use PSO algorithm for various fields in Engineering.

5.1 PSO for multi objective optimization

The single objective optimization is not correct practically Because the effective outcome is not accurate in single objective optimization. This problem has overcome by multi objective (MO) optimization. This MO has become the latest area in research. In this multi-objective optimization issues, all the target functions independently optimized and finally determine the best value for every target. But there is a conflict between objects. Because of conflicting between objects, unfortunately the finding of optimal solution is highly impossible for every objective hence only a pareto perfect solution has been determined.

Particles are independent agents in traditional PSO. Based on their own companion and its own experience problem space can be search by particles. As given earlier formula for particles update cognitive is the former and the social part is the latter. Here, selection of gbest and pbest (social and cognitive guide) is the important issue of Multi-Objective PSO (MOPSO). In both, i.e., in traditional PSO and MOPSO, choosing cognitive guide is same. But the guide must be found based on pareto dominance. There are two steps involved in choosing social guide.

Step-1: Candidate pool creation used for the guide selection.

In PSO traditional, this guide has been chosen from pbest (or) local (or) pbest of neighbors. While in case of MOPSO, to save more pareto optimal solutions, the normal technique is using an external pool.

Step 2: Guide Selection: Choosing of gbest must satisfy the below standards.

The chosen guide should be capable of provide particles guidance effectively. This is because of improve the speed of convergence.

The selected guide required to give balanced search with pareto frontier. This is because to maintain population diversity.

To select social guide here two methods are specified.

1. Quantity standard

2. Roulette mode of selection

3. Quantity standard: Selection of social guide by following various procedures, but not random selection involvement.

4. Roulette mode of selection: Random selection based on various standards, in order to maintain population diversity.

Pseudo code for PSO algorithm:

Initialize population

for t = 1: maximum generation

 for i = 1: population size

  if f (x_i,_d (t)) < f (p_i (t)) then p_i (t) = x_i,_d (t)

   f (p_g (t)) = min (f (p_i (t)))

  end

  for d = 1: dimension

  v_i,_d (t + 1) = w_vi,_d (t) + C_1* r₁(p_i – x_i,_d (t)) + C_2* r₂ (p_g – x_i,_d (t))

  x_i,_d (t + 1) = x_i,_d (t) + v_i,_d (t + 1)

  if v_i,_d (t + 1) > v_max then v_i,_d (t + 1) = v_max

  else if v_i,_d (t + 1) < v_min then v_i,_d (t + 1) = v_min

  end

  if x_i,_d (t + 1) > x_max then x_i, d(t + 1) = x_max

  else if x_i,_d (t + 1) < x_min then x_i,_d (t + 1) = x_min

  end

 end

 end

end

For each particle

  Initialize particle

END

Do

 For each particle

   Calculate fitness value

   If the fitness value is better than the best fitness value (pbest) in history set current value

    as the new pbest

 End

Choose the particle with the best fitness value of all the particles as the gbest

For each particle

Calculate particle velocity

Update particle position

End

While maximum iterations or minimum error criteria is not attained (Table 1).

PSO Basic Variant	Function	Advantages	Disadvantages
Velocity Clamping (VC)	Control the global exploration of the particle.; Reduces the size of the step velocity, so that the particles remain in the search area, but it cannot change the search direction of the particle.	VC reduces the size of the step velocity so it will control the movement of the particle	If all the velocity becomes equal to the particle will continue to conduct searches within a hypercube and will probably remain in the optima but will not converge in the local area.
Inertia Weight	Controls the momentum of the particle by weighing the contribution of the previous velocity	A larger inertia weight in the end of search will foster the convergence ability.	Achieve optimally convergence strongly influenced by the inertia weight
Construction Coefficient	To ensure the stable convergence of the PSO algorithm	Similar with inertia weight	When the algorithm converges, the fixed values of the parameters might cause the unnecessary fluctuation of particles
Synchronous and Asynchronous Updates	Optimization in parallel processing	Improved convergence rate	Higher throughput: More sophisticated finite element formulations Higher accuracy (mesh densities)

Table 1.

The basic variant of PSO [3].

The PSO algorithm can be hybridized with several metaheuristic algorithm to balance the exploration and exploitation. Some algorithms have better efficiency in exploration, but they are poor in exploitation. Some algorithms will take high iteration to reach convergence and show poor performance. In this case the PSO is introduced to enhance the performance using,

$$V_i\left(K+1\right)={\mathit{\omega V}}_i\left(K\right)+{\rho }_i\ast C_1\left(P_i^{\mathit{\text{best}}}-X_i\left(K\right)\right)+{\rho }_2\ast C_2\left({\mathit{GX}}_i\left(K\right)\right)$$E17

$$X_i\left(K+1\right)=X_i\left(K\right)+V_i\left(K+1\right)$$E18

The proposed algorithm using PSO is written as,

1. Define population size (P), Initial values, C₁ and C₂ as acceleration constants, P_c the number of variables, and maximum iteration number as Max_iter.

2. Setting t = 0, as counter initialization.

3. for i = 1: if P ≥ i, do

4. Generate an initial random population X_i(t).

5. Fitness evaluation function of every search agent or solution f(X_i).

6. end for

7. repeat

8. Apply standard original Particle Swarm Optimization (PSO) Algorithm.

9. Apply the selection operate of another algorithm which are going to hybridize with PSO. (Ex: Genetic Algorithm (GA), Ant-Lion Optimization Algorithm (ALOA) etc.)

10. Partition the population X(t) into in to part_no sub-partitions, where each sub-partition $X^{\prime }\left(t\right)$ size is v x η.

11. for i = 1: part_no ≥ i do

12. Apply the arithmetical crossover as shown in procedure 1 on each sub-partition $X^{\prime }\left(t\right)$.

13. end for

14. Apply the algorithm which is hybridized with PSO on the whole population X(t).

15. Update the solution in the population X(t).

16. Set t = t + 1. Increase the iteration count.

17. Until iteration reaches the Max_iter. Satisfied the termination criteria.

18. Print the best solution (Figure 3).

Figure 3.

The flowchart of the hybridization with PSOA [2].

PSOA will be applied in various optimization areas, example, Energy-Storage Optimization, Image Processing, Economic operation of power system, Optimal location identification, analysis of slope stability, foundation and pile engineering, soil, and rock mechanics, underground and tunneling space design etc. PSOA will simulate the particle movement and will be applied in visual effects and special effects in some films.

Along with the conventional applications optimization problems, Swarm Intelligence (SI) will be used in acquisition of material from library, communications, classification of medical dataset, dynamic control, planning of heating system, tracking of moving objects, and prediction.

PSO Algorithm Visual Explanation:

Let us suppose a group of birds are flying randomly in an area in search of their food (Figure 4).

Figure 4.

Random movement of birds [4].

No bird knows exactly where the food is, but they know how far they are in each iteration, (Figure 5).

Figure 5.

Birds moving toward the food [4].

Birds they do not know the best position. If any member can find the desirable path to go, the rest of the members will follow quickly (Figure 6).

Figure 6.

Finding best path [4].

Finally, following the bird which is nearest to the food is an aim.

5.2 PSO algorithm demo with one example

Objective function used for Minimization:

$$\mathit{\text{Fitness}}=10\ast \left(X_1-1\right){}^{^}2+20\ast \left(X_2-2\right){}^{^}2+30\ast \left(X_3-3\right){}^{^}2$$

STEP – 1: PARAMETERS INITIALIZATION (Table 2):

8	9	1
9	6	1
3	5	10
10	2	10
10	5	8

Table 2.

Output [5].

   Number of variables: m = 3.

   Population size: n = 5.

   Inertia weight: Wmax = 0.9, W_min = 0.4.

   Acceleration Factor: C₁ = 2, C₂ = 2.

   Maximum Iteration: Max_iter = 50.

% Population Initialization

   LB = [0 0 0];

   UB = [10 10 10];

for i = 1: n

    for j = 1:m

    X₀ (i, j) = round (LB (j) + rand () _* (UB (j) – LB (j)));

    end

end

• Initialize velocity (v_i) randomly for each particle: (Table 3)

• v = 0.1 _* X₀ (i, j);

	X1	X2	X3
1st Particle	8	9	1
2nd Particle	9	6	1
3rd Particle	3	5	10
4th Particle	10	2	10
5th Particle	10	5	8

Table 3.

Initial population [5].

Initial velocity for First Particle:

v(X₁) = 0.1 _* 8 = 0.8

v(X₃) = 0.1 _* 9 = 0.9

v(X₁) = 0.1 _* 1 = 0.1

Initial velocity for Second Particle:

v(X₁) = 0.1 _* 9 = 0.9

v(X₃) = 0.1 _* 6 = 0.6

v(X₁) = 0.1 _* 1 = 0.1

Initial velocity for Third Particle:

v(X₁) = 0.1 _* 3 = 0.3

v(X₃) = 0.1 _* 5 = 0.5

v(X₁) = 0.1 _* 10 = 1.0

Initial velocity for Fourth Particle:

v(X₁) = 0.1 _* 10 = 1.0

v(X₃) = 0.1 _* 2 = 0.22

v(X₁) = 0.1 _* 10 = 1.0

Initial velocity for Fifth Particle:

v(X₁) = 0.1 _* 10 = 1.0

v(X₃) = 0.1 _* 5 = 0.5

v(X₁) = 0.1 _* 8 = 0.8

Initial Position (Xi) Randomly for each Particles (Table 4):

	X1	X2	X3
1st Particle	0.8	0.9	0.1
2nd Particle	0.9	0.6	0.1
3rd Particle	0.3	0.5	1.0
4th Particle	1.0	0.2	1.0
5th Particle	1.0	0.5	0.8

Table 4.

Initial velocity [5].

Current Position = Previous Position + Velocity

Current Position for First Particle:

X₁ = 8 + 0.8 = 8.8

X₂ = 9 + 0.9 = 9.9

X₃ = 1 + 0.1 = 1.1

Current Position for Second Particle:

X₁ = 9 + 0.9 = 9.9

X₂ = 6 + 0.6 = 6.6

X₃ = 1 + 0.1 = 1.1

Current Position for Third Particle:

X₁ = 3 + 0.3 = 3.3

X₂ = 5 + 0.5 = 5.5

X₃ = 10 + 1 = 11

Current Position for Fourth Particle:

X₁ = 10 + 1 = 11

X₂ = 2 + 0.2 = 2.2

X₃ = 10 + 1 = 11

Current Position for Fifth Particle:

X₁ = 10 + 1 = 11

X₂ = 5 + 0.5 = 5.5

X₃ = 8 + 0.8 = 8.8

STEP – 2: FITNESS EVALUATION $f\left(x_i^t\right)$ (Table 5)

	X1	X2	X3
1st Particle	8.8	9.9	1.1
2nd Particle	9.9	6.6	1.1
3rd Particle	3.3	5.5	11
4th Particle	11	2.2	11
5th Particle	11	5.5	8.8

Table 5.

Current position [5].

Fitness Evaluation for Every Particle:

The objective function taken for minimization:

$$F\left(x\right)=10\ast {\left(X_1-1\right)}^{^}2+20\ast {\left(X_2-2\right)}^{^}2+30\ast {\left(X_3-3\right)}^{^}2$$

$$F\left(x_1^0\right)=10\ast {\left(8.8-1\right)}^{^}2+20\ast {\left(9.9-2\right)}^{^}2+30\ast {\left(1.1-3\right)}^{^}2$$

$$F\left(x_1^0\right)=1.9649$$

$$F\left(x_2^0\right)=10\ast {\left(9.9-1\right)}^{^}2+20\ast {\left(6.6-2\right)}^{^}2+30\ast {\left(1.1-3\right)}^{^}2$$

$$F\left(x_2^0\right)=1.3236$$

$$F\left(x_3^0\right)=10\ast {\left(3.3-1\right)}^{^}2+20\ast {\left(5.5-2\right)}^{^}2+30\ast {\left(11-3\right)}^{^}2$$

$$F\left(x_3^0\right)=2.2179$$

$$F\left(x_4^0\right)=10\ast {\left(11-1\right)}^{^}2+20\ast {\left(2.2-2\right)}^{^}2+30\ast {\left(11-3\right)}^{^}2$$

$$F\left(x_4^0\right)=2.9208$$

$$F\left(x_5^0\right)=10\ast {\left(11-1\right)}^{^}2+20\ast {\left(5.5-2\right)}^{^}2+30\ast {\left(8.8-3\right)}^{^}2$$

$$F\left(x_5^0\right)=2.2542$$

g_Best = 1.3236 (Table 6)

FITNESS VALUE (t = 0)
1.9649
1.3236
2.2179
2.9208
2.2542

Table 6.

Choose the particle with best fitness value as g_Best [5].

STEP – 3: CALCULATION OF POSITION AND VELOCITY FOR EVERY PARTICLE

• Particle Position Calculation: $x_i^{t+1}=x_i^t+v_i^t$

• Particle Velocity Calculation:

  $$v_i^{t+1}={\mathit{wv}}_i^t+C_1{r}_1\left({\mathit{\text{xBest}}}_i^t-x_i^t\right)+C_2{r}_2\left({\mathit{\text{gBest}}}_i^t-x_i^t\right)$$

Velocity Update for at iteration (t = 0) for First Particle:

$$v_1^{0+1}={\mathit{wv}}_1^0+C_1{r}_1\left({\mathit{\text{xBest}}}_1^0-x_1^0\right)+C_2{r}_2\left({\mathit{\text{gBest}}}_1^0-x_1^0\right)$$

$$v_1^1=0.9\ast 0.8+2\ast \mathit{rand}\left(\right)\ast \left(8.8-8.8\right)+2\ast \mathit{rand}\left(\right)\ast \left(9.9-8.8\right)$$

$$v_1^1=1.0667$$

Calculating all the values as:

$$v_1^1=1.0667\mathit{\text{for}}{x}_1$$

$$v_1^1=-4.4719\mathit{\text{for}}{x}_2$$

$$v_1^1=0.0900\mathit{\text{for}}{x}_3$$

Particle Position Update: (Table 7)

	X1	X2	X3
1st Particle	1.0667	−4.4719	0.0900
2nd Particle	0.8100	0.5400	0.0900
3rd Particle	7.4888	2.5728	−18.3177
4th Particle	−0.1678	1.4286	1.4286
5th Particle	−1.2109	0.5286	−13.6635

Table 7.

Updated velocity for every particle [5].

Position Update for First Particle:

$$x_1^{0+1}=x_1^0+v_1^0$$

$$x_1^1=8.8+1.0667=9.8667$$

$$x_1^1=9.9+\left(-4.4719\right)=5.4281$$

$$x_1^1=1.1+0.900=2$$

STEP – 4: FITNESS EVALUATION $f\left(x_i^t\right)$ (Table 8)

	X1	X2	X3
1st Particle	9.8667	5.4281	2
2nd Particle	10.71	7.14	1.19
3rd Particle	10.83	8.07	−7.317
4th Particle	10.83	3.628	12.42
5th Particle	9.78	6.028	−4.8635

Table 8.

Updated positions for every particle [5].

Current Best Calculation Best [gBest]

Update Minimization: Calculate Fitness Value:

$$F\left(x\right)=10\ast {\left(X_1-1\right)}^{^}2+20\ast {\left(X_2-2\right)}^{^}2+30\ast {\left(X_3-3\right)}^{^}2$$

$$\begin{array}{c}\mathit{\text{FirstParticle}}\\ F\left(x\right)=10\ast {\left(9.8667-1\right)}^{^}2+20\ast {\left(5.4281-2\right)}^{^}2+30\ast {\left(2-3\right)}^{^}2=1.0512\end{array}$$

$$\begin{array}{c}\mathit{\text{SecondParticle}}\\ F\left(x\right)=10\ast {\left(10.71-1\right)}^{^}2+20\ast {\left(7.14-2\right)}^{^}2+30\ast {\left(1.19-3\right)}^{^}2=1.5695\end{array}$$

$$\begin{array}{c}\mathit{\text{ThirdParticle}}\\ F\left(x\right)=10\ast {\left(10.78-1\right)}^{^}2+20\ast {\left(8.07-2\right)}^{^}2+30\ast {\left(-7.317-3\right)}^{^}2=4.8866\end{array}$$

$$\begin{array}{c}\mathit{\text{FourthParticle}}\\ F\left(x\right)=10\ast {\left(10.78-1\right)}^{^}2+20\ast {\left(8.07-2\right)}^{^}2+30\ast {\left(-7.317-3\right)}^{^}2=3.6716\end{array}$$

$$\begin{array}{c}\mathit{\text{FifthParticle}}\\ F\left(x\right)=10\ast {\left(9.78-1\right)}^{^}2+20\ast {\left(6.028-2\right)}^{^}2+30\ast {\left(-4.8635-3\right)}^{^}2=2.9504\end{array}$$

Choose the Current Best as (Table 9)

Updated Fitness Value
1.0512
1.5695
4.8866
3.6716
2.9504

Table 9.

Updated fitness value [5].

if $f\left(x_1^1\right)<f\left(\mathit{\text{gBest}}\right)\mathit{\text{then}}$

$$\mathit{\text{gBest}}=x_1^1$$

if $f\left(x_1^1\right)<f\left(\mathit{\text{gBest}}\right)$

1.0512 < 1.3236

if $f\left(x_1^1\right)<f\left(\mathit{\text{gBest}}\right)\mathit{\text{then}}$

$$\mathit{\text{gBest}}=1.0512$$

if $f\left(x_2^1\right)<f\left(\mathit{\text{gBest}}\right)$

1.5695 < 1.3236

if $f\left(x_3^1\right)<f\left(\mathit{\text{gBest}}\right)$

4.8866 < 1.3236

if $f\left(x_4^1\right)<f\left(\mathit{\text{gBest}}\right)$

3.6716 < 1.3236

if $f\left(x_5^1\right)<f\left(\mathit{\text{gBest}}\right)$

2.9504 < 1.3236

gBest = 1.0512

if $f\left(x_i^t\right)<f\left(\mathit{\text{pBest}}\right)\mathit{\text{then}}$

pBest = $x_i^t$

if $f\left(x_1^1\right)<f\left(\mathit{\text{pBest}}\right)$

1.0512 < 1.9649

if $f\left(x_2^1\right)<f\left(\mathit{\text{pBest}}\right)$

1.5695 < 1.3236

if $f\left(x_3^1\right)<f\left(\mathit{\text{pBest}}\right)$

4.8866 < 2.2179

if $f\left(x_4^1\right)<f\left(\mathit{\text{pBest}}\right)$

3.6716 < 2.9208

if $f\left(x_5^1\right)<f\left(\mathit{\text{pBest}}\right)$

2.9504 < 2.2542

STEP – 5: UPDATE ITERATION (Table 10)

New pBest Value
1.0512
1.3236
7.2179
2.9208
2.2542

Table 10.

Updated fitness value [5].

Update t = t + 1 = 2

t = 2

STEP – 6:

gBest = 1.0512

Repeat this until stopping Criteria met (Table 11)

1st Particle

9.8667

5.4281

2

Table 11.

OUTPUT gBest & $x_i^t$ [5].

5.3 PSO algorithm demo with one more example

Find the minimum of the function using PSO algorithm.

$$f\left(x\right)=-x^2+5x+20$$

Using 9 particles with initial positions.

$$\begin{array}{c}{x}_1=-9.6,x_2=-6,x_3=-2.6,x_4=-1.1,x_5=0.6,x_6=2.3,x_7=2.8,x_8=8.3,\\ x_9=-10,\end{array}$$

Step 1: Evaluation of objective function as

$$\begin{array}{c}{f}_1^0=-120.16,f_2^0=-46,f_3^0=0.24,f_4^0=13.29,f_5^0=22.64,f_6^0=26.21,f_7^0=26.16,\\ f_8^0=-7.39,f_9^0=-30\end{array}$$

Let C1 = C2 = 1 and set initial velocities of the particles to zero.

$$v_1^0=0,v_1^0=v_2^0,v_3^0,v_4^0,v_5^0,v_6^0,v_7^0,v_8^0,v_9^0=0$$

Step 2: Set the iteration number as t = 0 + 1 and go to step 3.

Step 3: Find the P_best for every particle.

$$P_{\mathit{\text{best}},i}^{t+1}=\left\{\begin{array}{c}{P}_{\mathit{\text{best}},i}^t\mathit{\text{if}}{f}_i^{t+1}>P_{\mathit{\text{best}},i}^t\\ x_i^{t+1}\mathit{\text{if}}{f}_i^{t+1}\le P_{\mathit{\text{best}},i}^t\end{array}\right.$$

So,

$$P_{\mathit{\text{best}},1}^1=-9.6,P_{\mathit{\text{best}},2}^1=-6,P_{\mathit{\text{best}},3}^1=-2.6,P_{\mathit{\text{best}},4}^1=-1.1,P_{\mathit{\text{best}},5}^1=0.6,$$

$$P_{\mathit{\text{best}},6}^1=2.3,P_{\mathit{\text{best}},7}^1=2.8,P_{\mathit{\text{best}},8}^1=8.3,P_{\mathit{\text{best}},9}^1=10$$

Step 4: Gbest = max(Pbest) so Gbest = 2.3.

Step 5: Updating the velocities of the particle by considering the value of random numbers r1 =0.213, r2 = 0.876, C1 = C2 = 1, w = 1.

$$v_i^{t+1}=v_i^t+C1r_1^t\left[P_{\mathit{\text{best}},i}^t-x_i^t\right]+C2r_2^t\left[G_{\mathit{\text{best}}}^t-x_i^t\right];i=1,2,3,\dots ,9.$$

$$v_1^1=0+0.213\left(-9.6+9.6\right)+0.876\left(2.3+9.6\right)=10.4244$$

$$v_2^1=7.2708,v_3^1=4.2924,v_5^1=1.4892,v_6^1=0,v_7^1=-0.4380,v_8^1=5.256,v_9^1=-6.7452$$

Step 6: Update the values of position as well.

$$x_i^{t+1}=x_i^t+v_i^{t+1}$$

$$\begin{array}{c}{x}_1^1=0.8244,x_2^1=1.2708,x_3^1=1.6924,x_4^1=1.8784,x_5^1=2.0892,x_6^1=2.3,\\ x_7^1=2.362,x_8^1=3.044,x_9^1=3.2548\end{array}$$

Step 7: Finding objective function values of

$$\begin{array}{c}{f}_1^1=23.4424,f_2^1=24.739,f_3^1=25.5978,f_4^1=25.8636,f_5^1=26.0812,f_6^1=26.21,\\ f_7^1=26.231,f_8^1=25.9541,f_9^1=25.6803\end{array}$$

Step 8: Stopping Criteria.

If the terminal rule is satisfied, go to step 2. Otherwise stop the iteration and note the result.

PSO Algorithm Demo with nature inspired hybrid algorithm:

The optimal integration of Distributed Generation (DG) would be investigated by determining the reduced power loss, enhanced voltage at every bus, reduced total operating cost and enhanced voltage stability of the test network.

In this section, the candidate busses for DG placement and the capacity of the DGs are estimated with the help of Ant-Lion Optimization Algorithm (ALOA) methodology.

Later the elitism phase of the ALOA methodology for updating the DGs location will be done by introducing Particle Swam Optimization Algorithm (PSOA) in ALOA.

The ALOA imitates the ant-lion’s hunting mechanism. In ant-lion’s life cycle there are two main stages, namely, Larva stage or phase and adult stage. The antlion using its larva stage to find the food and the adult stage for reproduction. The inspiration for this ALOA is the larva stage. The antlion prepare one pit by digging in the sand in the shape of cone by moving in a circular way and throwing the sand out of the pit with its heavy jaw. After preparation of trap the antlion will wait for food. The level of hungry of an antlion and the size of the moon decides the trap size. If any pray come toward the surface of the trap, that will fall into the trap very easily. Then the antlion will catch the pray, when it is identified by the antlion.

The matrices M_ant-lion and M_ant stores the random places of the antlions and ants respectively,

$$\begin{array}{l}{M}_{\mathit{ant}-\mathit{\text{lion}}}=\left(\begin{array}{cccc}{\mathit{AL}}_{1,1}& {\mathit{AL}}_{1,2}& \dots & {\mathit{AL}}_{1,n}\\ {\mathit{AL}}_{2,1}& {\mathit{AL}}_{2,2}& \dots & {\mathit{AL}}_{2,d}\\ ⋮& ⋮& ⋮& ⋮\\ {\mathit{AL}}_{n,1}& {\mathit{AL}}_{n,2}& \dots & {\mathit{AL}}_{n,d}\end{array}\right)\\ \\ M_{\mathit{ant}}=\left(\begin{array}{cccc}{A}_{1,1}& A_{1,2}& \dots & A_{1,n}\\ A_{2,1}& A_{2,2}& \dots & A_{2,d}\\ ⋮& ⋮& ⋮& ⋮\\ A_{n,1}& A_{n,2}& \dots & A_{n,d}\end{array}\right)\end{array}$$E19

Antlion’s and ant’s fitness will be stored in M_fit-AL and M_fit-A matrices,

$$\begin{array}{l}{M}_{\mathit{fit}-\mathit{AL}}=\left(\begin{array}{c}f\left(\left[{\mathit{AL}}_{1,1},{\mathit{AL}}_{1,2},\dots ,{\mathit{AL}}_{1,n}\right]\right)\\ f\left(\left[{\mathit{AL}}_{2,1},{\mathit{AL}}_{2,2},\dots ,{\mathit{AL}}_{2,d}\right]\right)\\ ⋮\\ f\left(\left[{\mathit{AL}}_{n,1},{\mathit{AL}}_{n,2},\dots ,{\mathit{AL}}_{n,d}\right]\right)\end{array}\right)\\ \\ M_{\mathit{fit}-A}=\left(\begin{array}{c}f\left(\left[A_{1,1},A_{1,2},\dots ,A_{1,n}\right]\right)\\ f\left(\left[A_{2,1},A_{2,2},\dots ,A_{2,d}\right]\right)\\ ⋮\\ f\left(\left[A_{n,1},A_{n,2},\dots ,A_{n,d}\right]\right)\end{array}\right)\end{array}$$E20

5.3.1 Ant’s random movement

When ants are moving randomly in search space, then the antlions are ready to find them. All the ants randomly move in search space to find their food is given as,

$$\begin{array}{l}\mathit{ant}\mathit{\text{position}}=[0,\mathit{\text{cumulative}}\mathit{sum}\left(2^{\ast }\mathit{\text{random number}}-1\right),\dots \\ \mathit{\text{till}}max\mathit{\text{imum iteration}}]\end{array}$$E21

$$\mathit{\text{random number}}=\left\{\begin{array}{c}1\mathit{\text{if}}\mathit{rand}>0.5\\ 0\mathit{\text{if}}\mathit{rand}\le 0.5\end{array}\right.$$E22

The random movement of ants is restricted within search space. Hence, the ant’s position will be updated as,

$$\mathit{New}\mathit{ant}\mathit{\text{position}}=\frac{\left(\mathit{old}\mathit{ant}\mathit{\text{position}}-a_m\right)\left(d_m-c_m^t\right)}{b_m-a_m}+c_t$$E23

Where, a_m and b_m represents minimum and maximum of ant’s random walk, c^t_m and d^t_m represents minimum and maximum of m^th variable at t^th iteration.

5.3.2 Trapping of ants

The equation for the ant’s trapping into ant-lion’s pit is as,

$$\begin{array}{l}{c}_m^t=\mathit{Ant}-{\mathit{\text{lion}}}_n^t+c^t\\ d_m^t=\mathit{Ant}-{\mathit{\text{lion}}}_n^t+d^t\end{array}$$E24

Construction of trap

By using roulette wheel method, the best ant-lion is chosen.

Ants sliding toward ant lion

Ants sliding into the pit are represented by,

$$\begin{array}{l}{c}^t=\frac{c^t}{{10}^{\frac{\mathit{wt}}{s}}}\\ d^t=\frac{d^t}{{10}^{\frac{\mathit{wt}}{s}}}\end{array}$$E25

$$w=\left\{\begin{array}{c}2\mathit{\text{if}}t>0.1S\\ 3\mathit{\text{if}}t>0.5S\\ 4\mathit{\text{if}}t>0.75S\\ 5\mathit{\text{if}}t>0.9S\\ 6\mathit{\text{if}}t>0.95S\end{array}\right.$$E26

5.3.3 Catching the pray and reconstruction process of pit

If the ant’s objective function is better than that of the antlions, then the ant-lion moved to a new position to find the opportunity to catch the pray in a better way. The new ant-lion’s position is given as,

$$\begin{array}{l}\mathit{Ant}-\mathit{\text{lion}}\mathit{new}\mathit{\text{position}}=\mathit{Ant}\mathit{\text{position\hspace{0.17em}\hspace{0.17em}for}}\mathit{ant}\mathit{\text{objective funtion}}>\\ \mathit{ant}-\mathit{\text{lion objective function}}\end{array}$$E27

5.3.4 Elitism

In each stage the best solution is stored called Elitism. Each ant is assumed to be connected to an antlion using roulette wheel and is represented by,

$$\mathit{\text{elite}}\mathit{ant}=\frac{\mathit{\text{random walk around the selected}}\mathit{ant}-\mathit{\text{lion}}}{2}$$E28

5.3.5 The particle swarm optimization algorithm (PSOA)

This optimization algorithm is stochastic population-based method to solve optimization issues. PSO is a nature inspired meta-heuristic algorithm, works with a particles swarm. In a search space every particle is considered as a solution with two characteristics namely, its own velocity and position. The velocity represents the direction and the distance for optimize the position in next iteration. The position defines the present value of the solution. For the entire particle its own present best named as pbest and the overall global solution named as Gbest estimated.

5.3.6 Mathematical formulation for PSOA

PSO is a stochastic population-based metaheuristic to solve continuous optimization problems. The main idea of the metaheuristic came from the observation of behavior of natural organisms to find food. PSO works with a swarm of particles. Each particle is a solution to a problem in the decision space and has two characteristics: its own position and velocity. The position represents the current values in the solution; the velocity defines the direction and the distance to optimize the position at next iteration. For each particle i its own past best position p best i and the entire swarm’s best overall position G are remembered. In basic PSO the velocity and position of each particle are updated in the following equations,

$$v_i\left(k+1\right)={\mathit{wv}}_i\left(k\right)+{\rho }_1{c}_1\left(p_i^{\mathit{\text{best}}}-X_i\left(k\right)\right)+{\rho }_2{c}_2\left({\mathit{GX}}_i\left(k\right)\right)$$E29

$$X_i\left(k+1\right)=X_i\left(k\right)+v_i\left(k+1\right)$$E30

Where, $i$ is a particle index, $k$ is an iteration number, $v_i\left(k\right)$ is velocity, $X_i\left(k\right)$ is the position of particle $i$ at iteration $k$, $p_i^{\text{best}}$ is the best position found by particle $i$ (personal best), $G$ is the best position found by the swarm (global best, best of personal bests), $w$ is an inertia coefficient, $\left({\rho }_1,{\rho }_2\right)$ are random numbers in [0,1] interval, $c_1,c_2$ are positive constants representing the factors of particle’s attraction toward its own best position or toward the swarm’s best position (Figure 7).

Figure 7.

Flow chart of proposed PSOA-ALOA hybrid technique [6].

5.3.7 Hybridization of particle swarm optimization algorithm (PSOA) based ant-lion optimization algorithm (ALOA)

In this part of hybridization is, the PSOA has been associated to enhance the ALOA performance, by updating the ALOA elitism stage.

Initially the load flow analysis is carried out with the help of Backward/Forward (BW/FW) sweep method for the base case with the bus data and the load data of the test system without adding any extra load on to the system. Now with intend to reduce these power losses up to maximum possible extent the PSOA based ALOA is combined to form a hybrid methodology. With this methodology, the candidate busses, and capacities of DGs are estimated with the help of ALOA method, and then the locations of antlions are updated with the help of PSOA methodology. This will be evaluated with the help of the flowchart given for proposed hybrid methodology.

5.3.8 Specific contribution

Initially by using the ALOA standalone algorithm the DG locations and capacities were identified intend to reduce the total power losses and total operating cost and enhancement of voltage stability index and the voltage profile at every bus. All the values have been tabulated with different power factor conditions on various test networks with different types of DG units.

Now with the help of proposed hybrid algorithm the DG locations and capacities were identified intend to reduce the total power losses and total operating cost and enhancement of voltage stability index and the voltage profile at every bus. All the values have been tabulated with different power factor conditions on various test networks with different types of DG units.

In ALOA hybridized with PSOA the elitism phase of the ALOA for DG position were updated by using the following mathematical equations,

$$\begin{array}{l}\mathit{new}{\mathit{\text{volocity}}}_j\left(\mathit{kk}+1\right)=\mathit{\text{cons}}tan{t}^{\ast }\mathit{old}{\mathit{\text{velocity}}}_j\left(\mathit{kk}\right)\\ +\mathit{\text{inertia cons}}tan{t}_{1\ast }\mathit{\text{positive cons}}tan{t_1}^{\ast }(p_j^{\mathit{\text{best}}}-\mathit{\text{position}}\left(\mathit{kk}\right)\\ +\mathit{\text{inertia cons}}tan{t}_{2\ast }\mathit{\text{positive cons}}tan{t_2}^{\ast }\left(G_j^{\mathit{\text{best}}}-\mathit{\text{position}}\left(\mathit{kk}\right)\right)\end{array}$$E31

$$\mathit{new}{\mathit{\text{position}}}_j\left(\mathit{kk}+1\right)=\mathit{old}{\mathit{\text{position}}}_j\left(\mathit{kk}\right)+\mathit{new}\mathit{\text{velocity}}\left(\mathit{kk}+1\right)$$E32

This is because the performance of the ALOA has high iteration to reach the optimal solution and poor performance. To improve the performance of ALOA is based on the PSOA. This property leads to this hybridization of ALOA-PSOA for optimal integration of DG units were identified with intend to reduce the total power losses and total operating cost and enhancement of voltage stability index and the voltage profile at every bus. All the values have been tabulated with different power factor conditions on various test networks with different types of DG units.

The PSOA has still having some issues that ought to be resolved. Therefore, the future works on the PSOA will probably concentrate on the following:

1. Find a particular PSO algorithm which can be expected to provide good performance.

2. Combine the PSO algorithm with other optimization methods to improve the accuracy.

3. Use this algorithm to solve the non-convex optimization problems.