Personal Best Oriented Particle Swarm Optimizer

optimization is a population based stochastic optimization technique influenced by the social behavior of bird flocking or fish schooling.PSO shares many similarities with evolutionary computation techniques such as Genetic Algorithms (GA). The system is initialized with a population of random solutions and searches for optima by updating generations. However, unlike GA, PSO has no evolution operators such as crossover and mutation. In PSO, the potential solutions, called particles, fly through the problem space by following the current optimum particles. This book represents the contributions of the top researchers in this field and will serve as a valuable tool for professionals in this interdisciplinary field.


Introduction
Optimization problems are frequently encountered in many engineering, economic or scientific fields that engineers or researchers are seeking to minimize cost or time, or to maximize profit, quality or efficiency, of a specific problem. For example, economic dispatch of power generation, optimal allocation of resources for manufacture, design optimal plant to maximize production, and so many which are unable to enumerate completely. In addition, many optimization problems are very complex and hard to solve by conventional gradient-based techniques, particularly the objective function and constraint are not in closed forms. Thus, the development of a good optimization strategy or algorithm is of great value. In the past decade, particle swarm optimization (PSO) algorithm  attracts many sights around the world due to its powerful searching ability and simplicity. PSO simulates the swarm behavior of birds flocking and fish schooling that swarms work in a collaborative manner to search for foods as efficient and quick as possible. There are three different types of PSO which are frequently encountered in literature. They are constriction type PSO, constant inertia weight PSO and linearly decreasing inertia weight PSO. Each of them has been successfully applied to many optimization problems. While empirical studies have proven PSO's usefulness as an optimization algorithm, it does not always fit all problems. Sometimes, it may also get stuck on local optimal. In order to improve the performance, many variants of PSO have been proposed. Some of the proposed algorithms adopted new operations and some of the modifications hybridized with other algorithm. Although they are claimed better than original PSO algorithm, most of them will introduce extra mathematical or logical operations, which, in turn, making algorithm more complicate and spending more computing time. Especially, they, in general, did not present any theoretical models to describe its behavior and support such modifications. Many researchers have devoted to study how PSO works. They intended to discover the implicit properties of PSO and its weakness and strength via theoretical analysis. The first attempt to analysis PSO is made by Kenndey [Kennedy, 1998]. Meanwhile, Ozcan and Mohan showed that a particle in a simple one-dimensional PSO system follows a path defined by a sinusoidal wave with random amplitude and frequency. However, the effects of inertia weight are not addressed in that paper [Ozcan & Mohan, 1999]. In order to analyze the dynamics of PSO, Yasuda proposed a generalized reduced model of PSO accounting for the inertia weight. The stability analysis is carried out on the basis of both the eigenvalue analysis and numerical simulation [Yasuda et al., 2003]. Trelea has carried out a convergence analysis of PSO and then derived a graphical parameter selection guideline to facilitate choosing parameters [Trelea, 2003]. A formal analysis of the PSO is carried out by Clerc and Kennedy [Clerc & Kennedy, 2002]. By treating the random coefficients as constants, the analysis started from converting the standard stochastic PSO to a deterministic dynamical system. The resulting system was a second-order linear dynamic system whose stability depended on the system's eigenvalues. The parameter space that guarantees stability is also identified. A similar analysis based on deterministic model of the PSO was also carried out in identifying regions in the parameter space that guarantee stability [van den Berg, 2002]. Recently, stochastic convergence analysis of the standard PSO is reported in [Jian, 2007], where a parameter selection guide is also provided to ensure the convergence. Similar to genetic algorithm or evolutionary algorithm, PSO is also a population-based optimization technique. PSO searches for optimal solution via collaborating with individuals within a swarm of population. Each individual, called particle, is made of two parts, the position and velocity, and proceeds according to two major operations, velocity and position updating rules. Position and velocity represent the candidate solution and step size, a particle will advance in the next iteration, respectively. For an n-dimensional problem and a swarm of m particles, the i-th particle's position and velocity, in general, are denoted as where m is the number of particles, and superscript T stands for the transpose operator. Considering on the inertia weight PSO, the operations of position and velocity are expressed as: where ω is the inertia weight; c 1 and c 2 are two positive constants called acceleration constants; rnd(1) is a uniform random number in (0,1); d is the index of dimension; p g and p i are the best position ever found by all particles and the best position a particle ever found so far, respectively; t is the iteration count. Hereafter, p g and p i will be called the global best and personal best particle of the swarm, respectively, in this chapter. Personal best oriented particle swarm optimizer (PPSO), also a varient of particle swarm optimization, is a newly developed optimization solver [Chen & Yeh, 2006a ]. PPSO uses the same velocity updating rule as PSO. However, the position updating rule is replaced by (3).

1)
The modification came from the observation that since p id is the best particle ever found so far, it may locate around the vinicity of the optimal solution. Fotunately, previous studies showed that PPSO performs well both in testing on a suite of benchmark functions and applying to economic dispatch problems of the power system and others [Chen & Yeh, 2006a, 2006b, 2008. However, all the results were obtained from emperical studies. The main drawback of PPSO may lie in a fragile theory basis at first galance. No theoretical www.intechopen.com analysis has been found in literature, although it not only can help reveal the behavior of particles but also the convergent property of the proposed approach. This chapter presents a therotical analysis of PPSO. Based on the analysis, the implicit behavior of PPSO will be revealed. Meanwhile, it also provide a guideline for parameters selection. It is well-known that the particle's trajectory of PSO is characterized by a secondorder difference equation. However, it will be clear later that a first order difference equation is sufficient to characterize the particle's behavior of PPSO. A simple mathematical model can help one easily grasp the features of PPSO and convergent tendency.

Analysis of PPSO
This section intends to construct the mathematical model of PPSO and, based on the developed model, study the property of PPSO. The analysis is started from a simplified deterministic model, a single particle and one dimension case, keeping all the parameters constants. After that, assuming some parameters be uniformly distributed random numbers, a stochastic model is then built to simulate the nature of PPSO in the next section.
2.1 Simple PPSO: one particle and one dimension case Each particle in PPSO represents a candidate solution of a specific problem. In other words, a particle is a position in a multidimensional search space in which each particle attempts to explore for an optimal solution with respect to a fitness function or objective function. In this section, a simple PPSO is first derived on which the following analysis is based. The canonical form of PPSO is represented as .
where ϕ 1 = c 1 ⋅rnd(1) and ϕ 2 = c 2 ⋅rnd(1) are two random numbers drawn uniformly from (0, c 1 ) and (0, c 2 ), respectively. Since (4) and (5) operate on each dimension exclusively, for notation simplicity, one can omit the subscript i and d of x id and v id , and retain subscript g and i of p g and p i to emphasize the difference between them. The analysis given below considers only one particle and one dimensional case. However, the results can easily be extended to multidimensional case without losing generality. Eqations (4) and (5) can now be rewritten as: Substituting (6) into (7), one has: where ϕ = ϕ 1 + ϕ 2 . Obviously, a first-order linear difference equation is sufficient to characterize the dynamic behaviors of the simple PPSO.

Deterministic model of PPSO
Now assume that both p i and p g are constants. Also assume that ϕ 1 and ϕ 2 are two constants. It turns out that the PPSO becomes a deterministic model described by a firstorder linear difference equation with constant coefficients. If the right-hand side of (12) is nonzero, it is a nonhomogeneous linear difference equation. The total solution of a nonhomogeneous linear difference equation with constant coefficients is the sum of two parts, the homogeneous solution, which satisfies the difference equation when the righthand side of the equation is zero, and the particular solution, which satisfies the difference equation with a nonzero function F(t) on the right-hand side. The homogeneous solution of a difference equation with constant coefficient is of the form Aλ t , where λ is called the characteristic root of the difference equation and A is a constant to be determined by the boundary (initial) condition. The homogeneous solution and particlar solution of (12) can be obtained readily and Here, subscript h and p are used to denote the homogeneous solution and particular solution. The total solution of (12) become is called the weighted mean of p g and p i . Given the initial condition x(0)=x 0 , the dynamic property of a particle is completely characterized by where x 0 is the initial vale of x(t) and A = (x 0 -p w ). Equation (12) and (17) represent the position or trajectory that a particle may explore in implicit and explicit form.

Convergence property of the deterministic PPSO
Apparently, if (ω -φ) satisfies the following condition The limit does exist whenever p w is an arbitrary point in the search space, i.e., p w is finite. It is obvious that if 0 < ω < 1, it leads to (1 + ϕω) > 0 since ϕ = ϕ 1 + ϕ 2 > 0, and the weighted mean p w is finite. Hereafter, finite p w and 0 < ω < 1 are assumed, unless stated explicitly. The feasible region in which x(t) is strictly converges for 0<ω<1 and -1<ϕ< 2 is plotted in Fig.1, where the gray area is the feasible region if stability is concerned, and the dark line on the center corresponds to ω = ϕ. Figure 1. The gray region is the feasible region which particle strictly converges for 0 < ω < 1 and -1< ϕ < 2. The centered dark-line on the gray area corresponds to ω = ϕ

Step size
The span the particle advances in the next step is calculated using the successive positions at t and (t+1), Define the step size as is the distance between the current position x(t) and the weighted mean, p w . Equation (26) tells that the step size is a multiple, defined by -(ω -ϕ -1), of the distance between x(t) and p w . If -(ω -ϕ -1) is positive, x(t) moves in aligning with the direction from current position to p w and, if -(ω -ϕ -1) is negative, x(t) moves on the opposite side. The former make particles moving close to p w and the latter make particles get far way from p w . Now, define a step size control factor, δ, as: Obviously, how long a particle will advance in next turn is controlled by the step size control factor δ. A large δ makes a particle to move far away from current position and a small value of δ makes a particle moving to nearby area. Meanwhile, it is interesting to note that if 0 < δ < 2, (27) becomes to This agrees with (19). In other words, if the step size control factor satisfies 0 < δ < 2, the deterministic PPSO converges to p w . Otherwise, it diverges.

www.intechopen.com
Clearly, under condition 0 < δ < 2, the deterministic PPSO is stable, otherwise, it is unstable. Since δ is a function of ω and ϕ, the choices of ω and ϕ affect the magnitude of the step size control factor, or, in other words, affect the stability of PPSO. Returning to (30), there are two cases are especially worthy to pay attention: This case corresponds to In such situation, x(t+1) moves to the region to the left of p w whenever p w is greater than x(t), or the region to the right of p w whenever p w is less then x(t).
This case corresponds to This means that x(t+1) advances to the region to the right of p w whenever p w is less than x(t), or the region to the left of p w whenever p w is greater then x(t). These two cases are illustrated in Figs.2 and 3. It is apparent that the step size control factor affects how far the particle moves. Since the step size is proportional to δ, a large δ corresponds to advancing in a large step and small δ corresponds to small step. Moreover, a positive δ makes x(t) move along the direction from x(t) to p w ,while a negative δ causes it move along the opposite direction. By controlling δ, or equivalently (ω -ϕ), particles movement will be totally grasped. It is expected that if δ is uniformly changed in (0, 2), then x(t) will vibrate around the center position, p w , the weighted midpoint of p g and p i . This is a very important property of PPSO. Similar phenomenon has been observed [Kennedy, 2003] and verified theoretically in PSO [Clerc & Kennedy, 2002].

Parameters selection
Equation (27) defines the step size control factor. It provides clues for determining parameters. First, confine the step size control factor within (δ min , δ max ), where δ min and δ max are the lower and upper limits of the step size control factor, respectively, it turns out that max min After proper rearrangement, (33) becomes According to (34), once the lower and upper bounds of the step size control factor are specified, the range of ϕ depends on ω. The most important is that a stable PPSO requires, based on (29), δ min = 0 and δ max =2. Substituting these two values into into (34), it has Equation (35) says that, if ϕ uniformly varies from (ω -1) to (ω + 1), x(t) will explore the region from (p w -dx) to (p w + dx). It also implies that it is possible to use a negative value of ϕ while PPSO is still stable. This fact has been shown in Fig.1. www.intechopen.com Figure 2. The next position, x(t+1), a particle will move to for 0 < δ < 1, (a) p w > x(t) and (b) p w < x(t) Figure 3. The next position, x(t+1), a particle will move to for 1 < δ < 2, (a) p w > x(t) and (b) p w < x(t) Since ϕ 1 and ϕ 2 are both positive numbers, so is ϕ. Fig.4 shows the case that ϕ is positive and is restricted to 0 < ϕ < 2.
If ω is assigned, from (27), one also has where ϕ min and ϕ max are the lower and upper limits of ϕ. Thus, one can use (36) to predict the range the particle attempts to explore for a specific range of ϕ, if ω is given. From (36), one can also readily verify that ϕ min = ω -1 and ϕ max = ω + 1 result in δ = 0 and 2, respectively, agreeing with (29). A graph of step size control factor versus φ with ω as parameter is plotted in Fig.5 for 0 < ϕ < 2 and 0 < ω < 1. The gray area corresponds to convergent condition since 0 < δ < 2. One can use this graph to evaluate whether the selected parameters result in convergence or not. Figure 4. The feasible region for 0 < ω < 1 and 0 < ϕ < 2 www.intechopen.com Figure 5. A plot of step size control factor δ versus ϕ with ω as parameter

Equilibrium point
According to the above analysis, one can conclude that, for a stable deterministic PPSO, each particle moves in discrete time along the trajectory defined by (12) or (17), with specific step size, and finally settles down at an equilibrium point p w . The equilibrium point is a function of ϕ 1 and ϕ 2 . Referring to (16), one can readily verify that if ϕ 1 > (1 + ϕ 2 -ω), the equilibrium point p w biases to p g, and biases to p i if ϕ 1 ≤ (1 + ϕ 2 -ω). However, the equilibrium point found in PSO is the midpoint of p g and p i since ϕ 1 = ϕ 2 is usually used in PSO.

3.Stochastic PPSO
Instead of constants, now, restore both ϕ 1 and ϕ 2 to be uniform random numbers in (0, c 1 ) and (0, c 2 ), respectively. The model of (12) and (17) are still applied except that ϕ 1 and ϕ 2 are now two uniform random numbers. Analysis of the dynamic behavior of this stochastic PPSO will be given by extending the analysis provided in the previous section, with the replacement of expectation value for ϕ 1 and ϕ 2 as well as x(t) from the probabilistic point of view. In the following analysis, the terms mean value, or simply mean, and expectation value will be used alternatively, in a looser mathematical standard, in the context.

Convergent property
Considering the explicit representation, Eq.(17), of the trajectory of a particle, since ϕ 1 and ϕ 2 are both uniform random numbers, the averaged dynamic behavior of a particle can be observed by its expectation value, i.e. www.intechopen.com where E(x(t)), E(p w ), and E(ϕ) are the expectation value of x(t), p w and ϕ, respectively. Here, ϕ 1 and ϕ 2 are assumed to be two exclusive uniform random numbers; and E(ϕ) = E(ϕ 1 ) + E(ϕ 2 ). Apparently, if the condition 1 < ω -E(ϕ)) < 1 is true, then According to (39), the trajectory of each particle converges to a random weighted mean, E(p w ), of p g and p i , where E(p w ) is finite for 0 < ω < 1, 0 < E(ϕ 1 ) and 0 < E(ϕ 2 ) as well as finite p i and p g . Owing to ϕ 1 and ϕ 2 are both uniform random numbers in (0, c 1 ) and (0, c 2 ), respectively, it has E(ϕ 1 ) = 0.5c 1 , E(ϕ 2 ) = 0.5c 2 and E(ϕ) = E(ϕ 1 + ϕ 2 ) = 0.5(c 1 + c 2 ). Thus, (40) becomes Obviously, for a stochastic model of PPSO, the random weighted mean p w is different from that obtained by deterministic model of PSO and PPSO. Meanwhile, for 0 < ω < 1, E(p w ) bias to p i . This means that particle will cluster to p i instead of p g .

Step size
Similar to deterministic PPSO, the step size of the stochastic PPSO can be computed by For a stochastic PPSO, the mean (expectant) step size a particle will move in next turn is computed from (43), which is a multiple of the mean distance between the random weighted mean E(p w ) and mean current position E(x(t)). Similar to deterministic PPSO, the mean step size control factor is defined as The step size and step size control factor are no longer static values but stochastic ones. Furthermore, for 0 < E(δ) < 2, from (45), it also has -1 < -(ω -E(ϕ) ) < 1 (46) Actually, (46) is the same as (38). Rearranging (46), one has Equations (46) and (47) are similar to (30) and (35), respectively, except that the constant ϕ (=ϕ 1 + ϕ 2 ) is replaced by sum of the expectation values of two random numbers. As concluded in the previous section, a stable stochastic PPSO equivalently means that the mean step size control factor of each particle's movement must be within the range of 0 < E(δ) < 2. In other words, if E(ϕ) lies between (ω -1) and (1 + ω), the system is stable.

Parameter selection for stochastic PPSO
This subsection discusses how to choose proper parameters for PPSO.

Inertia weight
Recall that E(ϕ) is a positive number since ϕ is the sum of two uniformly random numbers varying between (0, c 1 ) and (0, c 2 ), where c 1 and c 2 are two positive numbers. Now, consider the step size control factor governed by (45) for ω chosen from the following ranges: If E(ϕ) is assigned, Eqs.(48)-(51) specify the average possible value of step size control factor for different choosing ranges of inertia weight ω. For example, if E(ϕ) =1.5, E(δ) are 1.25, 1.75, 2.75 and 3.75 for ω = 1.25, 0.75, -0.25 and -1.25, respectively. Clearly, it is improper to have a minus value of ω since it will make particle violate the stability rule, i.e., the trajectory of particle diverges. To have a better vision of parameter selection for ω > 1 and 0 < ω < 1, it is better to explain with figure as illustrated in Figs.6 and 7 where the dotted lines represent the domain a particle may visit in next turn for the cases ω = 1.25 and 0.75 under the condition of E(ϕ) =1.5. The cross sign in the midpoint of two ellipses is the center of the search range. Here, only the case that E(p w ) located to the right of E(x(t)) is shown. However, similar aspects can be observed for E(p w ) located to the left of E(x(t)). Since E(ϕ) =1.5, the uniform random number ϕ varies from 0 to 3. The lower and upper step size control factors are -0.25 and 2.75, respectively, for ω = 1.25. These values are calculated using Eq.(27) . It is seen then in Fig.6 that the search area extends from -0.25E(dx) to 2.75E(dx). Although the upper value of E(δ) is greater than the upper limit of the step size control factor, the expectation value of the step size control factor is 1.25, which obeys the stability rule given in Eq.(38). From Fig.6, one can also find that if ω is greater than unity, particle is possible to search the region to the left of E(x(t)). Meanwhile, the greater ω is, the more the search area shift to left of E(x(t)), which will reduce diversity of particle because particles move to the vinicity of E(x(t)). Now, refer to Fig.7, for ω = 0.75 and E(ϕ) =1.5,the searach domain are in 0.25E(dx) and 3.25E(dx) with mean of 1.75E(dx). This parameter setting also obeys the stability criterion. It seems both cases of parameter choice is proper. However, refer to Eq.(37), the trajectory of a particle is mainly governed by the term (ω -E(ϕ)) t . If (ω -E(ϕ)) is too small, E(x(t)) will vanish quickly and particle may get stuck on local optimum. In other words, the value of (ω -E(ϕ)) represents an index for evaluation of the prematurity of particles. Therefore, it is better to have 0 < ω < 1, and empirical studies have shown that it is proper to choice of inertia weight in 0.7 < ω < 0.8. Figure 6. The area the particle will explore for ω = 1.25 and E(ϕ) = 1.5 Figure 7. The area the particle will explore for ω = 0.75 and E(ϕ) = 1.5

Acceleration coefficient
Recall that c 1 and c 2 are referred to acceleration coefficients, and ϕ is the sum of two uniform random numbers in (0, c 1 ) and (0, c 2 ). The lower and upper limits of ϕ are then 0 and (c 1 +c 2 ), respectively. To determine c 1 and c 2 , it has to consider from three aspects: prematurity, population diversity and particle stability.

www.intechopen.com
If ϕ varies from 0 to (c 1 +c 2 ) uniformly, from Eq.(27), the lower bound of the step size control factor is determined by the choice of ω, i.e., while the upper bound is set by ω, c 1 and c 2 , which is For simplicity, it usually has c 1 =c 2 =c, Eq.(53) becomes Accounting for stability, in terms of step size control factor, staibility criterion is descibed as Approximate the expectation value E(δ) by the average value of δ min and δ max , it has () Based on (56), one can determine the acceleration coefficients once ω and E(δ) is assigned. For example, let ω = 0.75 and E(δ) = 1.75 (stisfies Eq.(55)), solve Eq.(56) for c. It is obtained c=1.5. The acceleration coefficients are then set to c 1 =c 2 =1.5. The lower and upper bounds of the step size control factor computed by Eq.(52) and Eq.(54) are 0.25 and 3.25, respectively. The range the particle will search is shown in Fig.7 for this example. It is seen that the search domain stretchs over from 0.25E(dx) to 3.25E(dx), where E(dx) = E(p w ) -E(x(t)) is the distance between expectation values of the random weighted mean, p w, of p g and p i and current particle position x(t). Of course, this is not the unique parameters setting for PPSO. Frequently, it is required to compare the performances between PSO and PPSO. In such situation, the common used parameters for PSO (ω=0.729, c 1 =c 2 =1.494) fit to PPSO since E(ϕ) = 1.494, and E(δ) = 1.765 which satisfies Eq.(55).

Equilibrium point
Both PPSO and PSO define the particles as potential solutions to a problem in a multidimensional space with a memory of its ever found best solution and the best solution among all particles. PPSO generates a sequence of x(t) iteratively, and if x(t) is a stable sequence, it has where the random weighted mean E(p w ) defined in (42) is the equilibrium point of the sequence. As an optimization solver, it is expected that E(p w ) is the optimum solution. It is seen from (57) that if p g = p i = p, E(p w ) = p. This means that particle settles down at the global best ever found, i.e., PPSO is expected to constantly update the personal best and global best solutions ever found, and finally converges to E(p w )= p g = p i , the optimum solution or near optimum solution of the problem at hand.
Note that the random weighted mean of PSO is defined as [Kennedy, 1999 andvan den Bergs, 2003]  Oviuously, the particles of PSO and PPSO will converge to different equilibrium points. Therefore, in additon to the equilibrium points, the trajectories of PSO and PPSO are also different since trajectories of PPSO and PSO are characterised by a first-order and secondorder difference equations [Trelea, 2003, Yasuda et al., 2003, van den Bergh, 2003, respectively. These are the distinctive features of PSO and PPSO.

Example Trajectories
To see the properties between PSO and PPSO, the trajectories the particle traversed are investigated by a primitive PSO and PPSO model, where p g and p i are set as two arbitrarily constants. To keep thing simple and have a better observation, trajectories of one dimension are considered here. Both the trajectories are generated with same initial condition, i.e., same initial values for position and velocity. Meanwhile, both PSO and PPSO use the same value for the parameters, ω, c 1 and c 2 that they are set as ω=0.729 and c 1 =c 2 =1.494. Each of the trajectories is constructed by 10000 points and, for fair comparison, each points is generated using the same random numbers for both PSO and PPSO at each time step. The pseudo-code for generating the trajectories is shown in Fig.8, where x(t) and y(t) are the positions of PSO and PPSO at time step t; vx(t) and vy(t) represent the velocity of PSO and PPSO, respectively; x(0) and y(0) is the initial positions, vx(0) and vy(0) are the initial velocities of PSO and PPSO, respectively. 1g /* pseudo code for evaluation PSO and PPSO */ Set ω,c ,c,p a n dp ; Initialize x(0), y(0), vx(0) and vy(0); For t 1 to 10000 { c rnd(1); c rnd(); vx(t) ω vx(t 1) (p x(t 1)) (p x(t 1)); vy(t) ω vy(t 1) (t 1)) (p y(t 1)); x(t) vx(t) x(t 1); y(t) vy(t) p ;  (0) and vy(0) that are arbitrarily set to -50, 10, 0, 0, 20 and 20, respectively. The gray lines in the centre of the figures represent the mean values of x(t) and y(t). They are denoted as µ x and µ y for x(t) and y(t), respectively. It is seen obviously that both the trajectories of PSO and PPSO randomly vibrate, or oscillate around the mean values within a limited ranges. The mean values are obtained as µ x = -20.009, and µ y = -15.288. These two values very close to the theoretical random weighted mean of p g and p i , defined in (58)    Recall that PSO has bell-shaped distribution of the trajectory centred approximately at p w , i.e., the weighted mean which equals to the midpoint of p g and p i [Kennedy, 2003]. This feature also has been observed in PPSO. Refer to Figs.(11) and (12), the histogram plots of the distribution of x(t) and y(t) are illustrated. In these figures, the distributions of the trajectories are drawn in grey lines and the vertical dash-line denoted the mean value of the trajectory. The horizontal and vertical axes represent the values of the trajectory and the occurrences a particle ever explored. The plots of the horizontal axis extend from (µ x -3σ x ) to (µ x + 3σ x ) and (µ y -3σ y ) to (µ y + 3σ y ) for PSO and PPSO, respectively, where σ x and σ y are the standard deviations of x(t) and y(t). Obviously, the distribution of the trajectory of the PPSO is also a bell-shaped centred at the random weighted mean. For a comparison, the normal distribution with mean µ x and standard deviation σ x for PSO and mean µ y and standard deviation σ y for PPSO are drawn in thick solid lines. Clearly, although PSO and PPSO works based on different mathematical models, they have similar dynamic behaviour.

Conclusion
This chapter intends to provide a theoretical analysis of PPSO to clarify the characteristics of PPSO. The analysis is started from a simplified deterministic model, a single particle and one dimension case, keeping all the parameters constants. After that, assuming the acceleration coefficients as uniformly distributed random numbers, a stochastic model is then built to describe the nature of the PPSO. With the assumption, it is shown that a firstorder difference equation is sufficient to describe the dynamic behaviour of the particles. Based on the models, the convergence property is studied and the guidance for parameters selection is provided. Trajectories comparison between PSO and PPSO are also presented. It is found that, similar to PSO, the particles of PPSO also stochastically explore for optimal solution within a region centered approximately equals to a random weighted mean of the best positions found by an individual (personal best) and its neighbours (global best). Like PSO, bell-shaped distribution of the particle's trajectory is also observed in PPSO. However, the centres of the www.intechopen.com distribution of PSO and PPSO are different so that leading to different equilibrium points and, hence, different results and performances. The results derived in this chapter justify the possibility of PPSO to be an optimization algorithm. Simulation results have been shown that PPSO performs in general better than PSO on a suite of benchmark functions. However, it does not imply that PPSO is a local or global search algorithm even the condition of stability is met. Further research is thus required to improve the search capability.