The PSO family: deduction, stochastic analysis and comparison

The PSO algorithm can be physically interpreted as a stochastic damped mass-spring system: the so-called PSO continuous model. Furthermore, PSO corresponds to a particular discretization of the PSO continuous model. In this paper, we introduce a delayed version of the PSO continuous model, where the center of attraction might be delayed with respect to the particle trajectories. Based on this mechanical analogy, we derive a family of PSO versions. For each member of this family, we deduce the first and second order stability regions and the corresponding spectral radius. As expected, the PSO-family algorithms approach the PSO continuous model (damped-mass-spring system) as the size of the time step goes to zero. All the family members are linearly isomorphic when they are derived using the same delay parameter. If the delay parameter is different, the algorithms have corresponding stability regions of any order, but they differ in their respective force terms. All the PSO versions perform fairly well in a very broad area of the parameter space (inertia weight and local and global accelerations) that is close to the border of the second order stability regions and also to the median lines of the first order stability regions where no temporal correlation between trajectories exists. In addition, these regions are fairly similar for different benchmark functions. When the number of parameters of the cost function increases, these regions move towards higher inertia weight values (w=1) and lower total mean accelerations where the temporal covariance between trajectories is positive. Finally, the comparison between different PSO versions results in the conclusion that the centered version (CC-PSO) and PSO have the best convergence rates. Conversely, the centered-progressive version (CP-PSO) has the greatest exploratory capabilities. These features have to do with the way these algorithms update the velocities and positions of particles in the swarm. Knowledge of their respective dynamics can be used to propose a family of simple and stable algorithms, including hybrid versions.