A Random Walk Analysis of Search in Metaheuristics

Random walks are a useful modeling tool for stochastic processes. The addition of model features (e.g. finite travel in one direction) can provide insight into specific practical situations (e.g. gambler’s ruin). A series of random walk experiments are designed to study the effects of selection, exploration, and exploitation during the search processes of metaheuristics. We present a set of random walk conditions which leads to greater movement as the dimensionality of the sampling distributions increases. We then implement a version of Simulated Annealing in a similar search space which also achieves improving performance with increasing dimensionality. Conversely, we show that standard Particle Swarm Optimization has decreasing performance with increasing dimensionality which is consistent with the expected effects of the Curse of Dimensionality. These experiments give us insights into future methods that metaheuristics might be able to employ to defeat the Curse of Dimensionality (in globally convex, continuous domain search spaces).