Self-organization in Probabilistic Neural Networks

S. A. Kauffman (1993) explored the law of self-organization in random Boolean networks, and K. Inagaki (1998) also did it in neural networks partially. The aim of the paper is to show that probabilistic neural networks (PNNs) hold the order, even though the weights, the thresholds, and the connections between neurons are determined randomly; PNNs are recurrent networks and controlled by a probabilistic transition rule based on a Boltzmann machine. In addition, the deterministic transient neural networks (DNNs) which are the special networks of PNNs are studied extensively. From simulations, it is shown that in DNNs the dynamics follow the square-root law and there is another new critical point for the distribution of the thresholds. In addition, it is shown that in PNNs the averages of the Hamming distance between the attractors of DNN and PNN stay around a certain value depending on the thresholds and the gradient of the Sigmoidal function. These results can be explained by the sensitivity to the initial conditions of DNNs.