A scalable multi-signal approach for the parallelization of self-organizing neural networks

Abstract Self-Organizing Neural Networks (SONNs) have a wide range of applications with massive computational requirements that often need to be satisfied with optimized parallel algorithms and implementations. In literature, SONN have been generally parallelized with GPU computing according to a single-signal paradigm: each GPU thread manages one or more nodes of the network and works concurrently on one input signal at the time. This paper presents two contributions. The first one is the experimental proof that the single-signal approach for SONNs is not optimal for the task, as it is intrinsically sequential at its core and thus inherently limited in its performance. The non-optimality of the single-signal paradigm is illustrated via a specific and simplified benchmark. The second contribution is the introduction of a new multi-signal paradigm for the parallelization of SONNs, whereby multiple signals are processed at once in each iteration hence allowing different GPU threads to work on different signals. The advantages of the multi-signal approach are shown through several benchmarks involving the Self-Organizing Adaptive Map (SOAM) algorithm as a basis for evaluation. Having a graph-based termination condition that depends on the features of the network being grown, the SOAM algorithm allows assessing both functional equivalence and performances of the paradigm proposed without relying on arbitrary thresholds. Nonetheless, the evaluation proposed has a broader scope since it refers to a unified framework for the GPU parallelization of a generic SONN.