Functional specialization in redundant modular neural networks

Biological and engineering experience suggests that modularity is an important aspect in the design of complex systems such as neural networks. However, functional specialization in adaptive modular networks, where a network module becomes more specialized for a specific function than other modules, is not well understood. Such understanding is needed, both to guide the growing number of computational studies of modular neural networks, and to comprehend the functional specialization that is an important property of many biological systems. This dissertation presents a systematic, three part study of various aspects of functional specialization in artificial modular neural nets. First, the factors causing emergence of specialization during learning in modular networks are investigated. It is shown that various parametric asymmetries, such as different module sizes or asymmetric learning rates, can easily produce a strong degree of specialization in the network, even in the absence of direct competition between modules. Second, networks with specialized modules are shown to be capable of withstanding substantial amounts of damage to the model, either through the changes in the system's internal state (weights) or through automatic rerouting of the information processing flows to undamaged regions. Third, a more fundamental question of emergence of individual and population specialization through evolution is studied by simulating the evolutionary process with a genetic algorithm. It is demonstrated that by utilizing such fitness criteria as minimization of a network's energy consumption, response time, and density of connections, the evolutionary search is influenced to consistently discover asymmetric architectures of modular networks having functional specialization. The results presented here will be of interest to those studying various computational aspects of modularity in neural networks, such as value of redundancy, competition among modules, and the overall dynamics of modular systems.