A Methodology for Neural Network Architectural Tuning Using Activation Occurrence Maps

Finding the ideal number of layers and size for each layer is a key challenge in deep neural network design. Two approaches for such networks exist: filter learning and architecture learning. While the first one starts with a given architecture and optimizes model weights, the second one aims to find the best architecture. Recently, several visual analytics (VA) techniques have been proposed to understand the behavior of a network, but few VA techniques support designers in architectural decisions. We propose a hybrid methodology based on VA to improve the architecture of a pre-trained network by reducing/increasing the size and number of layers. We introduce Activation Occurrence Maps that show how likely each image position of a convolutional kernel’s output activates for a given class, and Class Selectivity Maps, that show the selectiveness of different positions in a kernel’s output for a given label. Both maps help in the decision to drop kernels that do not significantly add to the network’s performance, increase the size of a layer having too few kernels, and add extra layers to the model. The user interacts from the first to the last layer, and the network is retrained after each layer modification. We validate our approach with experiments in models trained with two widely-known image classification datasets and show how our method helps to make design decisions to improve or to simplify the architectures of such models.