Two-phase Dynamic Routing for Micro and Macro-level Equivariance in Multi-Column Capsule Networks

Abstract The capability of multi column convolutional networks in identifying local invariant features helps improve its performance on image classification tasks to a large extent. Suppression of non maximal activations in a convolutional network, however, can lead to loss of valuable information, as scalar activations typically only ,encode the presence (or absence) of a feature in an input image, providing no additional information. Capsule networks, on other hand, learn richer representations by propagating non-maximal activations to higher layers, encoding the agreement between neurons at various layers on the presence (or absence) of a feature into a fixed-length vector. Traditional capsule networks, however encodes agreements for micro and macro-level features of an input image with same precedence. Such an uniform agreement protocol can hinder the repsentation capability of a network, especially for datasets that contain objects with independently deformable components. To address this, we propose a novel two-phase dynamic routing protocol that computes agreements between neurons at various layers for micro and macro-level features, following a hierarchical learning paradigm. Experiments on seven publicly available datasets show that a multi-column capsule network that encodes an input image following our routing protocol performs competitively or better than contemporary multi-column convolutional architectures andtraditional capsule networks on a classification task.Implementations of the networks used in this paper have been made available at: github.com/DVLP-CMATERJU/TwoPhaseDynamicRouting.