MLP bilinear separation

In this paper, we present a thorough mathematical analysis of the use of neural networks to solve a specific classification problem consisting of a bilinear boundary. The network under consideration is a three-layered perceptron with two hidden neurons having the sigmoid serving as the activation function. The analysis of the hidden space created by the outputs of the hidden neurons will provide results on the network’s capacity to isolate two classes of data in a bilinear fashion, and the importance of the value of the sigmoid parameter is highlighted. We will obtain an explicit analytical function describing the boundary generated by the network, thus providing information on the effect each parameter has on the network’s behavior. Generalizations of the results are obtained with additional neurons, and a theorem concerned with analytical reproducibility of the boundary function is established.