Classifying Historical Azulejos from Belém, Pará, Using Convolutional Neural Networks

The cultural heritage of a city is of great importance for the maintenance and enhancement of its history. Users can use innovative technologies such as Computer Vision to emphasize the city’s treasures attractively and playfully. In Belem, Para, azulejo is a meaningful cultural heritage case, which goes back to its foundation. The image recognition can facilitate and speed up the search for an azulejo and its historical information since, although cataloged, its visual appearance identifies it better than a name, which implies that the image-based search is much more natural than a text-based search. In this way, this work presents a prototype that uses Convolutional Neural Networks (CNN) to classify the azulejos from Belem by an image-based search. CNN’s training used two image datasets. The first contains images that show azulejos and other environmental elements (for instance, walls, doors, streets, and people). The second dataset contains images that show only azulejos, and in both datasets, they only have one type of azulejo per image. The trained model consists of twelve different types of azulejos, representing the recognizable classes. Thus, after training, the tflite (Tensorflow Lite) model is generated with azulejos classes to be used in the mobile device image classification task. Finally, we developed an application in which the user takes a photo, and the system sends it to the classification module that contains the trained CNN model. After the image classification process, the module returns the five classes’ values with the best accuracy and historical details about the azulejos.