Evaluation of Convolutional Neural Network Architectures for Chart Image Classification

Many information visualization techniques map abstract data into visual representations to present the underlying information in a more understandable way. However, when data is not available, automatic methods are necessary to extract the underlying data from the chart images. This paper focuses on chart image classification, which is a fundamental step of chart analysis and data extraction. The performance of most conventional classifiers depends on the feature extraction technique applied, usually requiring a domain-specific design for creating the features. In this context, Convolutional Neural Network (CNN) has gained attention not only because it has achieved good results in many computer vision tasks but also because it can learn representations of images without the design of a feature extractor specific for a domain problem. Thereby, this paper proposes an evaluation of CNN architectures (VGG19, Resnet-50, and Inception-V3), comparing them with conventional classifiers (HOG features combined with KNN, Naive Bayes, Random Forest and Support Vector Machine). An artificially generated dataset of 10 classes of chart images was used for training and testing. Furthermore, another dataset composed of chart images collected from the internet was used for validation. The results show that Resnet-50 and Inception-V3 performed best achieving an accuracy of 77.76% and 76.77% respectively on the validation set. The CNN models outperformed the conventional methods, where the best combination was HOG+SVM, achieving 45.03 %. Moreover, some similar chart types were expected to be misclassified, and they were discussed based on the confusion matrix.