Locally and multiply distorted image quality assessment via multi-stage CNNs

Abstract The majority of existing objective Image Quality Assessment (IQA) methods are designed specifically for singly and globally distorted images, which are incapable of dealing with locally and multiply distorted images effectively. On the one hand, artificially extracted features in traditional IQA methods are insufficient to represent quality variations in locally and multiply distorted images. On the other hand, the IQA methods suitable for both locally and multiply distorted images are scarce. In view of this, an IQA method based on multi-stage deep Convolutional Neural Networks (CNNs) is proposed for locally and multiply distorted images in this paper. The method adopts a three-stage strategy, which are distortion classification, quality prediction of single distortion and comprehensive assessment, respectively. Firstly, three datasets of locally, multiply and singly distorted images are designed and established. Secondly, a local and multiple distortion classifier, a distortion type classifier and prediction models of single distortions are obtained based on CNN models in their corresponding stages. Thirdly, the predicted results of single distortions are weighted by the output confidence probability of the classifiers, thus obtaining the final comprehensive quality. Experimental results verified the advantages of the proposed method in measuring the quality of locally and multiply distorted images.