Contrastive Learning based Multi-task Network for Image Manipulation Detection

The popularity of image editing techniques and user-friendly editing software have seriously reduced the authenticity of the images. Detection and localization of image manipulations are becoming urgent problems to be solved. Although many existing solutions attempt to address these problems, most works can only solve one specific type of manipulations. Furthermore, some methods need heavy, time-consuming preprocessings and/or postprocessings to localize tampered region, resulting in disconnection and under-optimization of the model. In this paper, a contrastive learning based multi-task network is proposed for the localization of multiple image manipulations. Multi-scale tampered patch classifications and pixel-wise tampered region semantic segmentation are integrated into an end-to-end multi-task network. The consistency of different region statistical properties is measured by contrastive learning to enhance the feature representation ability of the proposed network, improving the performance of tampered patch detection. Various scale tampered patch detections cooperate to localize the tampered region boundaries from coarse to fine. Prediction Pyramid composed of different scale patch detection results provides comprehensive guidance for pixel-wise semantic segmentation of the tampered region. Experimental results on four standard image manipulation datasets demonstrate the better performance of the proposed model.