Better freehand sketch synthesis for sketch-based image retrieval: Beyond image edges

Abstract With the rapid development of electronic touch screen and pressure-sensing devices, research on freehand sketches has become a hotspot in recent years. In this paper, we first propose a new freehand sketch generation model (FHS-GAN), which is based on the deep architecture of dual generative adversarial nets (GANs). We construct a model which utilize the deep convolutional neural network (CNN) and GAN to produce freehand sketches. We then propose an improved deep CNN model as a validated network, which is based on Faster R-CNN, to measure the similarity of real sketches and generated freehand sketches by FHS-GAN, and we test the improved model using the produced sketches with two large sketch datasets. The experiments show that the proposed FHS-GAN framework achieves state-of-the-art results in comparison with other baseline models. Furthermore, the generated sketches can be used for other sketch recognition tasks, such as in a pre-processing step for application in sketch-based image retrieval (SBIR) and fine-grained sketch-based image retrieval (FG-SBIR). Overall, our FHS-GAN model is important for the development of freehand sketches.