A Critical Evaluation of Recent Deep Sketch Models’ Reconstruction Abilities

Drawing a sketch is a uniquely personal process that depends on previous knowledge, experiences, and current mood. Hence, the success of deep generative sketch models depends on user expectations. Yet, the unconditional generation ability of these models does not consider human-centered metrics in the training step. To achieve this kind of training process, we first need to understand the factors behind human perception on successful generative examples. We designed a user study where we asked twenty-one people from different disciplines to determine these factors. In this study, participants ordered output sketches from most to least recognizable of four recent generative models (Autoencoder, DCGAN, SketchRNN, and Sketchformer) from most to least recognizable. The results suggest that success in representing the distinct feature of a category is more important than other attributes such as spatial proportions or stroke counts. We shared our code, the interactive notebooks, and field study results to accelerate further analysis in the area.