TimNet: A Text-Image Matching Network Integrating Multi-Stage Feature Extraction with Multi-Scale Metrics

Abstract Text image comparison is a core component in web consistency testing across browsers. Previous studies mainly focused on general image comparison, which could not best suit the proposed task since text images share unique properties. In this paper, we introduce a novel Text-Image Matching Network (TimNet) to detect text differences in image pairs. A Multi-Stage Feature Extraction module (MFE) is integrated in TimNet to extract not only non-textual content features but also text-like features that are temporally aligned. Moreover, TimNet is composed of a Multi-Scale Metric module (MM) to measure the similarity between two text images with various scales and aspect ratios. Accordingly, a Text-Image Similarity Database (TISDB) consisting of 615.6k text-image pairs of English characters, Chinese characters, and Arabic numbers was established. Extensive experiments were conducted to demonstrate that our TimNet outperforms existing state-of-the-art methods.