A Deep Neural Network to Detect Keyboard Regions and Recognize Isolated Characters

Some robot systems need to interactively auto-type on touch screens using a digital camera as the input source. To do so, it is important to have an algorithm that reliably detects the keyboard region and locates and recognizes its characters from an image. However, today most research efforts in the optical character recognition (OCR) area is focused on scene text detection and recognition. Though these algorithms work well on images with strong noise impacts, they perform poorly on keyboard texts that have isolated characters. In this paper, we present a framework to solve this problem. The algorithm consists of three steps: (i) a Single Short Detector (SSD) to search the keyboard regions from an input image, (ii) a deep neural network model that locates and recognizes the individual characters~(case sensitive) in one pass for a keyboard region, and (iii) a semi-supervised corrector that fixes the errors in step 2 using hand-crafted features. We will show that this algorithm is able to process different types of keyboards, and is robust to varying noise levels and text occlusions.