Understanding mobile document capture and correcting orientation errors

Smartphone cameras are increasingly used for document capture in daily life. To understand user behaviors, we performed two studies: (1) an online survey (n=106) to understand general smartphone camera usage behaviors related to information capture, as well as participants' experiences of orientation errors, and (2) a controlled lab study (n=16) to understand detailed document capture behaviors and to identify patterns in orientation errors. According to our online survey, 79.30% of the respondents reported experiencing orientation errors during document capture. In addition, our lab study showed that more than 90% of landscape capture tasks result in incorrect orientation. To solve this problem, we systematically analyzed the user behavior during document capture (e.g., video sequences and photographs taken or hand grip used) and propose a novel solution called ScanShot, which detects document capture time to help users correct orientation errors. ScanShot tracks the direction of gravity during document capture and monitors the users rotational or tilting movements of to update changes in orientation automatically. Our results confirm that document capture with 93.44% accuracy; in addition, our orientation update mechanism can reduce orientation errors by 92.85% using a gyroscope (for rotation) and 81.60% using an accelerometer (for micro-tilts). HighlightsWe study user behavior of document capturing using mobile camera by online study (n=106) and in-lab experiment (n=16).We analyze the erroneous orientation problem during document photo capturing.We propose a technique devised for inferring the user's document capture intention.We device two methods for correcting orientation errors while document capturing.The proposed methods show that more than 80% of error are be corrected.