Towards Personalized Annotation of Webpages for Efficient Screen-Reader Interaction

To interact with webpages, people who are blind use special-purpose assistive technology, namely screen readers that enable them to serially navigate and listen to the content using keyboard shortcuts. Although screen readers support a multitude of shortcuts for navigating over a variety of HTML tags, it has been observed that blind users typically rely on only a fraction of these shortcuts according to their personal preferences and knowledge. Thus, a mismatch between a user's repertoire of shortcuts and a webpage markup can significantly increase browsing effort even for simple everyday web tasks. Also, inconsistent usage of ARIA coupled with the increased adoption of styling and semantic HTML tags (e.g., , ) for which there is limited screen-reader support, further make interaction arduous and frustrating for blind users. To address these issues, in this work, we explore personalized annotation of webpages that enables blind users to efficiently navigate webpages using their preferred shortcuts. Specifically, our approach automatically injects personalized 'annotation' nodes into the existing HTML DOM such that blind users can quickly access certain semantically-meaningful segments (e.g., menu, search results, filter options, calendar widget, etc.) on the page, using their preferred screen-reader shortcuts. Using real shortcut profiles collected from 5 blind screen-reader users doing representative web tasks, we observed that with personalized annotation, the interaction effort can be potentially reduced by as much as 48 (average) shortcut presses.