ReMap: Multimodal Help-Seeking

ReMap is a multimodal interface that enables searching for learning videos using speech and in-task pointing. ReMap extends multimodal interaction to help-seeking for complex tasks. Users can speak search queries, adding app-specific terms deictically. Users can navigate ReMap's search results via speech or mouse. These features allow people to stay focused on their task while simultaneously searching for and using help resources. Future work should explore how to implement more robust deictic resolution and more modalities.