A user interface evaluation environment using synchronized video, visualizations and event trace data

This paper presents a simple but very powerful technique to support user interface evaluation along with a prototype open environment — I-Observe, the Interface OBServation, Evaluation, Recording, and Visualization Environment. I-Observe's technique requires recording user interface sessions in multiple modalities, both as a trace of interesting events and through video images. It provides tools to allow the user interface evaluator either to analyze and visualize the event tracer data, or to combine the event trace and video modalities. The analyst using I-Observe can search the event stream for patterns of interesting or important user actions, then use the recorded timestamps associated with these actions to present only the sections of the video recording of interest. This allows the analyst to study, for example, all places where the user invokes a help system or a particular command to be observed, without manually searching the recording or sitting through long sessions of unrelated interactions. By combining the precise recording of automatic event trace capture with the rich contextual information that can be captured in a video and audio recording, this technique allows analyses to be performed that would not be practical with either media alone.