SubjectBook: Hypothesis-Driven Ubiquitous Visualization for Affective Studies

Analyzing affective studies is challenging because they feature multimodal data, such as psychometric scores, imaging sequences, and signals from wearable sensors, with the latter streaming continuously for hours on end. Meaningful visual representations of such data can greatly facilitate insights and qualitative analysis. Various tools that were proposed to tackle this problem provide visualizations of the original data only; they do not support higher level abstractions. In this paper, we introduce SubjectBook, an interactive web-based tool for synchronizing, visualizing, exploring, and analyzing affective datasets. Uniquely, SubjectBook operates at three levels of abstraction, mirroring the stages of quantitative analysis in hypothesis-driven research. The top level uses a grid visualization to show the study's significant outcomes across subjects. The middle level summarizes, for each subject, context information along with the explanatory and response measurements in a construct reminiscent of an ID card. This enables the analyst to appreciate within subject phenomena. Finally, the bottom level brings together detailed information concerning the inner and outer state of human subjects along with their real-world interactions - a visualization fusion that supports cause and effect reasoning at the experimental session level. SubjectBook was evaluated on a case study focused on driving behaviors.