Musical Brush: Exploring Creativity Through an AR-Based Tool for Sketching Music and Drawings

The economic growth and social transformation in the 21st century are hardly based on creativity. To help with the development of this skill, the concept of Creativity Support Tools (CST) was proposed. In this paper, we introduce Musical Brush (MB), an artistic mobile application whose main focus is to allow novices to improvise music while creating drawings. We investigated different types of interactions and audiovisual feedbacks in the context of a mobile application that combines music with drawings in a natural way, measuring their impact on creativity support. In this study, we tested different user interactions with real-time sound generation, including 2D drawings, three-dimensional device movements, and visual representations on Augmented Reality (AR). A user study was conducted to explore the support for creativity of each setup. Results showed the suitability of the association of Musical Brush with augmented reality for creating sounds and drawings as a tool that supports the exploration and expressiveness.