Personalized Interactive Storyboarding utilizing Content Based Multimedia Retrieval

Abstract An explosion of digital multimedia technologies that permit quick and easy uploading of any multimedia file to the web, coupled with the rapid advancement of hardware and software have resulted in petabytes of multimedia data being available on the internet. In this paper, we provide a framework that rapidly integrates these data for personalized storyboarding utilizing Content Based Multimedia Retrieval (CBMR) techniques. A storyboard is a narrative of audio, text, video and images of a particular topic that are linked systematically. People have different abilities to understand a story/topic according to their learning styles coupled with experience, age, knowledge, gender etc. Personalized storyboard multimedia learning help the user dictate the learning process in a creative way through multiple means of representation, expression and engagement. This process involves active learning, both behavioral as well as cognitive. Learning is constructive, and information learned is remembered at a deeper level. The use of multimedia promotes meaningful learning that can be transferred or generalized to other situations. In this paper, various learning styles are discussed. The goal of the research was to design and implement a novel approach that integrates human reasoning with computerized algorithms for multimedia storyboarding for learning. By systematically coupling human reasoning and computerized process, we attempt to minimize the barrier between the human's cognitive model of what they are trying to storyboard and the computers understanding of the user's task. We present our model which utilizes a fusion of content based image retrieval, content based video retrieval, content based audio retrieval, and text based retrieval techniques with human computer interaction based relevance feedback to enhance the learning process through personalized multimedia storyboarding. To illustrate the advantages of this model in a greater detail, we showcased the concepts of this model utilizing the game of cricket. We summarize the paper by discussing the future in this area.