This paper presents research based on the creation and development of two Brain Controlled Interface (BCI) based film experiences. The focus of this research is primarily on the audio in the films; the way that the overall experiences were designed, the ways in which the soundtracks were specifically developed for the experiences and the ways in which the audience perceived the use of the soundtrack in the film. Unlike traditional soundtracks the adaptive nature of the audio means that there are multiple parts that can be interacted with and combined at specific moments. The design of such adaptive audio systems is something that is yet to be fully understood and this paper goes someway to presenting our initial findings. We think that this research will be of interest and excite the Audio-HCI community.
This workshop examines the interplay between people, musical instruments, performance and technology. Now, more than ever technology is enabling us to augment the body, develop new ways to play and perform, and augment existing instruments that can span the physical and digital realms. By bringing together performers, artists, designers and researchers we aim to develop new understandings how we might design new performance technologies. Participants will be actively encouraged to participant, engaging with other workshop attendees to explore concepts such as; immersion, augmentation, emotion, physicality, data, improvisation, provenance, curation, context and temporality, and the ways that these might be employed and unpacked in respect to both performing and understanding interaction with new performance-based technologies that relate to the core themes of immersion and emotion.
#Scanners is a bio-responsive digital arts experience that blurs the lines between cinematics and neuroscience. Using a commercially available wireless EEG device, #Scanners presents a specially commissioned film that is dynamically altered both visually and aurally in accordance with the viewer's levels of Attention and Meditation, as calculated by the EEG device. The system has been demonstrated to audiences across Europe and has allowed us to explore design opportunities around both extents of, and awareness of control with otherwise passive multimedia experiences. At CHI2016, viewers will be able to experience the #Scanners system, watch their own personalized version of the film. Spectators will be able to see viewers, their levels, and their effect on the film.
We report on the design and deployment of systems for the performance of live score accompaniment to an interactive movie by a Networked Musical Ensemble. In this case, the audiovisual content of the movie is selected in real time based on user input to a Brain-Computer Interface (BCI). Our system supports musical improvisation between human performers and automated systems responding to the BCI. We explore the performers' roles during two performances when these socio-technical systems were implemented, in terms of coordination, problem-solving, managing uncertainty and musical responses to system constraints. This allows us to consider how features of these systems and practices might be incorporated into a general tool, aimed at any musician, which could scale for use in different performance settings involving interactive media.
In the overlap between Human-Computer Interaction (HCI) and Cinematics, sits an interest in physiological responses to experiences. Focusing particularly on brain data, Neurocinematics has emerged as a research field using Brain-Computer Interface (BCI) sensors. Where previous work found inter subject correlations (ISC) between brain measurements of people watching movies in constrained conditions using functional magnetic resonance imaging (fMRI), we seek to examine similar responses in more naturalistic settings using functional Near Infrared Spectroscopy (fNIRS). fNIRS has been shown to be highly suitable for HCI studies, being more portable than fMRI and more tolerant of many natural movements than Electroencephalography (EEG). Early results found significant ISC, which gives a lot of hope and potential for using fNIRS in Neurocinematics.
Conventional wisdom has it that trustworthy autonomous systems (AS) should be explainable, dependable, controllable and safe tools for humans to use. Reflecting on a portfolio of artistic applications of TAS leads us adopt an alternative stance and to propose five provocative challenges for AS: that they should look beyond failure to improvisation, beyond explainability to interpretation, beyond control to surrender, beyond caution to playfulness and beyond being tools to becoming co-creators. We reflect on how these challenges imply new considerations of trustworthiness in terms of artistic competence, sincerity and steadfastness, and of responsible innovation in terms of responsible irresponsibility that empowers humans to playfully explore the human-like qualities and boundaries of the technologies.
Interactive films have been around for almost a century, yet they have suffered repeatedly from critical, commercial and interactional failings. We propose that brain-computer interfaces can offer interactions with narratives and encourage cinematic engagement by minimising active control. We ask what are the problems inherent to interactive cinema? Can real-time interactions via a brain-computer interface (BCI) construct cinematic content? And how do groups of individuals experience brain-controlled cinema designed for individual, shared or distributed control? Our review of related work motivates the interactional choice of using Passive BCI with real-time cinematic construction to synchronise rhythms of the viewers blinking, Attention and Meditation to the rhythms of cinema. We use the Performance Led Research in-the-Wild methodology to probe public deployments of our films, and we describe user interactions in-the-Wild during screenings of multiple designs of two interactive films: three single user, three multi user, and a live score performance. Our descriptions of BCI mappings to cinematic techniques and production strategies to produce interactive content efficiently contributes to the understanding of practical interactive cinema production. In our results we define 1) different stages of control; discovery, conscious and unconscious, 2) awareness of the affective loop, 3) a shifting prominence of engagement between the narrative, the visual qualities and the agency of users’ interactions. We offer a dynamic view of control; people’s experiences are shifting from awareness of their self, the film, and their control. Our hyper-scanning multi-user study introduces the concept of effects moving across groups, working together to produce engaging experiences, and instances of group members disrupting other’s experience by deciding to unilaterally take control of the film. Our discussion contributes to our understanding of passive interactions with narrative systems. Our research contributions include our insights into seven designs of two brain-controlled films. We define two taxonomies, of control and group control, and produce insights into value to audiences of brain-controlled films. We show the development of affective loops of physiological response and cinematic content, and provide new design directions and practical implications for interactive filmmakers.
What for and how will we design children's technologies in the transhumanism age, and what stance will we take as designers? This paper aims to answer this question with 13 fictional abstracts from sixteen authors of different countries, institutions and disciplines. Transhumanist thinking envisions enhancing human body and mind by blending human biology with technological augmentations. Fundamentally, it seeks to improve the human species, yet the impacts of such movement are unknown and the implications on children's lives and technologies were not explored deeply. In an age, where technologies such as under-skin chips or brain-machine interfaces can clearly be defined as transhumanist, our aim is to reveal probable pitfalls and benefits of those technologies on children's lives by using the power of design fiction. Thus, main contribution of this paper is to create diverse presentation of provocative research ideas that will foster the discussion on the transhumanist technologies impacting the lives of children in the future.
'%3e%3cpath fill='%23012169' d='M0 0v30h60V0z'/%3e%3cpath stroke='%23fff' stroke-width='6' d='m0 0 60 30m0-30L0 30'/%3e%3cpath stroke='%23C8102E' stroke-width='4' d='m0 0 60 30m0-30L0 30' clip-path='url(%23b)'/%3e%3cpath stroke='%23fff' stroke-width='10' d='M30 0v30M0 15h60'/%3e%3cpath stroke='%23C8102E' stroke-width='6' d='M30 0v30M0 15h60'/%3e%3c/g%3e%3c/svg%3e)
