Creative Performance in Play: A Synthesis of Audio, Visual and Narrative for Maximising Expressive Potentials in Gameplay

Abstract

This thesis explores expressive audio-visual performance in video games. It has been developed via the author’s creative and technical background in the field of electronic music. Primary aims of this research are: • Introduce key electronic music concepts into the field of video games in order to make gameplay as expressive as electronic music performance, by offering greater potential for original creativity in gameplay, and a more co-creative experience • Compare expressive attributes inherent in electronic music, to audio, visual and narrative elements of gameplay. Thus find analogs in those areas where similar levels of expressiveness can be developed and integrated as co-creative gameplay • Generate a conceptual framework, technical realization, and creative realization (playable game providing narrative framing for performative interactions) to contribute to the field of expressive and co-creative performance in games This research is primarily intended as a contribution to the field of video games. By beginning with a small scale-focus on moment to moment gameplay this conceptual framework engages with expressive potentials already existent in the field of electronic music. By mapping dynamic gameplay parameters (e.g. avatar movement, proximities, cursor interactions) to musical parameters (e.g. DSP effects settings), a gameplay prototype was developed, rehearsed and iterated upon. This allowed for the development of performance objects (e.g. synthesizers, camera filters, etc), iterative refinement of any given object towards more expressive gameplay, and development of inter-object-relationships. The combination of a large diversity of objects, and a small set of data-flow languages facilitates a large array of interdependent routings between audio, visual and narrative functions. Expressive potentials of audio-visual gameplay combine with narrative context to enable a co-creative experience for the player where they can situate themselves as an interdependent unit in the shifting contexts of performative play.