Dream was an online live performance in early 2021 and was a collaboration between several partners, among them Royal Shakespeare Company, the Philharmonia Orchestra, Marshmallow Laser Feast, Manchester International Festival and EPIC Games. The stated goal of Dream was to explore new ways of interacting with audiences while bridging the disciplines of theatre, music, video games and other forms of digital media. The musical goals were to enable a high level of musical interactivity while at the same time coupling that with the use of high-quality recorded orchestral music. This article is not a study of the Dream project as a whole, but focuses on the interactive music that was generated in realtime by the actors and powered by Reactional Music. Apart from briefly discussing the technology itself, this article will describe and analyse some of the working processes involved in composing interactive music, as well as discuss some of the future implications of the use of technology from the video game industry in new areas such as live performances and audience interaction. The aim of this is to see if it’s possible to envision what kind of new artistic output that can come from these new technological environments.

The plot

A dreamlike journey into a forest: a story of transformation. Inspired by Shakespeare’s iconic play, A Midsummer Night’s Dream.

Puck invites you on a magical journey between dusk and dawn. With a cloud of fireflies lighting the way, you explore the forest and encounter four sprites, Moth, Peaseblossom, Cobweb and Mustardseed. But storm clouds gather on the horizon, threatening destruction.

Dream features a symphonic score based on recordings by the Philharmonia Orchestra, conducted by Principal Conductor & Artistic Advisor Esa-Pekka Salonen. The recordings are expanded by music created in real time by the movements of the performers. This living, dynamic soundtrack adapts and interacts with the narrative and the pre-recorded orchestral tracks. 

This paper describes the use of the Gestrument Engine as a tool to develop a more articulated understanding of the interaction between user and digital instruments in musical creativity. Through the use of the Gestrument Engine, the paper seeks to develop a new model, the Musical Interactivity Area. This is a visualisation of the area between a composed piece of music and a playable instrument, mapping out possibilities that arise when using tools where it is possible to both pre-define the confines of the input data as well as the structural and temporal musical content. The Gestrument Engine is used by defining several alternative versions of the same musical rules, and later positioning these versions within the Musical Interactivity Area to exemplify the model.