What happens when two musicians who have developed performance practices with intelligent systems meet? What new forms of communication and expressivity emerge when systems capable of functioning as co-creators, instruments, and even compositions interact? How does this transform the musician’s role? These questions drive the experimentation between clarinetist Robert Ek and violinist Halla Steinunn Stefánsdóttir, who open the doors to their work in the second concert of the AURA series—at a time when the performance practice of so-called intelligent instruments is still in its early stages.Robert Ek works on developing augmented clarinets within the GEMM cluster at Piteå School of Music—Luleå University of Technology. Halla Steinunn Stefánsdóttir has explored a performance practice with intelligent performance platforms and their data curation at the Intelligent Instruments Lab. Her activity includes work with the Living Looper, designed by Victor Shepardson at the lab. She has recently expanded to incorporate Hans Jóhannsson's “rafklón” in her playing, which premiered at the opening of the Steina: Playback exhibition.

Natalie Eriksson has her roots in Norrtälje, north of Stockholm. She took her masters degree at the DePaul School of Music in Chicago, USA, and also studieed at the Malmö Academy of Music. In her everyday life shew works as a clarinet teacher at Kulturskolan in Trelleborg, but she also freelances around the Öresund region. Throughout her career, Natalie has specialised in contemporary art music and has performed works by Verrando, Kintsugi, Perini, and more.

Robert Ek has, as a clarinettist, specialised in contemporary music as a chamber musician and soloist. With great curiosity and dedication, he works together with composers to develop the repertoire for his instrument. He has recorded around 20 records and premiered a large number of works as a soloist and chamber musician. Since 2007 he is a member of the acclaimed ensemble Norrbotten NEO, the country’s only ensemble with a national mission to work with contemporary music. Norrbotten NEO premieres a number of works by Swedish and international composers every year. In 2012, the ensemble received fst’s interpretation award for their work with contemporary music. Robert has performed and collaborated with musicians, composers and conductors from different parts of the world. He likes to work in the border country between arts and has worked closely not only with composers but also with writers, choreographers and filmmakers. Robert has toured Europe, North America, Asia and Australia and played at festivals such as Warsaw autumn, Huddersfield Contemporary Music Festival, June in Buffalo, Faithful (Berlin), ISCM world music days in Stockholm and Tallinn, Musica Viva (Lisbon), Beijing Contemporary Music Festival.

In recent years, his work on developing the repertoire has focused on live electronics and developing the clarinet as an augmented instrument. Robert is also a PhD candidate at LTU since 2019. At the core of this doctoral project lies the iterative process where new electro-acoustic instrumental systems are designed and then used and tested in collaborative processes and in artistic practice. This oscillation is well described by Per Anders Nilsson (2011) as “design time” and “play time”. Where “design time deals with conception, representation, and articulation of ideas and knowledge outside time … and play time deals with embodied knowledge, bodily activity, and interaction in real time” (Nilsson, 2011). Adding to this framework is the reflective process of observing and analysing how the systems behave and interact with the performer. Performing with these systems requires a close relationship and intimate understanding of the system and the relationship between body and technology. This embodied knowledge that artistic research strives for is a felt knowledge.

Music of the Indeterminate Place is a collaborative composition by the TCP/Indeterminate Place quartet with networked hyperorgans. The members of the quartet perform from four locations, housing unique instruments: three different pipe organs and a bell tower in a town hall square. All instruments can be remotely controlled and the quartet interacts with them in several ways, none of which involve playing a conventional keyboard. The different locations are connected via the internet so that high-quality audio and data can be exchanged in real time, making it possible for the quartet to perform together while in different places.

In this piece, the quartet explores the qualities of the network through a new interaction design. Latency, network feedback, interconnectivity matrices, and other specificities of the network are reconfigured throughout the performance to form a music that is everywhere and nowhere: the Music of the Indeterminate Place.

"Pendulum Songs & Swings" emerged from an artistic research project led by clarinettist Robert Ek, investigating a musician’s movements in playing, using different artistic lenses. The research took place as a series of artistic lab sessions with Robert Ek as a performer, Lidia Wos as choreographer, Emmalisa Pauly as photographer, and Kent Olofsson as composer. A rich sonic, visual, and choreographic material was created that is currently being analysed and will be presented later in an audio-visual form. From this abundance of material a few small, singular, but generative ideas form the basis for the composition; sonic and musical ideas, as well as visual and choreographic: two clarinet multiphonics, and two movements while playing: one circular, rotational, and one as a pendulum. 

This audio paper seeks to understand the agency of sounds and instruments in designing intra-actions between humans and their environment. It further explores how different mediating technologies influence the ways professional musicians experience place through sound. In addition to the core research group, members of the contemporary music ensemble Norrbotten NEO participated in these site-specific explorations, which were devised as part of the ecological urban sound art project Invisible Sounds. Through a long-term collaborative artistic process, described in the audio paper, an hour-long composition titled Sounds by Water and Trees was produced (Hultqvist et al., 2024).

The mediation between humans and the environment was driven by audio technologies on site, utilising sensors, hydrophones, and surround sound microphone arrays, along with their deep entanglement with musical instruments. The interview materials and the original audio recordings have been further analysed by revisiting the multi-modal recordings and exploring the individual experiences of each participant through stimulated recall analysis (Östersjö et al., 2023). The audio paper expresses all these layers of signification while emphasising the individual experience expressed by the participating musicians. The project is characterised by recursive loops that flow continuously between subject and object, self and world, order and chaos in rhythmically cohering and complexifying patterns. We provide examples of how such complexifying patterns may emerge through a participatory and multidimensional engagement with sound in tandem with the mediation enabled by the technologies used.

This system was initially created by clarinettist Robert Ek in 2022 and it is

discussed in the article “Playing with resistance” accepted for NIME 2024. It

focuses on non-pitched sound material from the clarinet, like tongue rams, key

clicks, and different types of air sounds. The audio input from the clarinet is fed

into a modular synthesizer where it is processed. The module mainly used in this

patch is the Mutable Instruments Beads, a texture synthesizer working with

granulation. The sensor data from the physical gestures are converted to control

voltage (CV) to manipulate parameters in the modular setup. I use two types of

gestural information to control this system, namely bell elevation and tilt. The

elevation controls the size of the grains which in this patch results in pitch

modulation of the grains. This is mapped so that lowering the bell will result in a

lower pitch and a denser sonority. The tilt controls the amount of feedback in the

granular loop. This physical gesture, to tilt the clarinet, is a rather unusual

movement that creates a sense of resistance. Coupling this with feedback, where a

stronger resistance in the physical gesture results in larger feedback and more

complex sound, clearly connects to the resistance of the instrument. The patch

also uses the incoming audio as sensor data to generate a different CV signal: here

it’s directly connected to the resistance in the clarinet. This CV signal controls the

Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Copyright remains with the author(s).

DOI:http://dx.doi.org/10.1145/0000000.0000000

Music Proceedings of the International Conference on New Interfaces for Musical Expression

NIME’24, 4–6 September, 2024, Utrecht, The Netherlands##:2 Author #1 et al.

density in the grain generation, the more air pressure I apply to the bore in my

clarinet, the denser the generated sounds get. These three parameters are also

coupled to each other so that they behave differently in relation to each other’s

current state. The result is a system that provides clear connections between the

clarinet’s resistance, input modalities and the remediated sounding result. In one

way the data fed into this system are quite rudimentary, especially the gestural

input but how they interact with each other and the felt connection to the

instrument's resistance results in a system that feels and behaves in a complex and

expressively rich way.

Creating this system was informed by my use of the MiM sensor bell for more

than seven years. Given the extensive time I have spent working with the sensor

component of the system, my awareness of how certain bodily movement creates

particular gestural input is rather detailed. This understanding, combined with the

knowledge gained through analysing performance videos through an embodied

perspective, helps me to make informed decisions in the process of designing the

mapping that controls the intermedial translation.

Instrument design is not just a matter of hardware, it also concerns strategies for software mapping of input to output data. I will in this paper report on how an augmented clarinet that I, together with a team at LTU began developing in 2015 has continued to develop over the past seven years. The focus will be on the development of artistic applications within the system. Performers frequently describe the resistance of their instrument as a manifestation of the challenges they encounter when playing. One can argue that the goal of a skilled performer is to get rid of resistance, but it is in fact a central part of the relationship between performer and instrument. Acquiring technique and skill seems to be a way for the performer not to overcome resistance but to learn how the instrument responds to force. Realizing the importance of resistance in the artistic process we need to ask ourselves; how can we use this knowledge when creating new instruments? Returning to video documentation of performances with two different mappings and through stimulated recall analysis I seek a deeper understanding of how software, mapping and performance practice interacts, forming the basis of the artistic expression.

This global hyperorgan performance is a site-specific exploration, connecting sonic traces, archive recordings, interviews and field recordings to memory of place. The places share historical moments of division, and of reconciliation. Each location holds one or more pipe organs, which can be controlled remotely. In the creation of the piece, dividing the quartet in two duos is a fundamental building block. While the duos develop individual materials, the composition also explores how they can be united. One of the connecting points is the Fantasia in c-minor BWV 562 by J.S. Bach, which provides the basic material for the interaction with the pipe organs. 

Jenny Hettne arbetar innovativt och gärna i nära samarbete med musiker där hon utforskar musikens klangliga möjligheter. Hon komponerar för såväl orkester som mindre besättningar, ibland i kombination med elektronik. Hennes akustiska musik är starkt influerad av elektroakustisk musik och vice versa. Stycket As they flutter about för basklarinett och elektronik är komponerat i nära samarbete med och tillägnat klarinettisten Robert Ek. Musiken startar i basklarinettens djupaste register, i ett statiskt ljudlandskap av knappt skönjbara variationer, lätt böljande eller försiktigt skorrande. I styckets andra del expanderar rörelserna; vibrerar, fladdrar och skimrar. Solisten och den elektroniska stämman fungerar som en duo där de ibland byter funktion med varandra: speciella tekniker på instrumentet skapar en närmast elektronisk klang medan obearbetade klarinettinspelningar hörs i den elektroniska stämman.