Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
An End-to-End Musical Instrument System That Translates Electromyogram Biosignals to Synthesized Sound
Department of Computing Goldsmiths, University of London, London, SE14 6NW, United Kingdom.
Luleå University of Technology, Department of Social Sciences, Technology and Arts, Music, Media and Theater. Fakultät Gestaltung, Universität der Künste Berlin.ORCID iD: 0000-0001-9685-4702
School of Computing Engineering and the Built Environment Edinburgh, Napier University, 10 Colinton Road, Edinburgh, EH10 5DT, United Kingdom.
Rebel Technology, C Sant Isidre 38, 08912, Badalona, Spain.
Show others and affiliations
2023 (English)In: Computer music journal, ISSN 0148-9267, E-ISSN 1531-5169, Vol. 47, no 1, p. 64-84Article in journal (Refereed) Published
Abstract [en]

This article presents a custom system combining hardware and software that senses physiological signals of the performer's body resulting from muscle contraction and translates them to computer-synthesized sound. Our goal was to build upon the history of research in the field to develop a complete, integrated system that could be used by nonspecialist musicians. We describe the Embodied AudioVisual Interaction Electromyogram, an end-to-end system spanning wearable sensing on the musician's body, custom microcontroller-based biosignal acquisition hardware, machine learning–based gesture-to-sound mapping middleware, and software-based granular synthesis sound output. A novel hardware design digitizes the electromyogram signals from the muscle with minimal analog preprocessing and treats it in an audio signal-processing chain as a class-compliant audio and wireless MIDI interface. The mapping layer implements an interactive machine learning workflow in a reinforcement learning configuration and can map gesture features to auditory metadata in a multidimensional information space. The system adapts existing machine learning and synthesis modules to work with the hardware, resulting in an integrated, end-to-end system. We explore its potential as a digital musical instrument through a series of public presentations and concert performances by a range of musical practitioners.

Place, publisher, year, edition, pages
MIT Press Journals , 2023. Vol. 47, no 1, p. 64-84
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Musical Performance
Identifiers
URN: urn:nbn:se:ltu:diva-108694DOI: 10.1162/comj_a_00672ISI: 001262355500008Scopus ID: 2-s2.0-85200370503OAI: oai:DiVA.org:ltu-108694DiVA, id: diva2:1891549
Funder
EU, European Research Council, (FP/2007-2013) / ERC Grant FP7-283771EU, Horizon 2020, grant agreement no. 789,825
Note

Godkänd;2024;Nivå 0;2024-08-22 (joosat);

Funder: French Agence Nationale de la Recherche (ANR-21-CE38-0018);

Full text license: CC BY

Available from: 2024-08-22 Created: 2024-08-22 Last updated: 2025-10-21Bibliographically approved

Open Access in DiVA

fulltext(1387 kB)226 downloads
File information
File name FULLTEXT01.pdfFile size 1387 kBChecksum SHA-512
64e42f303688a81b06327dc257266641be2f3573f704049e0a32993a29df1f0f9f4b28e1e47193b04b98e94199c6d158487ef336e777a2964eb23d2b2ab97929
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Authority records

Visi, Federico

Search in DiVA

By author/editor
Visi, Federico
By organisation
Music, Media and Theater
In the same journal
Computer music journal
Other Electrical Engineering, Electronic Engineering, Information Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 226 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 473 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf