What are biosignals?
Biosignals are biological signals from the human body that can be constantly monitored through the use of technology. Most biosignals are electrical in nature and refer to changes in voltage across specific parts of the body (Tanaka & Ortiz, 2019).
How have biosignals been used in music and art?
One of the first examples of biosignals being used to enrich music is the BioMuse, which uses neuroelectric and myoelectric signals (EEG and EMG) to create a “biocontroller” that adds to instrumental performances (Knapp & Lusted, 1990; Tanaka, 1993). This device was created in the early 1990s, and has since paved the way for biosignals in music. More recently in 2017-8, Elaine Chew used electrocardiogram (ECG, or heartbeat) data, especially periods including arrhythmias, in her pieces, Arrhythmia Suite.
Biosignals such as electromyography (EMG) and mechanomyography (MMG) have been used in Atau Tanaka’s piece, Kagami, and Marco Donnarumma’s Ominous (Tanaka & Donnarumma, 2018). Other examples of biosignals used in musical performance include electrooculogram (EOG), mechanomyogram (MMG), galvanic skin response (GSR), c-reactive protein (CRP), and electrocardiograms (EKG/ECG) (Tanaka & Ortiz, 2019).
What are the benefits of using biosignals in music and art?
Biosignals allow for a deep connection between the performer and the music, as biological signal devices could be considered cyborg extensions of the human body, where the pieces embody the performers (Tanaka & Ortiz, 2019; McLuhan, 1994). These devices can also be used to incorporate active “choreography” into the musical performance in new ways (Tanaka, 1993). Biosignals can deeply connect performers to the music, and they can also be used for interaction with audience members.
How have EKGs been used before in music and art?
A specific type of biosignal is an EKG, or electrocardiogram, which measures the electrical signals of the heart. EKGs have been used in the past in The Heart Chamber Orchestra, which uses the signals from twelve different people to create a “living score” where the heartbeats are integral to the composition and change the outcome of the piece (Votava & Berger, 2012).
What is the Stethophone? How does it use EKGs?
The Stethophone is a musical device which captures heartbeat sounds in real time. In the Stethophone, EKG data is collected by a PulseSensor using an Arduino. The PulseSensor was created by World Famous Electronics and uses open source software available through GitHub. The EKG signal from the PulseSensor is then sent to Max/MSP through a serial port. Once the signal is recreated in Max/MSP, it is used as an envelope on the audio coming from a stethoscope microphone. By doing so, the external noise from the stethoscope can be filtered out. This dual approach to retrieving audio and eliminating external noise sets the Stethophone apart from current devices.
A prototype of the Stethophone was used in a performance of embody by Barbara Nerness, a multichannel performance featuring Michiko Theurer and Julie Zhu. The recording has been mixed down to binaural, so please wear headphones. More information on this piece can be found at http://www.barbaranerness.com/.
Live performance at Bing Concert Hall in May, 2019:
Another prototype of the Stethophone was used by The Living Earth Show to perform unbody by Barbara Nerness in December 2019, with documentation forthcoming.
Set-Up
Left: Barbara Nerness; Right: PulseSensor
Here are the steps you will need to take to run the Stethophone:
Connect the pulse sensor, stethoscope, Arduino, and computer in the configuration shown above. For the PulseSensor portion, you do not need the breadboard or LEDs; however, these can be used to test if the PulseSensor is working or not.
Download the Arduino file and make sure that the correct port is used (Tools > Port). Upload the code onto the Arduino and close the Arduino editor.
Download the Max MSP file. Replace the “j” in “port j” with the port number/letter used by the Arduino. To find out this number/letter, click on the “print” message and open the Inspector.
Click the trigger to start the Stethophone and turn on the audio.
Put the stethoscope over the heart to amplify the heartbeat. Put the pulse sensor on the finger.
Knapp, R. B., & Lusted, H. S. (1990). A Bioelectric Controller for Computer Music Applications. Computer Music Journal, 14(1), 42-47.
McLuhan, M. (1994). Understanding Media : the extensions of man. MIT Press.
Tanaka, A. (1993). Musical Technical issues in using Interactive Instrument Technology with
Application to the BioMuse. ICMC Proceedings, 124-126.
Tanaka, A., & Donnarumma, M. (2018). The Body as Musical Instrument. The Oxford Handbook of Music and the Body, 1-18.
Tanaka, A., & Ortiz, M. (2019). Gestural musical performance with physiological sensors, focusing on the electromyogram. In M. Lesaffre, P.-J. Maes, & M. Leman (Authors), The Routledge Companion to Embodied Music Interaction (pp. 420-428).
