SonicAnxiety

From CCRMA Wiki
Revision as of 21:19, 16 November 2014 by Vgrace (Talk | contribs)

Jump to: navigation, search

My controller consists of a sensors tracking breath and heart to control the chords, filtering and and rhythm a song in Ableton.

Similar or Inspiring pieces relating to breath and heart beat:

https://www.youtube.com/watch?v=GzLxrxGMMDo INSIDE-OUT is a source performance art piece by Andre Borges. The artist has a task mask on, wires attached to his chest and his hand is inside a box. We hear his breath and heart are amplified. Around 4 minutes in rattling and clanging sounds enter the room from an objects to the side of the performer. I appreciate the off-beat heart beat driving the piece forward and interacting with the other sounds heard.

http://www.instructables.com/id/Yoga-Breathalyzer/ Wind Sensor Used for Breath

http://gordophone.blogspot.com/2013/01/breath-sensing-101.html Breath Sensor with Tube

https://www.youtube.com/watch?v=n8fsMcMY2do In We Disappear (2nd half of Youtube V=video), the industrial sounds John Hopkins uses and the way he constantly manipulates the samples I admire. He keeps a relatively consistent chord profession but portrays it very differently depending on the percussive sounds laid over and additional processing effects used.

http://www.youtube.com/watch?v=rWixdA2xTSs Marina Abramovic & Ulay- Breathing In / Breathing Out (Death Itself)

http://www.youtube.com/watch?v=xEMbjnTJCHM Controlling a 4-story pipe organ with the Kinect- Ableton triggering ideas

http://www.youtube.com/watch?v=xC9XRpCOO6g Holly Herdon- Breathe

1) https://www.youtube.com/watch?v=5U_fN7NtSho
2) https://www.youtube.com/watch?v=9pn_b7OUO6I Robert Henke- using Granulating parameters

Other Inspiring Performance Art

https://www.youtube.com/watch?v=omDK2Cm2mwo&list=PLB9B4A00FC70FD6E3&index=35 RYOJI IKEDA : THE TRANSFINITE https://www.youtube.com/watch?v=3zoTKXXNQIU&list=PLB9B4A00FC70FD6E3&index=13

Technology out there already USB Breath Controller- http://www.tecontrol.se/products/usb-midi-breath-controller BC3A Microsoft Band - https://www.youtube.com/watch?v=CEvjulEJH9w

Things I need to have done for a minimal viable product: Have the breath, heart and body sensors synced with Ableton parameters with a flexible song created.

Things that you want to have done by the final deadline: All steps in the To Do list below.

Things that would be nice to have if I had unlimited time: Have accompanying visualizer responding to heart and breathing rate.


Necessary to make this idea happen:

• Build the breath sensor and attach to Arduino • Attach pulse sensor and accelerometer to Arduino. • Make MAX patch which reads heart and breathing data. • Have these triggers change specific aspects of song. eg: if heart rate is increasing 
after 10 seconds, shift chord quality. • Create conditions. e.g.: if amount of oxygen is increasing and is intervals between 
breaths are regular, play steady rhythm. If oxygen levels increase but breath timing irregular, then start to swing rhythm or play more syncopated beats. If oxygen levels are low and the breath timing is fast, the beats will become distorted or more randomized. • Test out conditions to see what works • Build 4 ft sq. cage and attach mask and heart sensors • Attach accelerometer to a belt or the side of cage • Make separate MAX patch for accelerometer data to be sent to Ableton to trigger loud 
crashing and scary samples • Attach additional pressure sensors to cage to use to make a slider for a track which can play breath and heart sounds imitating mine Cage sensors > Arduino > MAX > Ableton

parts I need to make it:

For Breath: • Barometric Sensor Breakout - T5403- pressure sensor which measures absolute pressure of surround air • 1 ft plastic or metal tubing For Heart Rate: • SEN-11574 Pulse Sensor • AD8232 Single Lead Heart Rate Monitor SEN-12650? Maybe... For Sharp Movements • RedBot Sensor - AccelerometerSEN-12589 • Muscle Sensor v3 KitSEN-11776 - possibly instead of accelerometer Ascetics • Metal bars, or thick metal sheet for cage • Door hinges and lock (possibly sensor for lock?) • Face mask for mout • Flexiforce Pressure Sensor - RTL-11380- for direct musical controls inside the cage 
(if necessary)

http://moderndevice.com/product/wind-sensor/ https://www.sparkfun.com/products/12650 https://www.sparkfun.com/products/11574

accelerometer https://www.sparkfun.com/products/12589

Cagedrawing.jpg

Cagewood.jpg Cage partially finished (no roof)

Hatch.jpg Roof: only opening in cage is through the hatch on top

Windarduino.jpg Here is the wind sensor hooked up to Arduino which has been programmed to send data to Max/MSP where it is processed in the following Max/ MSP and Max for Live patches attached.


So far I have a fully functional program linking WindSensor > Arudino > Max/MSP > Max for Live > Ableton which allows my breath control the song.


Here is the Hardware set up I used.

  Hardware Setup: 
 Wind Sensor Signals    Arduino
GND                           GND
+V                              5V
RV                              A1    
TMP                            A0   

This Arduino code I used to calibrate the Wind Sensor that I based on the information and code can be found on Paul Badger website [[1]]

There initially was an issue with the serial port which for some reason needed to be 57600. Also, the sensor was outputting a float value which could not be sent over serial and needed to be converted to an int.

 Serial.begin(57600);
  int i;
i = (int) WindSpeed_MPH; 
  
  
   //  Serial.write(255);	//send separator byte

Initially I used the serial print function to see the incoming data from the sensor which later was changed to write to output the data to Max.

   Serial.write(i);

SashaS.jpg The Max patch I used to receive data was based on a simplified version of Sasha Leitman's Max patch from Lab which received data from a single sensor.

From there I send out wind (the "s Wind" object) which is returned as inputs into my effects control patches included below:'


I made a sample Ableton session with selection of different chords and drum patters which I each put on different tracks. Track 1 has different loops of samples of sounds I recorded while building the cage. Track 2 are various short drum patters which also loop. Track 3 are single chords.

Abletoning3.jpg Currently I have written two different types Max for Live patches. Ones that constantly change the value of an effect based on the sensor output, and another which switches tracks based on a condition set.

First I made a simple bit crusher Audio effect I made in Max for Live and hooked it up to the Wind sensor data. When no wind is present the sensor reads around 35 and when you blow will full force into the wind sensor, the number decreases to 0. I scaled this data in the bit crusher to set the range of the BitRate between 1 and 24. The result is when you blow into the sensor more distortion is added to the track in Ableton.

Bitcrush.jpg My simple "Bit Crusher" Max for Live patch in Max for Live.

I made a Max patch which communicated with Ableton as a whole instead of directly controlling the sound in an individual track. This is different from a default MIDI or Audio Max for Live effect.

This is the first part of the patch below where it is labeled "Central Control" in brown.

Below the Central Control are the groups of slot triggers for tracks 1, 2 $ 3. If statements filter the direct Wind data to trigger different samples for Track 2 which has different banks of MIDI drum patterns.

Since the value the sensor is outputting, "Wind", is changing about every 20 ms, I found it inconvenient to use this method in Track 2 to accomplish my goal for smoothly changing the chords. I decided the program should wait as long a few breaths would take and find the average "Wind" which can be compared to the previous time interval to determine if amount of breath is increasing or decreasing.

The patch which stores all the recent Wind values and takes the average can be found on the lower part of the Max patch.

Control.jpg

The integer box at the bottom with the comment "???" is a strange invisible number which kept throwing off the "windsum" patch which should output only when the "sum" message is pressed. The strange thing is, is that the integer box in green directly connected to the outlet of the storing wind table does not show this strange low number. Luckily it is much lower then almost all of my averages so I just fixed the problem by filtering it out with an if statement below it before sending the wind sum (s windsum) For some reason, the number does not display in the integer box directly below the first wind table outlet. It does appear when the wind sum is sent out though...

Goals for final week:

1) Set up heart/ pulse sensors to Arduino, send out through serial port to Max patch. (see image below)

2) Modify Max patch to receive multiple numbers from different Arduino ports and send them out to more Max and Max for Live effects

3) Create new track triggering Max patch to control drums- this is so I can have breath averages controlling the chords and the heart controlling the drums separately

4) Continue composing

Heartsensor.jpg Heart/Pulse Sensor