Lab 3: Sensors and Physical Interface Design
The goal of this lab is to control Faust synthesizers with sensors connected to the Teensy. Various mapping strategies are also introduced.
Electronic Basics
Sensors and Teensy Basics
- A wide range of sensors act as resistors (e.g., Force Sensitive Resistors, photoresistors, SoftPots, flex sensors, etc.) and can be used to dynamically change the amount of current circulating in a circuit.
- Microcontrollers such as the Teensy can be used to digitize the signal delivered by a sensor in a circuit (which is the same as measuring the amount of current in the system at regular time intervals).
- Connect your Teensy to your breadboard as follows (GND to the blue - strip and 3.3v to the red + strip):
- WARNING: the 5v pin of the Teensy can completely fry it: make sure that you NEVER use it
- Make a circuit connecting a photoresistor (part of your kit) to the analog input 0 (A0) of the Teensy (the diagram says FSR but the circuit should be the same for your photoresistor):
- If you don’t know where the A0 pin is on your Teensy you can refer to the “Pinouts” section on this webpage: https://www.pjrc.com/store/teensy40.html
- Notice the use of a 10k pull down resistor.
- Let’s now retrieve the photoresistor signal on your computer and plot it in the debugger! First, upload this program to your Teensy:
void setup() {
Serial.begin(9600); // initializing serial port
}
void loop() {
int sensorValue = analogRead(A0); // retrieving sensor value on Analog pin 0
Serial.println(sensorValue);
delay(30); // wait for 30ms
}
- Note that we pause the loop for 30ms at every cycle to not overload the serial output.
- Open the serial debugger (the loop icon on the top right corner of the Arduino software) and make sure that your system is working by covering the photoresistor with your hand. Note that the output value is a 10 bits integer (value between 0 and 1023). If nothing is plotted, make sure that the right port has been selected in
Tools/Port
- We now want to control the parameter of a Faust-generated sound processing object with the photoresistor. Reuse the Faust sawtooth oscillator program from the Faust-Teensy tutorial (https://faust.grame.fr/doc/tutorials/index.html#band-limited-sawtooth-oscillator-on-the-teensy), compile it as a Teensy audio object and import it your Arduino/Teensy program as such:
#include <Audio.h>
#include "FaustSawtooth.h"
FaustSawtooth faustSawtooth;
AudioOutputI2S out;
AudioControlSGTL5000 audioShield;
AudioConnection patchCord0(faustSawtooth,0,out,0);
AudioConnection patchCord1(faustSawtooth,0,out,1);
void setup() {
AudioMemory(2);
audioShield.enable();
audioShield.volume(0.1);
}
void loop() {
int sensorValue = analogRead(A0);
float frequency = sensorValue + 50;
faustSawtooth.setParamValue("freq",frequency);
}
sensorValue
is used to compute frequency
(that’s what we call the “mapping”) which in this case will be a value between 50 and 1073Hz.
- Shading the photoresistor with your hand should now change the frequency of the generated sawtooth.
Exercise: Controlling Multiple Synth Parameters
- Try to integrate one of the potentiometers of your kit to your circuit and plug it to one of the available analog inputs of the Teensy. Since the audio shield uses some of the pins of the Teensy, you should check which ones are actually “free” in the “Signals to Teensy” section of https://www.pjrc.com/store/teensy3_audio.html.
- If you don’t know how to connect the potentiometer to the Teensy, a quick google search should probably help you ;).
- Add a gain control parameter to the Faust program and control it with the potentiometer.
Discrete Events
Looping Through Notes
The goal of this section is to make an instrument with the following behavior:
- it should loop through the notes of a simple melody,
- every time a button is pushed, a new note is triggered,
- when the button is not pressed, we hear no sound.
First, add a button to your circuit. The buttons provided in your kits have 4 pins (2 sets of 2 pins connected to each other). Place the button as follows on the breadboard:
Then connect one of the 2 pins to power and the other pin to the digital input 0 of your Teensy. You should add a pull-down resistor to this pin too. Note that an analog input could have been used as in the previous examples, but since we’re retrieving a discrete signal it doesn’t really make sense.
- Compile the following program in the Faust Web IDE for the Teensy (you can give the name you want, in the next steps of the lab we kept the same name as before:
FaustSawtooth
):
import("stdfaust.lib");
freq = hslider("pitch",60,0,127,1) : ba.midikey2hz;
gate = button("gate");
process = os.sawtooth(freq)*gate;
- Note that the frequency here is computed from a MIDI pitch number.
- On the Teensy side, run the following program:
#include <Audio.h>
#include "FaustSawtooth.h"
FaustSawtooth faustSawtooth;
AudioOutputI2S out;
AudioControlSGTL5000 audioShield;
AudioConnection patchCord0(faustSawtooth,0,out,0);
AudioConnection patchCord1(faustSawtooth,0,out,1);
int notes[] = {60,60,60,62,64,62,60,64,62,62,60}; // the melody
int cnt = 0; // index to loop through the notes
bool change = true; // did the note status changed?
bool on = false; // what is the note status?
void setup() {
pinMode(0, INPUT); // configuring digital pin 0 as an input
AudioMemory(2);
audioShield.enable();
audioShield.volume(0.1);
}
void loop() {
if (digitalRead(0)) { // button is pressed
if(!on) change = true; // did the status of the button changed
on = true;
}
else {
if(on) change = true; // did the status of the button change?
on = false;
}
if(change){ // if the status of the button changed
if(on){ // if the button is pressed
faustSawtooth.setParamValue("gate",1);
faustSawtooth.setParamValue("pitch",notes[cnt%11]);
cnt++; // next note
}
else{
faustSawtooth.setParamValue("gate",0);
}
change = false; // status changed
}
}
Notes are saved in an integer array as MIDI note numbers. When digital pin 0 receives a signal, the on
variable is set to true
and vice versa. The change
variable is used to make sure that we only send messages when necessary.
Exercise
Continuous parameters can be controlled in parallel of this button/looping mechanism. Add a lowpass filter to your sawtooth oscillator, e.g.,:
process = os.sawtooth(freq)*gate : fi.resonlp(ctFreq,5,0.5);
and control the value of the cutoff frequency with the photoresistor on your circuit. Spend some time thinking about the mapping of the cutoff frequency: what works best in this context?
Hint:
The min
and max
functions are often useful when mapping a sound synthesis parameter to a sensor. For instance, you probably noticed that the photoresistor doesn’t “fully exploit” the 0-1023 range offered by the 10bits converter. A common practice is to normalize the sensor output so that you can fully take advantage of its range. For example, say that you see that the output range of your sensor is 500-900, you could normalize its output with something like:
normSens = min(1023,max(0,sens - 550)*3.3);
Here subtracting 550 to the raw sensor output (sens
) and then comparing that to 0 with max
will make sure that the lowest value of the sensor is 0. Multiplying the result of this operation by 3.3 and comparing the result to 1023 with min
will ensure that the highest value of the sensor is 1023.
Assignment (Due on Jan. 29, 2020)
- Design an instrument running on the Teensy combining a bunch of sensors and a sound synthesizer implemented in Faust.
- Feel free to explore the various sensors available in the MaxLab (since you’re just making a “one time” prototype, make sure that you put them back where you found them once you’re done).
- Your instrument should involve at least one discrete element (i.e., a button) and one continuous one (i.e., a potentiometer).
- Think about the mapping strategies you’ll be using on the sensors (e.g., Does it make percussive sounds? Are all the parameters linear as opposed to logarithmic, for example? Etc.).
- Make sure that your instrument is expressive, artful, beautiful, playable, etc. You could obviously fulfill the requirements for this assignment by using any of the code of this lab, but we expect more: be creative!
- Think about the form (vs. function) of your instrument. Feel free to wrap it into something, dress it, etc. Don’t be afraid to use duct tape for that.
- Make a demo video of your instrument. The content of this video is pretty open. The only requirements are that it should contain a short music statement, it shouldn’t be boring, and it shouldn’t exceed 1:30 mins.
- Post the video on YouTube and send the link to Doga and Romain along with the Faust code of your instrument by Jan. 29, 2020.
- Good luck and have fun!
Submissions
- Raul Altosaar
- Monica Ngozi Anuforo
- Noah Louis Bailyn
- Andrea Baldioceda
- Noah Nathan Berrie
- Hannah Lee Choi
- Engin Daglik
- Richard Chaffin Givhan
- David Mendoza
- Nolan Ashvin Miranda
- John Mistele
- Mike Robert Mulshine
- Jeremy Thomas Raven
- Samantha Elinon Silverstein
- Kathlynn Marie Simotas
- Jan Stoltenberg
- Kangrui Xue