The Mobile Synth Workshop Series took place at CCRMA (Stanford University) between June 19-30, 2017. They allowed participants to both write software and make hardware interventions on their mobile device, transforming it into a custom musical instruments. This page presents some of the workshops' material as well as the final projects done by the students who took them.
Click on the picture to download the sources.
Click on the picture to download the sources. GitHub page: https://github.com/edmondhowser/somethingelse.
Click on the picture to download the sources.
This example code shows how to control a SmartKeyboard app using a Teensy board by sending MIDI messages. The app is based on a sawtooth and a triangle oscillators that are switched on and off by the Teensy depending on the needs of the performance.
"MIDI" USB type has to be chosen when uploading the code to the Teensy!!
#include <Bounce.h>
int ccFreqSaw = 10; // MIDI CC for the pitch of the sawtooth
int ccFreqTri = 11; // MIDI CC for the pitch of the triangle
int ccSwitch = 12; // MIDI CC to switch between the saw and the tri
int midiChannel = 0;
int sawNotes[] = {64,65,67}; // MIDI notes for the saw
int triNotes[] = {80,78,77}; // MIDI notes for the tri
void setup() {
}
void loop() {
// Switching between tri, saw, or both
int instrSwitch = random(3);
usbMIDI.sendControlChange(ccSwitch,instrSwitch,midiChannel);
for(int i=0; i<3; i++){
// Reading and sending notes
if(instrSwitch == 0 || instrSwitch == 2){
usbMIDI.sendControlChange(ccFreqSaw,sawNotes[i],midiChannel);
}
if(instrSwitch == 1 || instrSwitch == 2){
usbMIDI.sendControlChange(ccFreqTri,triNotes[i],midiChannel);
}
delay(500); // wait for 500 ms
}
}
declare interface "SmartKeyboard{
'Number of Keyboards':'1',
'Max Keyboard Polyphony':'0',
'Keyboard 0 - Number of Keys':'1',
'Keyboard 0 - Send Freq':'0',
'Keyboard 0 - Piano Keyboard':'0'
}";
import("stdfaust.lib");
freq0 = hslider("freq0[midi:ctrl 10]",50,0,127,1) : ba.midikey2hz : si.smoo;
freq1 = hslider("freq1[midi:ctrl 11]",50,0,127,1) : ba.midikey2hz : si.smoo;
switch = hslider("switch[midi:ctrl 12]",0,0,127,1) : int;
process = os.sawtooth(freq0)*((switch == 0) | (switch == 2)) +
os.triangle(freq1)*((switch == 1) | (switch == 2)) : *(0.5) <: _,_;
SmartKeyboard
App With SensorsThis example shows how to control the frequency of a sawtooth wave oscillator with an external sensor (an FSR for instance) plugged to one of the analog inputs of the Teensy.
#include
bool debug = false;
void setup() {
if(debug){
Serial.begin(9600); // initializing serial port
}
}
void loop() {
int sensorValue = analogRead(A0); // retrieving sensor value on Analog pin 0
if(debug){
Serial.println(sensorValue); // printing sensor value
}
else{
int midiCC = 10; // must match the faust configuration
int midiValue = sensorValue*127/1024; // value between 0-127
int midiChannel = 0; // actually doesn't matter
usbMIDI.sendControlChange(midiCC,midiValue,midiChannel); // sending control message
}
delay(30); // wait for 30ms
}
import("stdfaust.lib");
declare interface "SmartKeyboard{
'Number of Keyboards':'1',
'Max Keyboard Polyphony':'0',
'Keyboard 0 - Number of Keys':'1',
'Keyboard 0 - Send Freq':'0',
'Keyboard 0 - Piano Keyboard':'0'
}";
freq = hslider("frequency[midi:ctrl 10]",100,100,2000,1) : si.smoo;
process = os.sawtooth(freq) <: _,_;
This code is based on the ideas behind Equilibrium by Jordan Nobles. The original score can be found on www.jordannobles.com collection of Open Score Works (1997-2016) Jordan Nobles.
The interface is a set of five keyboard with 7 buttons. The first button intentionally is silent. Each row represents a set of fifths.
Instructions (based upon original score)
Play freely in your own time. Only ever play the given notes, in a random order. Players can pause at any time to breathe and to listen before proceeding all notes should be soft and quiet throughout the piece. The duration of the piece is variable but should probably be decided on ahead of time there are no parts or formal score... Any musicians play freely from available notes. There is no pulse, tempo, measures or exact synchronization.
// Edmond Howser
// CCRMA Mobile Synth Workshop 6/2017
import("stdfaust.lib");
declare interface "SmartKeyboard{
'Number of Keyboards':'5',
'Max Keyboard Polyphony':'1',
'Keyboard 0 - Number of Keys':'7',
'Keyboard 1 - Number of Keys':'7',
'Keyboard 2 - Number of Keys':'7',
'Keyboard 3 - Number of Keys':'7',
'Keyboard 4 - Number of Keys':'7',
'Keyboard 0 - Piano Keyboard':'0',
'Keyboard 1 - Piano Keyboard':'0',
'Keyboard 2 - Piano Keyboard':'0',
'Keyboard 3 - Piano Keyboard':'0',
'Keyboard 4 - Piano Keyboard':'0',
}";
g = hslider("g[acc: 0 1 -10 0 10]",0.5,0,1.0,0.01) : si.smoo;
key = hslider("key",0,0,6,1);
keyboard = hslider("keyboard",0,0,4,1);
t = button("gate");
// Pitches & Frequencies
XX = 0;
C1 = 32.70;
G1 = 49.00;
C2 = 65.41;
D2 = 73.42;
E2 = 82.41;
F2 = 87.31;
G2 = 98.00;
A2 = 110.00;
B2 = 123.47;
C3 = 130.81;
D3 = 146.83;
E3 = 164.81;
F3s = 185.00;
G3 = 196.00;
A3 = 220.00;
B3 = 246.94;
D4 = 293.66;
E4 = 329.63;
F4s = 368.99;
B4 = 493.88;
C4s = 277.18;
G4s = 415.30;
A4 = 440.00;
C5s = 554.37;
D5s = 622.25;
E5 = 659.25;
F5s = 739.99;
G5s = 830.61;
D6s = 1244.51;
A6s = 1864.66;
E7s = 2783.83;
B7s = 4186.01;
// Frequencies for Keyboard 0
freq0 = (keyboard == 0)*(key==0)*XX + (keyboard == 0)*(key==1)*G2 +
(keyboard == 0)*(key==2)*D3 + (keyboard == 0)*(key==3)*A3 +
(keyboard == 0)*(key==4)*E4 + (keyboard == 0)*(key==5)*B4 +
(keyboard == 0)*(key==6)*F5s;
// Frequencies for Keyboard 1
freq1 = (keyboard == 1)*(key==0)*XX + (keyboard == 1)*(key==1)*F2 +
(keyboard == 1)*(key==2)*C3 + (keyboard == 1)*(key==3)*G3 +
(keyboard == 1)*(key==4)*D4 + (keyboard == 1)*(key==5)*A4 +
(keyboard == 1)*(key==6)*E5;
// Frequencies for Keyboard 2
freq2 = (keyboard == 2)*(key==0)*XX + (keyboard == 2)*(key==1)*E2 +
(keyboard == 2)*(key==2)*B2 + (keyboard == 2)*(key==3)*F3s +
(keyboard == 2)*(key==4)*C4s + (keyboard == 2)*(key==5)*G4s +
(keyboard == 2)*(key==6)*D5s;
// Frequencies for Keyboard 3
freq3 = (keyboard == 3)*(key==0)*XX + (keyboard == 3)*(key==1)*D2 +
(keyboard == 3)*(key==2)*A2 + (keyboard == 3)*(key==3)*E3 +
(keyboard == 3)*(key==4)*B3 + (keyboard == 3)*(key==5)*F4s +
(keyboard == 3)*(key==6)*C5s;
// Frequencies for Keyboard 4
freq4 = (keyboard == 4)*(key==0)*XX + (keyboard == 4)*(key==1)*C2 +
(keyboard == 4)*(key==2)*G2 + (keyboard == 4)*(key==3)*D3 +
(keyboard == 4)*(key==4)*A3 + (keyboard == 4)*(key==5)*E4 +
(keyboard == 4)*(key==6)*B4;
signal0 = os.osc(freq0);
signal1 = os.osc(freq1);
signal2 = os.osc(freq2);
signal3 = os.osc(freq3);
signal4 = os.osc(freq4);
envelope0 = en.adsr(5,2.5,80,5,t) * g;
envelope1 = en.adsr(5,2.5,80,5,t) * g;
envelope2 = en.adsr(5,2.5,80,5,t) * g;
envelope3 = en.adsr(5,2.5,80,5,t) * g;
envelope4 = en.adsr(5,2.5,80,5,t) * g;
synth = signal0*envelope0*(keyboard == 0) +
signal1*envelope1*(keyboard == 1) +
signal2*envelope2*(keyboard == 2) +
signal3*envelope3*(keyboard == 3) +
signal4*envelope4*(keyboard == 4);
process = synth <: _,_;
// Casey Kim - "Phlute"
// CCRMA Mobile Synth Workshop 6/2017
declare interface "SmartKeyboard{
'Number of Keyboards':'2',
'Max Keyboard Polyphony':'0',
'Inter-Keyboard Slide':'1',
'Mono Mode':'1',
'Keyboard 0 - Number of Keys':'13',
'Keyboard 1 - Number of Keys':'13',
'Keyboard 0 - Lowest Key':'72',
'Keyboard 1 - Lowest Key':'60',
'Rounding Mode':'0'
}";
import("stdfaust.lib");
freq = hslider("freq",440,50,5000,0.01);
ctfreq = hslider("ctfreq[acc: 0 0 -10 0 10]",3000,300,5000,0.01) : si.smoo;
vib = hslider("vib[acc: 1 0 -10 0 10]", 0,0,0.02,0.01);
process = an.amp_follower(0.17)*os.triangle(freq + os.osc(5)*vib*freq) : fi.resonlp(ctfreq,7,1)
<: dm.zita_rev1;
// Marit Brademann
// CCRMA Mobile Synth Workshop 6/2017
declare interface "SmartKeyboard{
'Number of Keyboards':'1',
'Max Keyboard Polyphony':'0',
'Keyboard 0 - Number of Keys':'2',
'Keyboard 0 - Send Freq':'0',
'Keyboard 0 - Piano Keyboard':'0'
}";
import("stdfaust.lib");
key = hslider("key",0,0,1,1) : int;
record = key;
frequency = hslider("frequency[acc: 0 0 -10 0 10]",1000,10,2000,0.01) : si.smoo;
del = hslider("del[acc: 1 0 -10 0 10]",1023,0,2047,0.01) : si.smoo;
cnt0 = +(1)~*(record) : -(1) : %(192000);
cnt1 = +(1)~*(1-record): -(1) : %(192000);
process = rwtable(192000, 0.0, cnt0, _, cnt1)*(1-record : si.smoo) :_*(os.osc(frequency)*0.5 + 0.5) : de.fdelay4(2048,del);
Based on Henry Cowell's and Léon Theremin's Rhythmicon and on a tuning system by la monte young, as described by Kyle Gann.
// Matt Mitchell
// CCRMA Mobile Synth Workshop 6/2017
declare interface "SmartKeyboard{
'Number of Keyboards':'3',
'Max Keyboard Polyphony':'10',
'Keyboard 0 - Number of Keys':'5',
'Keyboard 1 - Number of Keys':'5',
'Keyboard 2 - Number of Keys':'5',
'Keyboard 3 - Number of Keys':'5',
'Keyboard 4 - Number of Keys':'5',
'Keyboard 0 - Piano Keyboard' : '0',
'Keyboard 1 - Piano Keyboard' : '0',
'Keyboard 2 - Piano Keyboard' : '0',
'Keyboard 3 - Piano Keyboard' : '0',
'Keyboard 4 - Piano Keyboard' : '0',
'Keyboard 0 - Key 0 - Label':'49/32',
'Keyboard 0 - Key 1 - Label':'147/128',
'Keyboard 0 - Key 2 - Label':'441/256',
'Keyboard 0 - Key 3 - Label':'1323/1024',
'Keyboard 1 - Key 0 - Label':'7/4',
'Keyboard 1 - Key 1 - Label':'21/16',
'Keyboard 1 - Key 2 - Label':'63/32',
'Keyboard 1 - Key 3 - Label':'189/128',
'Keyboard 1 - Key 4 - Label':'567/512',
'Keyboard 2 - Key 0 - Label':'1/1',
'Keyboard 2 - Key 1 - Label':'3/2',
'Keyboard 2 - Key 2 - Label':'9/8',
'Keyboard 0 - Send X':'0',
'Keyboard 1 - Send X':'0',
'Keyboard 2 - Send X':'0',
'Keyboard 0 - Send Y':'0',
'Keyboard 1 - Send Y':'0',
'Keyboard 2 - Send Y':'0',
'Rounding Mode':'0'
}";
import("stdfaust.lib");
keyboard = hslider("keyboard",0,0,2,1) : int;
key = hslider("key",0,0,4,1) : int;
keyID = (keyboard*5) + key;
gate = button("gate") : si.smoo;
pitchref = 220;
timeref = hslider("timeref[acc: 0 0 -10 0 10]", 2,0.1,7,0.01) : si.smoo;
drive = hslider("drive[acc: 1 3 -10 0 10]", 0,0,1,0.01) : si.smoo;
ratio =
(49/32) * (keyID==0) +
(147/128) * (keyID==1) +
(441/256) * (keyID==2) +
(1323/1024) * (keyID==3) +
//keyID 4 is blank
(7/4) * (keyID==5) +
(21/16) * (keyID==6) +
(63/32) * (keyID==7) +
(189/128) * (keyID==8) +
(567/517) * (keyID==9) +
(1/1) * (keyID==10) +
(3/2) * (keyID==11) +
(9/8) * (keyID==12);
pan = os.osc(ratio*timeref/3.5)*0.3+0.5;
process = os.sawtooth(pitchref*ratio) * (os.osc(timeref*ratio)+1)/2*gate
:ef.cubicnl(drive,0) : fi.resonlp((12000)*(os.osc(ratio/7)+4)/5,7,1) <: _*pan,_*(1-pan) : dm.zita_rev1;
import("stdfaust.lib");
g = hslider("gain",0.5,0,1,0.01) : si.smoo;
f = hslider("freq",440,50,5000,0.01);
b = hslider("bend[midi:pitchwheel]",1,0,10,0.01) : si.polySmooth(t,0.999,1);
t = button("gate");
freq = f*b;
envelope = en.adsr(0.1,0.1,80,0.1,t)*g;
process = os.sawtooth(freq) :> _*envelope <: _,_;
sudo mv /opt/X11/lib/libXt.6.dylib{,.bak} & cp /opt/X11/lib{/flat_namespace,}/libXt.6.dylib
)A simple trumpet instrument based on a sawtooth wave oscillator and a Butterworth lowpass filter.
declare interface "SmartKeyboard{
'Number of Keyboards':'5',
'Max Keyboard Polyphony':'1',
'Inter-Keyboard Slide':'1',
'Mono Mode':'1',
'Keyboard 0 - Number of Keys':'13',
'Keyboard 1 - Number of Keys':'13',
'Keyboard 2 - Number of Keys':'13',
'Keyboard 3 - Number of Keys':'13',
'Keyboard 4 - Number of Keys':'13',
'Keyboard 0 - Lowest Key':'77',
'Keyboard 1 - Lowest Key':'72',
'Keyboard 2 - Lowest Key':'67',
'Keyboard 3 - Lowest Key':'62',
'Keyboard 4 - Lowest Key':'57',
'Rounding Mode':'2'
}";
import("stdfaust.lib");
g = hslider("gain",0.5,0,1,0.01) : si.smoo;
f = hslider("freq",440,50,5000,0.01);
b = hslider("bend[midi:pitchwheel]",1,0,10,0.01) : si.polySmooth(t,0.999,1);
y = hslider("y",0.5,0,1,0.01) : si.smoo;
t = button("gate");
freq = f*b;
envelope = en.adsr(0.1,0.1,80,0.1,t)*g;
cutoff = 200 + y*5000;
process = os.sawtooth(freq) : fi.lowpass(3,cutoff) :> _*envelope <: _,_;
zero = hslider("zero",0,-1,1,0.01) : si.smoo;
del = hslider("del",1,1,50,1);
// lowpass/highpass
myLP(b1,N) = _ <: _@N*b1,_ :> _;
process = no.noise : myLP(zero,del);
import("stdfaust.lib");
feedback = hslider("feedback",0.99,0,1,0.01) : si.smoo;
f = hslider("freq",440,50,1000,0.01) : si.smoo;
gain = hslider("gain",1,0,1,0.01);
gate = button("gate");
// Karplus-Strong
string(freq,fdbk) = +~(de.fdelay4(1024,del)*fdbk : reflexionFilter)
with{
del = ma.SR/freq;
reflexionFilter = _ <: (_'+_)/2;
};
process = gate : ba.impulsify*gain : string(f,feedback) <: _,_;
If you're a Mac user, make sure that XCode is properly installed on your system (using the app store, not some shady source ;) ). Even if you don't plan to develop apps for iOS, installing XCode will take care of setting up the OSX dev tools that are are needed to use Faust on your computer. If you plan to develop for iOS, more specific information about your system configuration/version (some system update might be in order...) is provided in the section on Setting Up Your System for Mobile Dev.
No specific configuration is required for all other platforms (Linux and Windows).
Make sure that you have a descent text editor installed on your system (e.g. not TextEdit). If you're not sure, we recommend Atom which has a package for Faust programming.
If you're on MacOS or Linux, you should install the Faust compiler on your system. Since we might want to add features or fix some bugs ;) to the version of Faust used during the workshop, you should install it from our git repository. To do so, open a terminal window and type the following commands consecutively (press the return key for each new line):
mkdir ms1
cd ms1
git clone https://github.com/grame-cncm/faust.git
cd faust
make
sudo make install
The first command creates an ms1
folder in your home directory, the next one changes the
current location of your terminal to ms1
. git clone
copies the source code of
Faust in the current folder (now ms1
), make
(this might take a while) builds
the Faust compiler and sudo make install
installs Faust on your system (you'll have to
enter your password for that). If you have any troubles with those steps (we know that some of you
might never have used a terminal before), shoot me an e-mail.
To make sure that Faust was properly installed, just type:
faust
and you should get an error message in return saying something like:
Error no input file
Keep the faust
folder that was created when cloning the repository since we might need
it later during the workshop.
If you use Windows, you should get the Faust source directly from github: https://github.com/grame-cncm/faust. No need to compile the Faust compiler as on the Mac and Linux as you'll do everything in FaustLive (see following section).
Get FaustLive and install it on your system. Don't forget to read the installation instructions contained in the package: this is very important!
Follow the instructions on this page. For Android, you can stop before the paragraph starting with "In order to be able to compile Android apps [...]". For iOS, don't forget to create a free developer account on the Apple Website.
Windows Users: even though the instructions
say that faust2smartkeyb
is not working on Windows (which is only partly true), you should still
install Android studio on your system following the same instructions as for Linux and OSX (also stop before
the paragraph starting with "In order to be able to compile Android apps [...]").
Come turn your cell-phone and/or tablet into a completely standalone musical instrument for live performance and learn about the following topics:
This workshop is part of the Mobile Synth workshop series and should be taken in preparation for the Mobile Synth: 3D Printing and Musical Interface Design for Smart-Phone workshop. In conjunction, these workshops allow participants to both write software and make hardware interventions on their mobile device, transforming it into a custom instruments.
This workshop is intended for musicians, makers, engineers, computer scientists, etc. It doesn't require any previous background in computer programming or signal processing and is open to everyone. We expect participants to bring their own laptop and mobile device (smartphone and/or tablet). The following devices will be supported during the workshop:
In this workshop, participants will “augment” their smartphone to turn it into a musical instrument for live performance. Augmentations can be both passive (e.g., 3D printed custom case, passive amplifier, handles, etc.) and active (e.g., sensors, custom interface, etc.). The following topics will be covered:
This workshop is part of the Mobile Synth workshop series and the Mobile Synth I: Mobile App Development for Sound Synthesis and Processing in Faust workshop must be taken in preparation for it. In c onjunction, these workshops allow participants to both write software and make hardware interventions on their mobile device, transforming it into a custom instrument.
Mobile Synth I must be taken in preparation for this workshop.
The registration fee of this workshop includes a sensor kit (comprising a Teensy Microncontroller and a series of sensors) and covers various expenses related to digital fabrication (3D printing, laser cutting, etc.).
Music + Video
Romain Michon & Ge Wang
Special Thanks
Orchi Das, Tim O'Brien, Yuval Adler, Victoria Grace, and CCRMAlites present during video shoot!
Lyrics
Yo, I'm Romain, vive la France,
I love baguettes and my Aix en Provence!
I program in Faust like Marcel Proust,
I'm the illest monkey-fighting coder in the house!
Yo! listen up, we got this workshop
this summer at CCRMA, gonna rock nonstop
(WHOA!!) that's right, folks, use your smartphones
turn them into flutes, drums, and saxophones!
With microphones, iPhones, and overtones,
enrich your mind like it's on the Dow Jones!
We gonna synthesize sound with DSP,
into crazy instruments, that's just the first week!
design interfaces for iOS / Android,
[REFRAIN]
[Pipophone solo]
Week 2. It's gonna get real.
Granzow's gonna hit you with a steering wheel (of Knowledge!!!)
with lasers, sensors, and 3D printers,
turn all YALL from beginners into winners!
"oh dayum! romain what you call that?"
hmm, I don't know, Orchi, i just made that!
but it's round, and makes sound,
and it's sonically profound,
it's dope, made with code,
ima go with the flow!
[REFRAIN]
Hells yeah monkeyfightas,
register the website
or with my home girl Nette!
Source
All sounds in this video (except for the drum kit) were synthesized using smartphone-based
instruments made with Faust or
FaucK. The source code of some of
these instruments can be downloaded here if you want to try to play this song at home :).