The Mobile Synth Workshop Series took place at CCRMA (Stanford University) between June 19-30, 2017. They allowed participants to both write software and make hardware interventions on their mobile device, transforming it into a custom musical instruments. This page presents some of the workshops' material as well as the final projects done by the students who took them.

Mobile Synth II: 3D Printing and Musical Interface Design for Smartphone Augmentation

Final Projects

Marit Brademann - Mobile Hang

Click on the picture to download the sources.

Edmond Howser - Something Else

Click on the picture to download the sources. GitHub page: https://github.com/edmondhowser/somethingelse.

Casey Kim

Click on the picture to download the sources.

Day 1

Resources

Day 2

Controlling a SmartKeyboard App With a Teensy: Writing a Duet

This example code shows how to control a SmartKeyboard app using a Teensy board by sending MIDI messages. The app is based on a sawtooth and a triangle oscillators that are switched on and off by the Teensy depending on the needs of the performance.

Teensy Code

"MIDI" USB type has to be chosen when uploading the code to the Teensy!!

#include <Bounce.h>

int ccFreqSaw = 10; // MIDI CC for the pitch of the sawtooth
int ccFreqTri = 11; // MIDI CC for the pitch of the triangle
int ccSwitch = 12; // MIDI CC to switch between the saw and the tri
int midiChannel = 0;
int sawNotes[] = {64,65,67}; // MIDI notes for the saw
int triNotes[] = {80,78,77}; // MIDI notes for the tri

void setup() {
}

void loop() {
  // Switching between tri, saw, or both
  int instrSwitch = random(3);
  usbMIDI.sendControlChange(ccSwitch,instrSwitch,midiChannel);
  for(int i=0; i<3; i++){
    // Reading and sending notes
    if(instrSwitch == 0 || instrSwitch == 2){ 
      usbMIDI.sendControlChange(ccFreqSaw,sawNotes[i],midiChannel);
    }
    if(instrSwitch == 1 || instrSwitch == 2){
      usbMIDI.sendControlChange(ccFreqTri,triNotes[i],midiChannel);
    }
    delay(500); // wait for 500 ms
  }
}

Faust Code

declare interface "SmartKeyboard{
	'Number of Keyboards':'1',
	'Max Keyboard Polyphony':'0',
	'Keyboard 0 - Number of Keys':'1',
	'Keyboard 0 - Send Freq':'0',
	'Keyboard 0 - Piano Keyboard':'0'
}";

import("stdfaust.lib");

freq0 = hslider("freq0[midi:ctrl 10]",50,0,127,1) : ba.midikey2hz : si.smoo;
freq1 = hslider("freq1[midi:ctrl 11]",50,0,127,1) : ba.midikey2hz : si.smoo;
switch = hslider("switch[midi:ctrl 12]",0,0,127,1) : int;

process = os.sawtooth(freq0)*((switch == 0) | (switch == 2)) +
os.triangle(freq1)*((switch == 1) | (switch == 2)) : *(0.5) <: _,_;

Resources

Day 3

Controlling a Faust SmartKeyboard App With Sensors

This example shows how to control the frequency of a sawtooth wave oscillator with an external sensor (an FSR for instance) plugged to one of the analog inputs of the Teensy.

Teensy Code

#include 

bool debug = false;

void setup() {
  if(debug){ 
    Serial.begin(9600); // initializing serial port
  }
}

void loop() {
  int sensorValue = analogRead(A0); // retrieving sensor value on Analog pin 0
  if(debug){ 
    Serial.println(sensorValue); // printing sensor value
  }
  else{ 
    int midiCC = 10; // must match the faust configuration
    int midiValue = sensorValue*127/1024; // value between 0-127
    int midiChannel = 0; // actually doesn't matter
    usbMIDI.sendControlChange(midiCC,midiValue,midiChannel); // sending control message
  }
  delay(30); // wait for 30ms
}

Faust Code

import("stdfaust.lib");

declare interface "SmartKeyboard{
	'Number of Keyboards':'1',
	'Max Keyboard Polyphony':'0',
	'Keyboard 0 - Number of Keys':'1',
	'Keyboard 0 - Send Freq':'0',
	'Keyboard 0 - Piano Keyboard':'0'
}";

freq = hslider("frequency[midi:ctrl 10]",100,100,2000,1) : si.smoo;

process = os.sawtooth(freq) <: _,_;

Mobile Synth I: Mobile App Development for Sound Synthesis and Processing in Faust

Final Projects

Edmond Howser - Equilibrium

This code is based on the ideas behind Equilibrium by Jordan Nobles. The original score can be found on www.jordannobles.com collection of Open Score Works (1997-2016) Jordan Nobles.

The interface is a set of five keyboard with 7 buttons. The first button intentionally is silent. Each row represents a set of fifths.

Instructions (based upon original score)

Play freely in your own time. Only ever play the given notes, in a random order. Players can pause at any time to breathe and to listen before proceeding all notes should be soft and quiet throughout the piece. The duration of the piece is variable but should probably be decided on ahead of time there are no parts or formal score... Any musicians play freely from available notes. There is no pulse, tempo, measures or exact synchronization.

// Edmond Howser
// CCRMA Mobile Synth Workshop 6/2017

import("stdfaust.lib");

declare interface "SmartKeyboard{
	'Number of Keyboards':'5',
	'Max Keyboard Polyphony':'1',
	'Keyboard 0 - Number of Keys':'7',
	'Keyboard 1 - Number of Keys':'7',
	'Keyboard 2 - Number of Keys':'7',
	'Keyboard 3 - Number of Keys':'7',
	'Keyboard 4 - Number of Keys':'7',
	'Keyboard 0 - Piano Keyboard':'0',
	'Keyboard 1 - Piano Keyboard':'0',
	'Keyboard 2 - Piano Keyboard':'0',
	'Keyboard 3 - Piano Keyboard':'0',
	'Keyboard 4 - Piano Keyboard':'0',
}";

g = hslider("g[acc: 0 1 -10 0 10]",0.5,0,1.0,0.01) : si.smoo;

key = hslider("key",0,0,6,1);
keyboard = hslider("keyboard",0,0,4,1);
t = button("gate");

// Pitches & Frequencies
XX = 0;
C1 = 32.70;
G1 = 49.00;
C2 = 65.41;
D2 = 73.42;
E2 = 82.41;
F2 = 87.31;
G2 = 98.00;
A2 = 110.00;
B2 = 123.47;
C3 = 130.81;
D3 = 146.83;
E3 = 164.81;
F3s = 185.00;
G3 = 196.00;
A3 = 220.00;
B3 = 246.94;
D4 = 293.66;
E4 = 329.63;
F4s = 368.99;
B4 = 493.88;
C4s = 277.18;
G4s = 415.30;
A4 = 440.00;
C5s = 554.37;
D5s  = 622.25;
E5 = 659.25;
F5s = 739.99;
G5s = 830.61;
D6s = 1244.51;
A6s = 1864.66;
E7s = 2783.83;
B7s = 4186.01;

// Frequencies for Keyboard 0
freq0 = (keyboard == 0)*(key==0)*XX  + (keyboard == 0)*(key==1)*G2  +
	(keyboard == 0)*(key==2)*D3  + (keyboard == 0)*(key==3)*A3  +
	(keyboard == 0)*(key==4)*E4  + (keyboard == 0)*(key==5)*B4  +
	(keyboard == 0)*(key==6)*F5s;
// Frequencies for Keyboard 1
freq1 = (keyboard == 1)*(key==0)*XX + (keyboard == 1)*(key==1)*F2 +
	(keyboard == 1)*(key==2)*C3 + (keyboard == 1)*(key==3)*G3 +
	(keyboard == 1)*(key==4)*D4 + (keyboard == 1)*(key==5)*A4 +
	(keyboard == 1)*(key==6)*E5;
// Frequencies for Keyboard 2
freq2 = (keyboard == 2)*(key==0)*XX  + (keyboard == 2)*(key==1)*E2  +
	(keyboard == 2)*(key==2)*B2  + (keyboard == 2)*(key==3)*F3s +
	(keyboard == 2)*(key==4)*C4s + (keyboard == 2)*(key==5)*G4s +
	(keyboard == 2)*(key==6)*D5s;
// Frequencies for Keyboard 3
freq3 = (keyboard == 3)*(key==0)*XX  + (keyboard == 3)*(key==1)*D2  +
	(keyboard == 3)*(key==2)*A2  + (keyboard == 3)*(key==3)*E3  +
	(keyboard == 3)*(key==4)*B3  + (keyboard == 3)*(key==5)*F4s +
	(keyboard == 3)*(key==6)*C5s;
// Frequencies for Keyboard 4
freq4 = (keyboard == 4)*(key==0)*XX  + (keyboard == 4)*(key==1)*C2  +
	(keyboard == 4)*(key==2)*G2  + (keyboard == 4)*(key==3)*D3  +
	(keyboard == 4)*(key==4)*A3  + (keyboard == 4)*(key==5)*E4  +
	(keyboard == 4)*(key==6)*B4;

signal0 = os.osc(freq0);
signal1 = os.osc(freq1);
signal2 = os.osc(freq2);
signal3 = os.osc(freq3);
signal4 = os.osc(freq4);

envelope0 = en.adsr(5,2.5,80,5,t) * g;
envelope1 = en.adsr(5,2.5,80,5,t) * g;
envelope2 = en.adsr(5,2.5,80,5,t) * g;
envelope3 = en.adsr(5,2.5,80,5,t) * g;
envelope4 = en.adsr(5,2.5,80,5,t) * g;

synth = signal0*envelope0*(keyboard == 0) +
	signal1*envelope1*(keyboard == 1) +
	signal2*envelope2*(keyboard == 2) +
	signal3*envelope3*(keyboard == 3) +
	signal4*envelope4*(keyboard == 4);

process = synth  <: _,_;

Casey Kim - Phlute

// Casey Kim - "Phlute"
// CCRMA Mobile Synth Workshop 6/2017

declare interface "SmartKeyboard{
	'Number of Keyboards':'2',
	'Max Keyboard Polyphony':'0',
  'Inter-Keyboard Slide':'1',
	'Mono Mode':'1',
	'Keyboard 0 - Number of Keys':'13',
  'Keyboard 1 - Number of Keys':'13',
  'Keyboard 0 - Lowest Key':'72',
  'Keyboard 1 - Lowest Key':'60',
  'Rounding Mode':'0'
}";

import("stdfaust.lib");

freq = hslider("freq",440,50,5000,0.01);
ctfreq = hslider("ctfreq[acc: 0 0 -10 0 10]",3000,300,5000,0.01) : si.smoo;
vib = hslider("vib[acc: 1 0 -10 0 10]", 0,0,0.02,0.01);

process = an.amp_follower(0.17)*os.triangle(freq + os.osc(5)*vib*freq) : fi.resonlp(ctfreq,7,1)
<: dm.zita_rev1;

Marit Brademann - Doppler Loop

// Marit Brademann
// CCRMA Mobile Synth Workshop 6/2017

declare interface "SmartKeyboard{
	'Number of Keyboards':'1',
	'Max Keyboard Polyphony':'0',
	'Keyboard 0 - Number of Keys':'2',
	'Keyboard 0 - Send Freq':'0',
	'Keyboard 0 - Piano Keyboard':'0'
}";

import("stdfaust.lib");

key = hslider("key",0,0,1,1) : int;
record = key;
frequency = hslider("frequency[acc: 0 0 -10 0 10]",1000,10,2000,0.01) : si.smoo;
del = hslider("del[acc: 1 0 -10 0 10]",1023,0,2047,0.01) : si.smoo;

cnt0 = +(1)~*(record) : -(1) : %(192000);
cnt1 = +(1)~*(1-record): -(1) : %(192000);

process = rwtable(192000, 0.0, cnt0, _, cnt1)*(1-record : si.smoo) :_*(os.osc(frequency)*0.5 + 0.5) : de.fdelay4(2048,del);

Matthew Mitchell - A Well Tuned Rhythmicon

Based on Henry Cowell's and Léon Theremin's Rhythmicon and on a tuning system by la monte young, as described by Kyle Gann.

// Matt Mitchell
// CCRMA Mobile Synth Workshop 6/2017

declare interface "SmartKeyboard{
	'Number of Keyboards':'3',
	'Max Keyboard Polyphony':'10',
	'Keyboard 0 - Number of Keys':'5',
	'Keyboard 1 - Number of Keys':'5',
	'Keyboard 2 - Number of Keys':'5',
	'Keyboard 3 - Number of Keys':'5',
	'Keyboard 4 - Number of Keys':'5',
	'Keyboard 0 - Piano Keyboard' : '0',
	'Keyboard 1 - Piano Keyboard' : '0',
	'Keyboard 2 - Piano Keyboard' : '0',
	'Keyboard 3 - Piano Keyboard' : '0',
	'Keyboard 4 - Piano Keyboard' : '0',
	'Keyboard 0 - Key 0 - Label':'49/32',
	'Keyboard 0 - Key 1 - Label':'147/128',
	'Keyboard 0 - Key 2 - Label':'441/256',
	'Keyboard 0 - Key 3 - Label':'1323/1024',
	'Keyboard 1 - Key 0 - Label':'7/4',
	'Keyboard 1 - Key 1 - Label':'21/16',
	'Keyboard 1 - Key 2 - Label':'63/32',
	'Keyboard 1 - Key 3 - Label':'189/128',
	'Keyboard 1 - Key 4 - Label':'567/512',
	'Keyboard 2 - Key 0 - Label':'1/1',
	'Keyboard 2 - Key 1 - Label':'3/2',
	'Keyboard 2 - Key 2 - Label':'9/8',
	'Keyboard 0 - Send X':'0',
	'Keyboard 1 - Send X':'0',
	'Keyboard 2 - Send X':'0',
	'Keyboard 0 - Send Y':'0',
	'Keyboard 1 - Send Y':'0',
	'Keyboard 2 - Send Y':'0',
	'Rounding Mode':'0'
}";

import("stdfaust.lib");

keyboard = hslider("keyboard",0,0,2,1) : int;
key = hslider("key",0,0,4,1) : int;
keyID = (keyboard*5) + key;

gate = button("gate") : si.smoo;

pitchref = 220;
timeref = hslider("timeref[acc: 0 0 -10 0 10]", 2,0.1,7,0.01) : si.smoo;
drive = hslider("drive[acc: 1 3 -10 0 10]", 0,0,1,0.01) : si.smoo;

ratio =
	(49/32) * (keyID==0) +
	(147/128) * (keyID==1) +
	(441/256) * (keyID==2) +
	(1323/1024) * (keyID==3) +
	//keyID 4 is blank
	(7/4) * (keyID==5) +
	(21/16) * (keyID==6) +
	(63/32) * (keyID==7) +
	(189/128) * (keyID==8) +
	(567/517) * (keyID==9) +
	(1/1) * (keyID==10) +
	(3/2) * (keyID==11) +
	(9/8) * (keyID==12);


pan = os.osc(ratio*timeref/3.5)*0.3+0.5;


process = os.sawtooth(pitchref*ratio) * (os.osc(timeref*ratio)+1)/2*gate
	:ef.cubicnl(drive,0) : fi.resonlp((12000)*(os.osc(ratio/7)+4)/5,7,1) <: _*pan,_*(1-pan) : dm.zita_rev1;

Day 1

MIDI Controllable Synthesizer

import("stdfaust.lib");
g = hslider("gain",0.5,0,1,0.01) : si.smoo;
f = hslider("freq",440,50,5000,0.01);
b = hslider("bend[midi:pitchwheel]",1,0,10,0.01) : si.polySmooth(t,0.999,1);
t = button("gate");

freq = f*b;
envelope = en.adsr(0.1,0.1,80,0.1,t)*g;

process = os.sawtooth(freq) :> _*envelope <: _,_;

Resources

Day 2

Simple Trumpet App

A simple trumpet instrument based on a sawtooth wave oscillator and a Butterworth lowpass filter.

declare interface "SmartKeyboard{
	'Number of Keyboards':'5',
	'Max Keyboard Polyphony':'1',
	'Inter-Keyboard Slide':'1',
	'Mono Mode':'1',
	'Keyboard 0 - Number of Keys':'13',
	'Keyboard 1 - Number of Keys':'13',
	'Keyboard 2 - Number of Keys':'13',
	'Keyboard 3 - Number of Keys':'13',
	'Keyboard 4 - Number of Keys':'13',
	'Keyboard 0 - Lowest Key':'77',
	'Keyboard 1 - Lowest Key':'72',
	'Keyboard 2 - Lowest Key':'67',
	'Keyboard 3 - Lowest Key':'62',
	'Keyboard 4 - Lowest Key':'57',
	'Rounding Mode':'2'
}";

import("stdfaust.lib");

g = hslider("gain",0.5,0,1,0.01) : si.smoo;
f = hslider("freq",440,50,5000,0.01);
b = hslider("bend[midi:pitchwheel]",1,0,10,0.01) : si.polySmooth(t,0.999,1);
y = hslider("y",0.5,0,1,0.01) : si.smoo;
t = button("gate");

freq = f*b;
envelope = en.adsr(0.1,0.1,80,0.1,t)*g;
cutoff = 200 + y*5000;

process = os.sawtooth(freq) : fi.lowpass(3,cutoff) :> _*envelope <: _,_;

Simplest Lowpass Filter

zero = hslider("zero",0,-1,1,0.01) : si.smoo;
del = hslider("del",1,1,50,1);

// lowpass/highpass
myLP(b1,N) = _ <: _@N*b1,_ :> _;

process = no.noise : myLP(zero,del);

String Physical Model: Karplus-Strong

import("stdfaust.lib");

feedback = hslider("feedback",0.99,0,1,0.01) : si.smoo;
f = hslider("freq",440,50,1000,0.01) : si.smoo;
gain = hslider("gain",1,0,1,0.01);
gate = button("gate");

// Karplus-Strong
string(freq,fdbk) = +~(de.fdelay4(1024,del)*fdbk : reflexionFilter)
with{
  del = ma.SR/freq;
  reflexionFilter = _ <: (_'+_)/2;
};

process = gate : ba.impulsify*gain : string(f,feedback) <: _,_;

Resources

Day 3

Getting Ready for the Workshop

General Requirements

If you're a Mac user, make sure that XCode is properly installed on your system (using the app store, not some shady source ;) ). Even if you don't plan to develop apps for iOS, installing XCode will take care of setting up the OSX dev tools that are are needed to use Faust on your computer. If you plan to develop for iOS, more specific information about your system configuration/version (some system update might be in order...) is provided in the section on Setting Up Your System for Mobile Dev.

No specific configuration is required for all other platforms (Linux and Windows).

Make sure that you have a descent text editor installed on your system (e.g. not TextEdit). If you're not sure, we recommend Atom which has a package for Faust programming.

Installing Faust

If you're on MacOS or Linux, you should install the Faust compiler on your system. Since we might want to add features or fix some bugs ;) to the version of Faust used during the workshop, you should install it from our git repository. To do so, open a terminal window and type the following commands consecutively (press the return key for each new line):

mkdir ms1
cd ms1
git clone https://github.com/grame-cncm/faust.git
cd faust
make
sudo make install

The first command creates an ms1 folder in your home directory, the next one changes the current location of your terminal to ms1. git clone copies the source code of Faust in the current folder (now ms1), make (this might take a while) builds the Faust compiler and sudo make install installs Faust on your system (you'll have to enter your password for that). If you have any troubles with those steps (we know that some of you might never have used a terminal before), shoot me an e-mail.

To make sure that Faust was properly installed, just type:

faust

and you should get an error message in return saying something like:

Error no input file

Keep the faust folder that was created when cloning the repository since we might need it later during the workshop.

If you use Windows, you should get the Faust source directly from github: https://github.com/grame-cncm/faust. No need to compile the Faust compiler as on the Mac and Linux as you'll do everything in FaustLive (see following section).

Installing FaustLive

Get FaustLive and install it on your system. Don't forget to read the installation instructions contained in the package: this is very important!

Setting Up Your System for Mobile Dev

Follow the instructions on this page. For Android, you can stop before the paragraph starting with "In order to be able to compile Android apps [...]". For iOS, don't forget to create a free developer account on the Apple Website.

Windows Users: even though the instructions say that faust2smartkeyb is not working on Windows (which is only partly true), you should still install Android studio on your system following the same instructions as for Linux and OSX (also stop before the paragraph starting with "In order to be able to compile Android apps [...]").


Pre-Workshop Information/Adds


More information about this video

Information and Sign Up!

Mobile Synth I: Mobile App Development for Sound Synthesis and Processing in Faust

Come turn your cell-phone and/or tablet into a completely standalone musical instrument for live performance and learn about the following topics:

This workshop is part of the Mobile Synth workshop series and should be taken in preparation for the Mobile Synth: 3D Printing and Musical Interface Design for Smart-Phone workshop. In conjunction, these workshops allow participants to both write software and make hardware interventions on their mobile device, transforming it into a custom instruments.

This workshop is intended for musicians, makers, engineers, computer scientists, etc. It doesn't require any previous background in computer programming or signal processing and is open to everyone. We expect participants to bring their own laptop and mobile device (smartphone and/or tablet). The following devices will be supported during the workshop:

Register Here

Mobile Synth II: 3D Printing and Musical Interface Design for Smartphone Augmentation

In this workshop, participants will “augment” their smartphone to turn it into a musical instrument for live performance. Augmentations can be both passive (e.g., 3D printed custom case, passive amplifier, handles, etc.) and active (e.g., sensors, custom interface, etc.). The following topics will be covered:

This workshop is part of the Mobile Synth workshop series and the Mobile Synth I: Mobile App Development for Sound Synthesis and Processing in Faust workshop must be taken in preparation for it. In c onjunction, these workshops allow participants to both write software and make hardware interventions on their mobile device, transforming it into a custom instrument.

Mobile Synth I must be taken in preparation for this workshop.

The registration fee of this workshop includes a sensor kit (comprising a Teensy Microncontroller and a series of sensors) and covers various expenses related to digital fabrication (3D printing, laser cutting, etc.).

Register Here

Teaser Video Information

Music + Video
Romain Michon & Ge Wang

Special Thanks
Orchi Das, Tim O'Brien, Yuval Adler, Victoria Grace, and CCRMAlites present during video shoot!

Lyrics
Yo, I'm Romain, vive la France,
I love baguettes and my Aix en Provence!
I program in Faust like Marcel Proust,
I'm the illest monkey-fighting coder in the house!

Yo! listen up, we got this workshop
this summer at CCRMA, gonna rock nonstop
(WHOA!!) that's right, folks, use your smartphones
turn them into flutes, drums, and saxophones!
With microphones, iPhones, and overtones,
enrich your mind like it's on the Dow Jones!
We gonna synthesize sound with DSP,
into crazy instruments, that's just the first week!
design interfaces for iOS / Android,

[REFRAIN]
[Pipophone solo]

Week 2. It's gonna get real.
Granzow's gonna hit you with a steering wheel (of Knowledge!!!)
with lasers, sensors, and 3D printers,
turn all YALL from beginners into winners!
"oh dayum! romain what you call that?"
hmm, I don't know, Orchi, i just made that!
but it's round, and makes sound,
and it's sonically profound,
it's dope, made with code,
ima go with the flow!

[REFRAIN]

Hells yeah monkeyfightas,
register the website
or with my home girl Nette!

Source
All sounds in this video (except for the drum kit) were synthesized using smartphone-based instruments made with Faust or FaucK. The source code of some of these instruments can be downloaded here if you want to try to play this song at home :).