Emerging Technologies for Musical Audio Synthesis and Effects
Julius Smith, with Romain Michon
CCRMA, Stanford University
Linux Audio Conference 2015 - Mainz, Germany - April 9, 2015
Introduction
- Controllers
- Embedded Systems
- Mobile Platforms
- Webaudio
- 3D Printing
- Faust
- Trendy Research Topics
Controllers
Microcontroller Boards
What are they?
Why are they so successful?
The Arduino IDE
Are there alternatives to Arduinos?
Texas Instrument LaunchPad
The Wiring S Board
Smaller and cheaper...
DigiStump's DigiSpark
More connected...
Spark's Core and Electron
More wearable...
Arduino Lilypad
(Jacek Spiewla’s BeatGlove) and Adafruit Flora
More powerful...
Arduino Yun and
Intel Edison
What's next?
Embedded Systems
Embedded Linux Boards
Pros
- Extremely easy to use
- Runs Linux
- Power of the community
- Price
- Simplicity
- Versatility
- Plurality
Cons
- Plurality
- It is what it is...
- Limited power
- Booting time
- Compatibility with external hardware
- Sometime victim of their own success
Embedded programming environments
Mobile Platforms
The MoForte Guitar
- Full electric guitar + effects (implemented in Faust):
6 strings, general excitations, distortion,
compression, phaser, 5 bands parametric equalizer, feedback, wah pedal and autowah,
flanger, reverb
- Responds to accelerometers, gyros, touches (pluck), swipes (strumming), etc.
- Historically related to the Stanford Sondius Project, ca. 1995
A Word About Software Licenses
- moForte
guitar is
a commercial
offering using
FOSS
- Liberal
licensing such as in
FAUST makes this
possible
- GPL
motivates complete
rewrites from
scratch
- MIT
license is one of
the best (“just run
with it”), as are BSD, STK
- moForte
presently consists
of two guys (Pat and
Nick) plus JOS
(consulting), and commercial is their only option right now
- We
can contribute
freely when we
can, but otherwise,
commercial
products based on FOSS
should be ok
-
For general purpose
tools, consider the
MIT license
extended to
request that
bugfixes be
submitted
upstream
-
GPL makes sense for finished apps, but unrestricted libraries ideal
-
"Free for free use" + reasonable commercial license fee also works
Supporting and Organizing FOSS Development
- We can create better systems for prioritizing, addressing, and funding FOSS development
- Kickstarter model is good, but need something lighter weight for smaller software components
- Google Summer of Code (SOC) is great, but also large-grained and hard to get into
FOSS Small-Task Example
On the Faust list, a company wanted
the optimized filter-bank cases in
filter.lib to be extended to higher order
- Perfect example of a bid/hire item
- Company receives a
lower rate if
contributing the
improvement for all
(e.g., 50%)
- Developer sees the
same compensation,
making it possible to do
the work
- A general FOSS fund can support the discounted rate
iOS vs. Android: the sad truth
Device |
Touch-to-Sound |
Round-Trip |
iPhone 5 |
36ms |
13ms |
iPhone 6 |
30ms |
11ms |
iPad 2 |
45ms |
15ms |
iPad Air 2 |
38ms |
12ms |
Samsung Galaxy S5 |
72ms |
78ms |
Google Nexus 5 |
90ms |
92ms |
Google Nexus 7 |
130ms |
130ms |
Measured Latency on Different Mobile Devices at 44.1KHz (Buffer Length 512 samples)
Webaudio
What is good? What is working?
- Cross-platform
- Great for collaboration
- Open source in essence
- Wide variety of pre-made objects
- Simple to use
- Precise scheduling
What will be working?
(may be)
- MIDI support
- Audio workers
- Better performances
- More than stereo output
- Compatibility of all the web browsers with Jack?
A more global vision of the use of the webaudio API in the open source world:
The Faust PlayGround
Faust
Faust Architectures (1)
faust2alqt |
faust2alsa |
faust2jackserver |
faust2alsaconsole |
faust2andoid |
faust2jaqt |
faust2api |
faust2asm |
faust2javaswing |
faust2asmjs |
faust2au |
faust2ladspa |
faust2caqt |
faust2ck |
faust2lv2 |
faust2csound |
faust2dssi |
faust2mathdoc |
faust2graph |
faust2ios |
faust2max6 |
faust2jack |
faust2jackconsole |
faust2netjackconsole |
faust2netjackqt |
faust2octave |
faust2owl |
Faust Architectures (2)
faust2paqt |
faust2plot |
faust2puredata |
faust2ros |
faust2rosgtk |
faust2rpialsoconsole |
faust2rpinetjackconsole |
faust2sc |
faust2svg |
faust2vst |
faust2webaudio |
Getting Started with Faust
Research
Nonlinear Wave Digital Filters
Kurt Werner and Julius Smith at CCRMA
based on recent work at Aalto University and elsewhere
Scattering Delay Networks (SDN)
Enzo De Sena,
H. Hacihabiboglu, Z. Cvetković, & J. Smith
- Wall-to-wall reflections modeled by digital waveguides
- Extra waveguides match direct signal and early reflections
Conclusions
Thanks for your attention! :)