Music 220b: Winter 2001
Fernando Lopez-Lezcano, instructor
Christopher Burns, teaching assistant
Tamara Smyth, teaching assistant
Week 5: composing with large datasets
The issue of "data explosion" has come up a number of times. Computer music often requires huge amounts of data, especially when a composer uses parameter-intensive synthesis techniques like additive or granular synthesis. Algorithmic generation of parameters is one way of dealing with this issue, but there are others. For instance, there's plenty of data out in the world -- everything from U.S. Census counts to weather statistics to the yellow pages. Why not use it?
Well, it's a bit more complicated than that. There are at least two essential questions: what are the trends in the data that can be exploited to musical effect? And how do we map the data to our parameters, to get that effect? Or, to put it another way, how do we choose the data, and then how do we use it?
Trend analysis of a dataset can be incredibly complex, but we don't have to be statisticians to make it possible. For a simple example, think of the Dow Jones average -- on any given day it may rise or fall, but considered over the last seventy years it's risen substantially. If we wanted some kind of trend in our music which was highly variable on the local scale, and generally increasing on the large scale, we could choose a worse dataset. For a real-world example, consider Charles Dodge's Earth's Magnetic Field -- a composition based on, you guessed it.
For a case study, we'll consider soundfiles as datasets... after all, we've got a few of those around. And what is a soundfile but a string of numbers? (Well, those numbers have some interesting properties when sonified....) In particular we'll use speech soundfiles -- which you could (reductively) think of as a noisy oscillation between -1 and 1, with occasional stretches of near-zero between the words.
data applied at the sample level: envelope following
The "follower" instrument (contained in follower.ins) applies soundfile data at the sample level. There are a variety of ways we could do this -- switch between synthesis techniques when the amplitude crosses a certain threshold a certain number of times, try to extract and track the frequency of the soundfile (perhaps by counting zero-crossings), etc. etc. In this instance we're trying to capture the amplitude curve of the soundfile, and then apply that curve to a (totally different) soundfile.
The trick is to use a "decaying threshold": every time the amplitude of our analyzed soundfile jumps over a certain threshold, we reset the threshold to that peak (and raise the amplitude of the target soundfile to the same level). Then we let that threshold slowly drop -- eventually another peak will get past, and the threshold will jump up again. It's not a sophisticated way of catching transients or amplitude curves in general -- we'd do better to trace the power of the signal than the peak amplitude -- but it works alright....
(definstrument follower (source-filename target-filename
&key (source-srate 1.0)
(target-srate 1.0)
(start 0.0)
(duration nil)
(decay 0.00001))
(let* ((source-file (open-input* source-filename))
(source-file-duration (/ (sound-duration source-filename)
(abs source-srate)))
(target-file (open-input* target-filename))
(target-file-duration (/ (sound-duration target-filename)
(abs target-srate)))
(minimum-duration (if (null duration)
(min source-file-duration target-file-duration)
(min duration source-file-duration target-file-duration)))
(source-src (make-src :input source-file
:srate source-srate))
(target-src (make-src :input target-file
:srate target-srate))
(current-amp 0.0)
(test-amp 0.0))
(multiple-value-bind (beg end) (times->samples start minimum-duration)
(run
(loop for i from beg to end do
(setf test-amp (src source-src))
(if (> test-amp current-amp)
(setf current-amp test-amp)
(setf current-amp (- current-amp decay)))
(outa i (* current-amp (src target-src)))))
(close-input source-file)
(close-input target-file))))
The key thing to notice here is the way to open up soundfiles and use them in instruments (with the (open-input*), (make-src), and (src) functions). Once you can do that, you can use samples any which way....
data applied at the musical level: soundfiles as parameter lists
There's no reason why we can't apply data at the musical level as well as the soundfile level. For instance, here's an example which scans through a speech soundfile (we could also specify any other soundfile). It chooses the peak from a batch of 1000 samples (or any arbitrary sample-size) and applies that value to pitch; then does similar scans through additional sets of samples to find parameters for duration and timbre. Finally, it chooses the maximum of the pitch, duration, timbre triple and applies that to amplitude.... This is a somewhat crude example but with a more thoughtful design (and tuning of parameters) it might yield interesting results. We've already got a certain correlation between high pitches, long notes, and bright timbres -- the relationship to the original speech rhythms is perceptually obscure, but I would argue that there's an effect -- more dramatic over a longer time scale, I suspect.
Note that this runs slowly because it's not taking advantage of CLM's C optimization -- all of the examining of speech samples is done in interpreted lisp. No reason why we couldn't do something similar inside of a (run) loop....
(defun get-max-sample (soundfile survey-size)
(if (> survey-size 0)
(max (get-max-sample soundfile (- survey-size 1))
(abs (readin soundfile)))
0.0))
(with-sound (:statistics t :srate 44100 :scaled-to 0.9)
(let* ((soundfile (make-readin (open-input* "/usr/ccrma/snd/cburns/double1.snd" :start (round (* 2.5 44100)))))
(survey-size 1000)
(amplitude-scaler 0.5))
(loop for i from 0 by 0.1 below 30 do
(let* ((pitch (get-max-sample soundfile 1000))
(duration (get-max-sample soundfile 1000))
(index (get-max-sample soundfile 1000))
(amplitude (* amplitude-scaler (max pitch duration index)))
(scaled-pitch (+ 60 (* 600 pitch)))
(scaled-duration (* 4 duration))
(scaled-index (* 6 index)))
(if (> amplitude 0.1)
(fm-violin i scaled-duration scaled-pitch amplitude :fm-index index))))))
As always, there are countless ways to approach composition with datasets. Let us know what you come up with....