Don’t Look Away

November 17, 2020

https://www.youtube.com/watch?v=p0L6TbgL2pI

Why did I make this? I didn’t know what the result would be until I finished it, and I definitely ended up in an unexpected place. I threw in a kitchen sink of ideas (and got stuck with a lot of bugs along the way). There is two way communication between ChucK and Unity. The user can scroll the mouse to affect the cutoff of a low-pass filter on the synthesizer. All of the timing is controlled by ChucK. ChucK does the FFT analysis and Unity synchronizes the frequency bins as a float array.

Visual controls The user can move with WASD first-person controls and look around with the mouse. Scrolling the mouse affects the distance at which particles spawn. No matter where the user looks, particles spawn concentrically in the user’s gaze and persist over time.

Audio There are four instruments: a synthesizer, a hi-hat, a kick drum, and a cymbal crash.

Synthesizer: I sought to emulate some of the basic features of the Serum synthesizer. It has polyphony (so it can play chords), and each voice has 7 unison voices. 3 voices get panned left, and 3 get panned right. One stays centered. The 6 panned voices are detuned so that if you ordered them from lowest frequency to highest frequency, you’d alternate between the left channels and right channels. I think that this helps minimize phasing artifacts as much as possible. I also use the BlitSaw instead of the SawOsc to fix aliasing issues. The phase of the BlitSaw is randomized too. Envelopes control the amplitude levels and filter cutoffs.

The other instruments inherit from a custom Sampler class that uses SndBuf. The cymbal crash goes into an FFT and RMS analysis in ChucK. This loudness value gets synced to Unity and affects the dispersion of particles.

Visual algorithm In Unity, for each instrument, I make an array of 12 zeros, one for each note on a piano. I go through each bin in the FFT and add a value proportional to its loudness to the note that corresponds to that bin’s frequency. Then I have an approximate loudness of each note from the FFT. Then I draw a different lissajous for each note. All of the lissajous for an instrument are the same color, but if you see a difference in the aggregate lissajous shape between different instruments, that’s because of the difference in tonality of the instruments. The particles are transparent and blend additively.