an Audiovisualizer and Virtual-Space Sonifier




Arrow keys -> control up/down left/right movement, plus associated synthesizer frequencies

A/D keys -> control Z-axis, plus associated synthesizer frequencies

1/2/3 numeric keys -> switch between camera angles

Built for MacOS

Unity files here


This project started with one idea and ended up in an entirely different one. Originally, I wanted to play with the idea of a pond – ripples on the surface of the water conveying the waveform, and the height of the surrounding grassy bank associated with the frequency values. I even thought about including a sun that morphed with the waveform above. 

Quickly, my initial demos diverged from the naturalistic scene and into a much more virtual space. I enjoyed playing with abstract ways of shaping the spectrum and waveform, seeing all the different shapes one could make – concentric rings, spirals, and complex winding and looping forms. I liked the shiny blocky virtual aesthetic and stuck with it. As I was trying to think about how to shape the 'narrative', now simply working with abstract forms, I tried to think of something similarly sparse and abstract. I chose very simple tones – one sine, one square, and one triangle waveform – and tied the values of the camera on the x,y,z axis to each frequency. Thus the audiovisualizer visualizes not only the sound, but the camera navigating the virtual space. Inspired by Ch3, I tried to create something that is both the blueprint and the final product. 

I wasted a lot of time in this process trying to fine tune things like colors and shapes, leaving some pretty basic functionality for the very end. This completely came back to bite me. I was trying to connect my ChucK script to the Chunity listeners in my scene, and couldn't figure it out. So I reimported the starter files – unfortunately, because I didn't change the names of the 'Spectrum' and 'Waveform' scripts, all the color/position/shape fine-tuning was entirely erased, and I still didn't have functionality!  

I started from scratch, and was able to pretty quickly pull it back together, close to where it was before, with all functionality. Still, lesson learned – focus on functionality FIRST and follow instructions carefully to avoid wasting so much time! 





This was the laborious result of trying to make something appear like water and ripples. It didn't quite get that effect, but kind of close! Happy and excited with where it is at the moment. I want to play with materials and lighting more, to try to get to a more physical feeling. Also, perhaps subtleties of movement, fading, different coloration. I still have to figure out how to represent the spectrum – I have an idea but didn't have time to implement it yet. I also haven't quite figured out how to record camera movement or record properly either, as you might be able to tell.... One fun unexpected result of this was noticing that my attempt at ripples on a pond kind of looks like a vinyl record – an idea I think I'll use to craft the narrative.


Initial YouTube sources from this milestone: 





Progress is slow but continuing! Unity is fun & chunity makes sense. Looking forward to making more progress...

Re: audiovisualizer:

  1. My primary idea is a sort of pastoral natural scene with a pond and trees and birds, where the ripples of the lake represent the frequency spectrum. If I can figure out onset detection, I would want objects (rocks, leaves, small bugs, animals) dropping into the pond with each new onset to "initiate" the ripples. I would want the scene to be navigable, like a videogame.
  2. Secondary idea totally disregards the realism. I imagine a sort of rippling, colorful 'carpet' where the time domain moves from right to left, and the frequency domain becomes overlapping, vibrating, colorful vertical lines. It would be an abstract design with an emphasis on color and texture.