soundFloat is a 3D animated display that uses mic-input to alter elements of the display in real-time. It is written in C++, renders graphics with OpenGL, and processes incoming audio with RtAudio. A main component of the visual is a 3D waterfall plot of Fast Fourier Transform (FFT) magnitudes.
Download the project (includes readme.txt) here: soundFloat.zip
For a more extensive description, read below.
I constructed a water scene that is meant to be calming and enjoyable to watch. The waterfall plot of the FFT of incoming sound and its mirror started this water themed project, and the addition of fog made the water in the distance look similar to foam on a shore. To note, a new FFT is drawn only every 20 FFTs because I prefered the rate of drawing new waves and receding history to be slower. I aimed to create a fairly calm scene colored with many cool blues, purples, and greens. Initially, the waterfall plot and "party moon" are present, and soon enough a school of fish begin to swim across the bottom of the screen. The user can change a few additional elements.
There are a couple of key commands the user can take advantage of:
'm' | toggle horizontal moon movement on/off 'b' | toggle floaty on/off in the middle of the water 'r' | toggle raining fish on/off (and when it is on, horizontal swimming fish go away)
The others allow the user to change their viewpoint, moving through and around the scene. Note: I actually ran into many problems with view repositioning, so in the code, all of the objects are translated to make it seem as if you are changing the viewpoint. The effect is the same.
A few things respond to sound here other than the waterfall plot of the FFT:
The moon changes colors based on the first 361 time-domain buffer signals from sound input. The "inner" part of the moon is made up of many triangles drawn from the center point, and the color of each is correlated to a time-domain buffer signal. The triangles that make up the outer part of the moon (which is just a bigger moon layered under the smaller moon) are simply random shades of blue and green.
The floaty changes rate of horizontal movement and very slight rotation amount according to a fraction all the FFT signals summed together.
The school of fish have one horizontal path in each direction. The y-coordinate of this path is determined by using the first signal in the time-domain buffer. First the sine of the signal is taken, then shifted to be from 0-1, then randomly scaled negative or positive. The random negative/positive scaling is including so the fish are never out of range for an extended amount of time.
© 2013-15 - Holly Jachowski