HW2: Sound Peeking

Elisse Chow | October 11, 2023 — October 25, 2023

Floral Waves

This is Floral Waves, a real time audio visualizer that uses a variety of flower polar functions to drive the animation. Ever since middle school, I've had a slight obsession with these polar graphs, and have grown a massive collection of Desmos graphs containing most of the animated functions used here. Please see several of the ones used in this project here!

Instructions

To try this visualizer out, please download here!

Run with: chuck floral_waves.ck To begin the narrative, please press the "Start narrative..." button in the UI window.
Note: while the narrative is running, the other controls will be inactive.
Use the slider to change the polar function used being visualized.
Change the color palette by selecting from the dropdown menu.
Each time "Random" is selected, a new color palette is generated.

If you'd like to just hear the narrative piece, please download this file, and run it with chuck narrative.ck.

Reflection

After all the work put into this project, I can say that I am really proud of my visualizer. As I mentioned above, polar functions have always been a great love of mine, and in my Reading Response last week, I have had a history of incorporating these graphs into different projects as a CS major on the Graphics track. In a sense, this was also a callback to another musical visualizer that didn't use polar functions but rather physically simulated metal balls to "create" music that was really driven by an inputted MIDI file (see Wintergatan Flower here ). While the narrative was fun to create with Flute and Rhodey STKs, it's pretty clear that I don't have much musical composition experience. Honestly, I kind of just went into an online MIDI editor and tried to transcribe an old tune of mine that I used to sing for fun (it's missing the ending that I included in the narrative because I didn't end up saving the MIDI file and the tab reloaded T^T).

While I am happy with my final product, there's still a lot of things that I wanted to incorporate or improve but couldn't due to lacking time. For one, I was having a bit of a difficult time getting the narrative incorporated into the same file as the visualizer. I ended up having to refactor and rewrite the entire project and that was long and tedious, but well worth in the end. However, I ran into audio issues with crackling. Some parts were intentional including the peak of the chromatic scale at the beginning of the narrative, but otherwise it really detracted from the smooth effect I was hoping for. And since I wanted to keep some crackling while getting rid of others, I couldn't record through Blackhole and had to use my computer's default microphone to listen for the playback. Additionally, if I were to continue working on this project, I'd be interested in creating multiple waterfalls in space each corresponding to a different polar graph instead of changing on the one waterfall. Each time the user would change the graph choice, then I was thinking the view point could shift around the circle of options, showing all at once while zooming out, and then zooming in on the choice they chose. I think I'd love to work more with the 3D space, and I'm a bit sad I didn't incorporate it as much as I'd like.

With that all said, thank you for viewing my project!

---

Milestone 2: Design + Visualizing the Spectrum History

Download Project Here!

Command:

chuck Mouse.ck hw2_2.ck

Controls:

LEFT CLICK: Iterate through different graphs
RIGHT CLICK: Toggle animation on some graphs
MIDDLE CLICK: Toggle stem
                

Honestly, I'm a bit unsatisfied with the design because it feels like it's missing something crucial to make it make sense. I think when I think of animations, I really love working with flowers—especially polar flowers. But in this case I'm not sure I'm doing the FFT justice and showing the history clear enough. It feels that with the different graph animations we lose the information of the history and can barely see it through the points that are more recently created.

Regardless of these feelings, I had a lot of fun overall! I know I sort of started this in milestone 1, but I'm glad I now understand what the FFT stuff is doing and that I managed to create something has a semblance of smooth curves instead of really jagged ones. I really love how when you have sharp sounds that aren't continuous, the image gets preserved in the history and you can see it shrink.

Some definite limitations/things I'd like to improve upon include: the empty pizza slice at the far end of the spectrum (I asked about this on Discord, and it's apparent in sndpeek.ck but no luck at the moment), the overall aesthetic of the piece, the inputs (I'm using mouse clicks, but I'd like to use keyboard instead if possible), and the usage of the time domain wave—it's not the worst acting as a stem for the flower, but it doesn't look pretty and I want to see if there are other ways I can transform it but make it fit in with the overall story of the piece. Oh! And I'd love to clean up my code, because at the moment there aren't many comments and it's a bit unorganized.

---

Milestone 1: Brainstorming + ChuGL!

At the moment, I don't have an entirely thought out plan for my ideas for this assignment, but I have a sort of vague understanding of things I'd be interested in exploring.

For my first idea, I was interested in exploring the visual of a forest growing based on the shape of the FFT. I think it'd be something similar to the example shown in class with the airplane and the clouds, but maybe instead of an airplane, I'd have the camera follow a migrating bird. It's definitely not realistic to have trees pop out of the ground, but I think it'd be pretty, and something that highlights the beauty of nature.

Then for my second idea, I was playing around with the concept of cracking glass or cracks in the ground. In each case, there'd be something controlled by likely the volume of the input throwing an object that impacts the scene. The way the cracks/visuals are formed are based on the FFT. I think it'd be kind of fun to try to break through a series of walls and things and simulate that.

Finally, for my third idea, I think I might be interested in exploring more of an abstract visualization. I've already started down this path a little with today's ChuGL submission, but I was interested in how I could use sound to impact an animated polar graph. The speed of the animation could be determined by the volume of the sound, and certain coefficients in the polar function could be driven by the FFT. I know that I'd love to work in color more, but I don't have many ideas for that at the moment. But overall, I see this idea as either just an interesting experience or I could make it a game for the user to match a goal picture. Not entirely sure, but I'm definitely going to continue thinking on this.

For my ChuGL program, I decided to mess around a polar animation function that reacts to microphone input based on the amplitude. The higher the volume, the faster the animation progresses and the lighter the dots and lines appear. Unfortunately, the lines on the polar function are quite jaggy and I haven't figured out a good way to get more samples of the function without needing to scale the radius up a ton, but I'll keep working through this, and ideally in my final submission for HW2, I'll have something that works with more details about the waveform and the FFT instead of just volume.

Please watch the video to see the visualizer in action! I'm using Test Drive from the How To Train Your Dragon soundtrack to demonstrate what it can do at the moment.

See the fixed version here, with a prettier graph.