A Boat on the Waves (a play on Une Barque Sur L'Ocean), is a audio visualizer with the core concept of interaction through the frequency domain. The setting is staged as a slice-of-life scene with a boat drifting in the ocean. The clouds make up the time-domain waveform, and the waves the frequency spectrum. The main group of waves in the middle presents a waterfall plot of the current STFT as well as the previous 30 iterations. The waves, made up by the spectrum, exert rotational, vertical, and horizontal force on the boat. This allows the boat to smoothly interpolate its location on the crest and move along the waves while presenting a sensible visual representation. For example, a large wave from the left will tilt the boat right and push it towards the right. The higher the wave, the higher the boat will be too as a demonstration of its buoyancy.

I created this visualizer because I really like the slice-of-life aesthetic. When I usually think of narratives, or telling a story, there's always the classic "exposition, rising action. climax, denouement" paradigm. But this is highly unrealistic: reflecting on my life, and a bit of generalization to life in general, there are not that many moments that are categorizable into this structure. A life on this classic story structure would be so tumultuous in real life that I can't even begin to imagine the mental toll it would take. Thus, what is real to me is the mundane, and especially works that champion the mundane, because I believe that there is much more mundane in life than action-packed. Just think, is something like what I created more relatable, or is Homer's Odyssey, with many trials and tribulations closer to real-life experiences? In a narrative, does there have to be action?

Another aspect I wanted to highlight about the visualizer is a small commentary on the pace of things today. My attention span has noticeably decreased, no thanks to various online stimuli that capitalize off of this deficiency. However, in this scene, nothing substantial is happening. I believe this to be a small test of the vestigial ability to feel bored (a desire for desires as Tolstoy puts it).

Overall, this process was pretty smooth. I had a pretty clear narrative with a clear message that I wanted to deliver through in a set medium. Code-wise, not a lot was too challenging. This week, I was able to wrap my head around how CustomGeometry worked as well as how normals affected lighting, and was able to remake the clouds and waves that way. The boat physics was a bit of a challenge, however, but nothing a few basic kinematic equations couldn't help with. I will say that SLOrk helped a lot with the narrative coding, as it was much faster than before to execute on an idea that I had. I had a ton of different iterations that I went through for the narrative, like figuring out which parts I wanted to cut in order to create a cohesive abridged, ChucK-ified version of Ravel. Ultimately, I think it turned out pretty well. One of the challenges was introducing new material, but still keeping it "boring," as to not mislead the viewer into thinking that there would be more action going on, because the whole point is to just have nothing going on in a world where too much is going on too quickly.

I think I was pretty lucky to stumble on an idea that I thought was creative and worth pursuing pretty quickly. However, it wasn't my first idea, since my first idea met some pretty difficult obstacles early on. My first idea was to map sound to color, and have the window fill up with multicolored pixels according to the STFT. One of the biggest unexpected challenges with my first idea was the fact that there surprisingly wasn't a clear cut mapping from wavelength to hex color code. Furthermore, the more I implemented the idea and got a physical sense of how it would work, the more I realized that it might be too overwhelming on the user's end. For example, I noticed that it was pretty unintuitive what colors mapped to what frequencies since I don't usually think of colors on a continuum of wavelength, but instead as more discrete bins of colors. Moreover, since a pixel is pretty small, it was also hard to keep track of, since the history (previous pixels) was much more attention-grabbing than the current spectrum. This made it such that it was pretty bad at its job of being a sound visualizer and more of a novelty, something I didn't want since novelty wears off quickly, and once it did, it served little purpose for being able to visualize the sound. Thus, I switched to another idea, one that I soon would think of to be a better idea.

My second idea (the one I ended up moving forward with), was to create a small interactive display along with the audio visualizer. To do so, I came up with a setting of a boat on an ocean with spectrum waves. As the signal progresses in time, the boat moves along with the waves, oscillating back and forth depending on the frequency spectrum. With this, I could create something that still functioned as a working and easily readable audio visualizer, but with more interactivity and a stronger motivation to play around with it a lot more. Instead of just creating wacky waveforms, there's also opportunity for self-directed fun of trying to move the boat in various directions.

Throughout this process, I definitely encountered things that were easier and harder than expected, and overall I had a great time working with ChuGL. Something that was easier was that the whole structure of ChuGL was surprisingly intuitive, and I, with not a lot of graphics experience, was able to implement some advanced concepts to improve the framerate. For example, when I was adding the background waves, I added only around 40, but that was a substantial enough number such that it became a PowerPoint presentation. I was able to intuit (with the help of the tutorial) that it was due to the fact that there were 40 independent update calls that simply did the same thing, which was mapping the spectrum to the GLines GGen. Thus, what I was able to do was to use CustomGeometry and LineMaterial for the mesh so it only calculated the whole operation once, and then applied it to 40 GMeshes. It was definitely very satisfying to see that nothing changed, yet the framerate was much higher.

For some other parts of the project, I had to think outside the box to achieve my intended effect. For example, I needed a representation of a boat, but it was not feasible to use outside assets without a loader. To circumvent this, I used SphereGeometry and CustomGeometry extensively to recreate a similar effect. In some ways, I think this is more in line with the theme of the overall audio visualizer. I approached the visualizer using an almost sort of a minimalist aesthetic by stripping down natural things into their most basic shapes. Thus, a high-resolution, immensely detailed boat would look severely out of place in such a world. Furthermore, the rickety nature of the boat design that I did I feel adds to the aesthetic of just you and nature, like somehow a simple boat is closer to nature than a yacht would be.