Sound Library is an audio visualizer with the appearance of a library. The waveform is represented by the pair of alphabet strips, and the spectrum, together with its history, is abstracted as the length, width, and height of books. If the loudness of the audio hits a certain threshold, there will be paper falling. The corresponding narrative depicts how three different instruments with the same main melody but various frequency ranges try to join the original piece. As more of them collaborate with each other, their volume and rate become closer to the target, finally accomplishing the performance in a union.
Though vision dominates people's perception, sound encodes a large amount of information as well. Similar to read written texts, people should also read sounds. Accordingly, I picked alphabets and books to encode audio data. The user is surrounded by books, simulating how people perceive sounds. In addition, to represent a burst of information when the sound gets loud, I add the falling paper effect, indicating the sense that books can no longer restrain the pages. During this project, apart from unfamiliarity with the integration of ChucK and Unity, the implementation part is relatively straightforward. Instead, what really takes time is the design processes such as camera parameters and what frequency attributes to demonstrate. However, it is indeed making these choices that adds meaning and value to Sound Library.
Based on my first idea, I experimented with alphabets, pages, and books, then decided to use alphabets to represent waveform and books to represent spectrum. As they are all detailed objects, it is impossible to align hundreds of them, so I chose average, standard deviation, and maximum to summarize the mass information into limited number of alphabets and books. Scaling these values was definitely the most time-consuming part since each of them ranges quite differently. To decentralize the frequency changes, I also distributed the frequency bins unevenly to the books. After implementing these basic functionalities, I started to play with the perspective, and surprisingly, with the perspective shown in the video, the whole thing looks like an auditorium.
Unity: The Roll-a-Ball tutorial is quite easy to follow, familiarizing me with basic functionalities that I need when creating a game, including constructing game objects, reading keyboard inputs, animating objects with physics, controlling collisions between rigid bodies, and adding texts. Though most of the functionalities in Unity are handy thanks to the visualization, I find it hard to figure out which to use if I want to implement features that are not in the tutorial. Maybe it will take me a while to read through documentations each time until I master more about Unity.
Chunity: The first half is pretty straightforward, enabling me to insert ChucK scripts smoothly. Yet, later parts, such as the "ChucK Global Variables" section, require more time to follow up, and I may have a better understanding while actually applying them to my own applications. Also, I did not realize that I had to move the StreamingAssets folder to the top level until staring at the error saying that the audio file was not found.
Audio Visualizer: This tutorial builds a Unity application from a different perspective. Instead of first creating game objects and attaching scripts to control them as what Roll-a-Ball does, this audio visualizer is mostly driven by scripts alone with game objects as simply placeholders. In this way, I have discovered more similarities with my previous game design experiences using pure code frameworks, which may be applicable to Unity as well.