Homework Assignment #2
Milestone 3: The Frog Chorus
In the video, my narrative begins at 0:36. Press spacebar in the build to play the narrative, but before that you can free roam with the camera using the arrow keys and the mouse (hold right click on the mouse to move the POV).
I am operating on very little sleep and if I have to listen to Whitney Houston one more time I am going to lose it. I changed the frogs to be slightly different sizes, and each one corresponds to a different bin of frequencies, so the more low frequencies there are, the more the mouth of the left frog will move, the more mids, the more the mouth of the right, and the more highs, the more the mouth of the middle. I also had them bob up and down along with the spectrum history waves so it wouldn't be intersecting into them (although this issue still occurs if you get really loud). To create the narrative, I wanted to stick with the original song I used, so I played the clip in ChucK but fed it into a bunch of pitch shifters and chorus effects so that it would properly sound like a full chorus was singing -- also added in some small "bonus vocals" that line up with some appearing/disappearing frogs!
Milestone 2
Here's the video demo for this week's milestone. I present the frog chorus.
Originally, I wanted to do this with a singular face that stretched and distorted while yelling to visualize the sound, but I found altering the mesh of a complicated form like a head to just be a bit too difficult (though I might come back to it later). Instead, I pivoted towards something simpler and more enjoyable, 2 half-spheres that would rotate based upon the volume level to form a frog mouth, the frog tongue being the waveform, and the tongue of the frog increasing in length based on the volume as well! I definitely had some difficulty with figuring out how to rotate the half spheres in a way which was jointed, which I accomplished by having the spheres be the child of an empty object at a point located at the end of the sphere rather than the middle. This later caused some difficulties when trying to rotate the frog as a whole, as I originally was changing the absolute rotation rather than the local rotation. Then, I added in a mesh generated water which acts as a spectrum visualizer. It's a bit hard to see but it stores 16 of the previous spectrums, moving it along in a wave-like shape. Interestingly enough I found this idea to be much more flexible in expression and much more fun to work on, with it being so much more silly and lighthearted.
There is definitely a lot of room for fun experimentation here that I wanted to test out. For one, I am thinking of instead of directly translating the spectrum into the water, to lump them into 6-8 bins representing clumps of frequencies, then these can shoot up like a water spout depending on the amplitudes of those frequencies, and at the end of the water spout would be a lily pad with the frog on it! If not that, then some way for the frogs to be moved by the water, as right now it simply just cuts through the frog models. Also, I'd like to change the frog mouth movement to be more based on the difference between volumes at points rather than absolute volume, so that the mouth movement is more accented, since for some music pieces the frogs just leave their mouths open, which isn't all that interesting. Overall, I'm looking forward to continue experimenting and messing around with this, it has been super fun!
Milestone 1
Unity/Chunity Tutorials
Completing the first Unity tutorial and the first ChucK tutorial was fairly straightforward, I definitely think in this small scale demo it is pretty easy to follow everything that's going on but I can also see how this would get pretty out of hand on a larger scale project! I altered the wall impact sound to be the 'vine boom' sound effect and also added in another UI text element to display the loudest dac at any point in time. There seems to be a strange issue where the sound plays twice upon collision? Not sure what is happening there but maybe it's something I'll look into.
Visualizer Tutorial
The visualizer tutorial was really enjoyable as well! I definitely can start to see how this can be adapted to other sections within Unity combined with the previous two tutorials and I'm looking forward to doing a lot of experimenting.
Visualizer Brainstorming
I had a few ideas when consider how one might visualize these two forms:
- Rather than just expanding/moving the cubes that form the waveform/spectrum, maybe launching a single prefab object into the air (applying some amount of force onto it upwards based on the magnitude)? Especially if the prefab object is something a bit silly, I think it visually could get really insane really quickly, although in terms of technical limitations (you'd have to be doing physics calculations for hundreds of thousands of objects for even the shortest period of time) this one probably is not feasible -- I'll have to look into it more to see.
- Another idea would be to have the visualizer be a face screaming, where the waveform be visualized through the lips of someones mouth (mirrored to have upper/lower lip), and the spectrum be visualized through the tongue. I think this one is pretty within the realm of possibility, but the main thing is making it still recognizable as a face.
