homework #3

jack xiao

sequencer: doom

description

this sequencer portrays the impending doom that climate change poses to our world. the scene begins in a bright, colorful scene with rich green and blue. but there is a deep, ominous drone in the background that is controlled by the color of the sky as it changes from blue to red and the size of the sun as it expands (color -> pitch, sun size -> gain). the main component of the sequencer are pieces of trash that appear on the ground, each one emitting a different sound that is in some way related to the type of trash it represents (plastic crinkle, crushing an aluminum can, buzz of radioactive waste, scraping scrap metal). the playhead is a car emitting a small but significant amount of poisonous exhaust as it travels across the screen. there is also a factory on the left side, with a chimney that spews out smoke when toggled. as emissions pile up (both from the cars travelling across the screen and the factory), the sky is gradually covered by a dark, dense cloud of smog. the more the smog builds up, the more distorted the sounds become (using a bitcrusher chugin). eventually, it becomes unbearably noisy and we have no choice but to start over. much like climate change in the real world, there is nothing we can do to directly reverse the effects. every action taken, whetherr it is darkening the sky, placing a piece of trash, or emitting smoke, is permanent and cannot be undone. we must live with our mistakes. so, when the distortion eventually reaches a maximum, the only way to reset the sequence is to leave the planet. the rocketship on the right side of the screen can take off at any point, transporting us to a renewed, still livable land where the process of contamination begins once again.

video

build

final build (for MacOS - Intel64) (very laggy)

screenshots

the beginning

doomsday

 

just a bit of trash

doomsday

 

uh oh smog is building up

doomsday

 

it's geting bad out here

> doomsday

 

rocket lifts off

doomsday

 

rocket arrives on new land

doomsday

 

unity project

final unity project

instructions

reflection

the inspiration behind this project is my concern for the severe impacts that climate change is having/will have on our planet. this scene is designed to represent an expedited story of climate change, where in less than a minute we may very well have ruined the scene by blacking out the sky with smog and distorting sound beyond repair. i chose sounds that were dark, percussive, and ominous in order to convey the fearful nature of the scene.

while i did have more experience with unity at this point, i still did struggle with the intricacies of unity and its limitations in communicating with ChucK. as i mentioned in my milestone 2 reflection, i had great ambitions for this project, but much of those hopes were dashed by reality as i actually made the attempt to implement it. i really wanted to implement the reversal feature, but i didn't have the time. i still at least was able to achieve the core of my aesthetic vision for this project, and i think it at least conveys the message well. there are a lot of moving parts in this one, so it was hard to coordinate everything, and there is most definitely room for improvement (addition possible features not-yet-implemented listed in the section below). again, latency/lag were present in the final build, but weirdly running it in the unity editor was a lot smoother than running the full build on its own. one key artistic/technical choice i made was to make every action in the sequencer "permanent". once trash is placed or the sky turns redder or the sun grows larger, there is no going back unless you reset the entire sequencer by using the rocket. while this makes it much more difficult to use and perform the instrument, this is an intentional part of the design to represent the equally irreversible changes that we have caused to our planet, and the difficulty of getting a satisfactory output without making too many mistakes is similar to the care we must take in prerserving the planet, as mistakes are costly. i did enjoy the design process and i had fun playing arouond with the sounds and using the sequencer, but i definitely faced my fair share of crashes, struggles, and compromises along the way (+ 2 all-nighters...i've had a busy week...)

i tried using some assets from the asset store to make the trash/car/rocket look more realistic, but i couldn't get them to work and just ended up designing every item manually using various 3D objects. i did use a terrain asset from the store to paint the mountainous background,

room for improvement

 

 

 

milestone 2: prototype + progress update

progress update

unity just seems to do the best job of dashing all my hopes and dreams. i guess the challenge and limitations of unity are just things that i will have to get used to. while it was true that latency was no longer and issue with this project, i faced other problems that made it equally difficult to make progress. firstly, i had to spend some time balancing out my ambition with the realities of implementation. this was to be expected, but i did not realize the extent to which i would need to change my design and sacrifice parts of my initial vision in order to make the implementation practical. it took me a good many hours just to get the sequencer coordinated and working, but there is (thankfully) some good news: now that there is at least some kind of working sequencer with a functional playhead and some sounds, it is slightly easier now to iterate and make modifications toward the final product. for this milestone, i focused primarily on just getting a sequencer working, and now that there's something there, i can take the time to refine the sounds i use, add more features, and improve the interface quality.

at the moment, i only have a single audio element mapped out, and that's the sound the plays when we "throw trash" onto the ground, which in this case is just a white cube that appears on the ground in a specified position (this will be updated for the final sequencer). yhe location of the cube controls rhythm, still figuring out how to include pitch here. besides this sole audio element, I have most of the other visual elements running (not yet mapped to audio though). we have a factory chimney that emits smoke (can be toggled on/off), a sky that can change colors from a clear blue to an apocalyptic red, and a sun that can change size. i intend for the chimney to control distortion/noise, the sky color to control the pitch of a deep, background drone, and the sun to control the gain. one other cool feature: the longer the chimney runs and emits smoke, there is a smoky particle cloud in the sky that slowly begins to grow in size and density to emphasize the pollution. for the final sequencer, i will work on getting the rest of the audio elements mapped out and refining the visual elements with smoother animations.

next steps

video

 

 

milestone 1: tutoial/research reflecttion + initial design

 

reflection

now having some unity experience under my belt with the audio-visualizer, i was much more comfortable with the content this time around. thinking forward, i expect that i will have an easier time with this project because it will likely involve less latency problems (no live audio input, no arrarys of thousands of game objects in consant motion) which was one of my biggest struggles for the last assignment. this assignment also uses ChucK as a more fundamental part of the design and operation, which i am happy about since i have more experience with ChucK compared to unity and i feel more confident using it ambitiously. i didn't find the chickencer tutorial to be too complicated, and i definitely see how i can use elements of it for my own sequencer.

there also is a good deal of overlap between this sequencer project and the drum machine project that i did in music 220b. both involve manupulating different aspects of the music using ChucK. the difference is the one uses the laptop/mouse themselves as the interface, and the other involves designing a unity interface. still, i think many of the ChucK operations and techniques will be transferrable.

step sequencers are super common and i've definitely used some before (especially step sequencer vst plugins, many drum/rhythym plugins). i think generally, the design and interface of most step sequencers (and pretty much all of the ones i've used) have similar interfaces and similar ways of representing the musical characteristics (even if the aestheic of the sequencer itself may be vastly different). each sound gets a line, and plays on the highlighted/selected/toggled portions of the line. i think this similarity and standardization is useful (and very practical) for music making (especially for the music production industry in genres like pop, hip-hop, edm, etc hat use persistent looped rhythms/beats), so that all musicians/producers have sequencer skills that are transferrable and adaptable. however, it leaves less room for creative and artful design when it comes to interface design. a sequencer that has a uniquely designed interface with interesting and emotionally effective visual elements may be more fun too play around with, but less fun to utilize in a broader production project where control and consistency are favored. a sequencer that emphasizes visual experience may limit the number of deatures in the music that we are able to control, as opposed to more utilitarian designs that focus on giving the musician maximum capability to manipulate the sound. This project, on the other hand, might place more emphasis on the experience of the user itself, not the musical output.

 

 

below are some design sketches:

 

doomsday

doomsday

this is an apocalyptic cityscape/landscape controlling what i intend to be a sci-fi/doomsday sequence (kind of like the vibe of the soundtrack for dune). different aspects off the image will control and also reflect the different elements of the music, and there will be some movement/animation that also contributes to the visual experience that supplements the audio.

operation:

 

clock (again)

clock

this is a clock that controls a very rhythmic, somewhat standard drum machine. the uniqueness of this design comes from the interface, where we can see the drum machine in action via movement of the second hand, and we control a wide range of different sounds represented by the elements (that reside where the numbers would usually go). i plan to have each element be some object/image that embodies the sound it represents, and there may be little animations there that also enhance the visual experience.

operation:

 

rivers

rivers

this is a peaceful, calming sequencer to control a pastoral, fluid soundscape. the soundscape will intentionally not have any tempo control, as the sounds will be fluid and won't have many discernable tempo qualities. potential sounds are: running water, wind blowing, synth swells.

operation: