I wanted to explore the concept of synesthesia through an ASMR-y paint mixing video.
I messed around with a variety of sounds. First I tried some more techno sounds I found on the internet becuase they were titled things like "what blue sounds like". I didn't find this to have any sort of relation to the color and decided to go with nature sounds. When training my model I realized that the white noise sound was making the it choppy since there was a lot of white in the background. With the wekinator app, it was super simple to pinpoint this issue and remove the sound from the model.
You know when you bring carrots to the movie theater and want to sneak in a crunch at the most quiet scene, that's the inspiration.
Life is a highway, and I wanna ride it all night long....until some idiot that doesn't know how to drive cuts me off.
Working with Wekinator, FaceOSC, and VisionOsc was challenging, exciting, and most importantly, playful. It was difficult to get wekinator to recognize certain gestures accurately. I think this made for an interesting back-and-forth between training the model, adjusting the code, and learning how to "play" it better. In the beginning of my experiments, I would add more training data to increase its accuracy, however, this made the feedback loop longer. In road rage, I originally had 16 different rages and a host gestures, but failed to make the system more reliable with the addition of new training data. To shorten the feedback loop and increase accuracy, I pared it down the number of gestures which worked quite well. During my iterations with Chomp, I realized that adding an accuracy cutoff of < 0.2 resulted in fewer overlapping sounds. I wish I added this parameter to my other projects, but I completed this experiment last. Out of the two new experiments my favorite is Chomp. This really surprised me. Chomp is not the most interesting in terms of sounds, but it felt the most like an instrument when I was playing with it. In the same way I imagine a singer controls their vocal chords, I was controlling my lip shapes. As I spent more time with the model, I found that I was improving my skill to trigger specific sounds and play some beats. This experience of learning how to play it was very different from my other experiments. Perhaps what makes an instrument an instrument is learnable control, not just the ability to control it. I think this exists on a continuum. In the music world, I would put a grand piano as requiring the most learning, electronic keyboard requiring less, and a computer controlled piano requiring the least. In the art world, I would describe the two ends of the spetrum as oil painting and digital / iPad art. Both have a learning curve, but there is less inherent controllability in oil painting. Rather, oil painting requires a greater degree of learned controlability. Another interesting perspective from this is that we value a great oil painting more than a great piece of digital art. Is this the same for music? I think so. Why is that? Is it because more can be expressed when control is variable? Perhaps. Is it because in the same way a perfectly symmetrical face looks wrong to us, perfectly plucked strings lack some other quality? Is this "other quality" what makes art, art?
Support code for Color Mixing Model Support code for Road Rage Support code for Chomp
The starter code for this project was provided by Ge Wang. Sounds for the model were sourced from the following links.
Red Blue Yellow Black British Road Rage Life is a Highway Chomp