3/1/23

Weekinator Etude

The OSC command has entirely been working for me. I'm having the most issues with gettiing the OSC input to respond in order to train the model. However, these appears to prove difficult, especially for data that has to be sent over WiFi. For example, I had the most fun experiementing with this app called MotionSender that sends accelerometer and gyrometer data that the iPhone is capable of accessing. I mapped this to the FM Synthesis output, which had really fun results swinging my phone around. However, the second I switched to another wifi it stopped working and I'm not sure how the connection is established. I also experimented with the video processing we did in class, classifying placement of the face on screen . Lastly, I tried to implement as PS5 controller as an input and find different combinations of pressing the keys. In the end, I think I really want to get the IOS app working as an input as well as the video processing so that I can try my body percussion idea out. I want to augment the gestures I create in body percussion with the output file and see what results I can make. Here is an example of what the video processing aspect would look like:

Oooooofff Wekinator continued to throw me for a loop. I attempted to send OSC from iOS and ran into considerable issues. After trying to start a local network which interfered with my local host and trying to figure out the app MotionSender, I was finally able to use sensors within the iPhone as inputs into wekinator for PhoneWek. For SquareWek, I used the basic x and y input provided in the walkthrough and paired it with a granular synthesis output file in Chuck that used the 1st season of the anime Jojo's ending song. The checkpoint used processing to control the 5-parameter synthesizer. I realized that the overall system that I want to build from this is combining all of these takes into one. I wanted fluid motion but also reproducibility in my instrument so that I could create unique sounds when performing body percussion. The iOS system coming from an iPhone, which used an accelerometer and gyrometer, was far too noisy but gave interesting mappings to ranges of motion. The square gran synth was closer to the sound I was going for, and I feel that Processing could add more inputs to distinguish the sounds that come out of the instrument. This assignment really demands practice and figuring out fine tuning. I realized there are infinite ways to put all these wekinator inputs and outputs together and I determined to find the right combo. There were a few potential inputs that left out such as the pitch, row and yaw which could give more dimensionality. I was also playing with the grain position through keyboard input while allowing Wekinator to change the grain position as well. This created the sound that was produced in the SquareWek, which I hope to explore more. I may also experiment with the rate at which the input is sent as well as filtering that data.