"CHAI-GPT, what is King - Man + Woman?"

Find out my answer on this Video

I initially started off by putting together an AI-generated mosaic of Bon Iver's songs, which I titled 'Bon AIver'. He's my favourite artist, who often well-represents my internal solitude.

Then, I realised that I wanted to do something that was innately relevant, something more meta. Also, I missed using Chunity, which I learned during MUSIC256A days. I am still very new to AI, and this is my very first AI class, so I wanted to utilise it to represent some beginning stages of my AI learning.

The very first word2vec example that I learned was 'King - Man + Woman = Queen.' Ever since I came up with this example, I wanted to pursue this idea. I wanted a narrative where Chai is answering the question, in the style of ChatGPT, but using visuals though unity. This is how Chai-GPT-Visual came into being.

One regret that I had was that GPT is a misleading name, because I do not actually use the GPT NLP model here at all. This is something that I figured out through receiving class comment. Someone asked if this used the Chat-GPT, which is not the case.

Overall, I am satisfied with my concept, and how my ideas were ultimately ridiculous and playful, which was something that I was going for. I wasn't satisfied with my recording / video quality and recognised a last minute mistake. I had to record this while I was sleep-deprived, and ultimately, it didn't capture the 'Off with his head' transition.

Going forward, I hope to be able to more fully use the creative aspects of AI by familiarising myself more with the technical aspects of it. I sort of get the upchuck as of now, but I was still not very comfortable utilising the diverse aspects of the provided code other than changing a few parameters.

This was a really fun exercise, and I don't think I would have enjoyed AI-learning as much if I hadn't started with this course. I am looking forward to the coming parts of this class. I recently went through a lot, including being hit by a truck (no injury, thank goodness), but I do want to achieve and do more for the next project!

Bon AIver

Final

Video

PAST IDEAS BELOW

I used Bon Iver's songs to create a MosAIC that makes everything moody and melancholic. I am reading my reflection on the first etude which is synthesised using three songs: Skinny Love, Holocene, and iMi.

Video

Phase 1

Phase 1 was incredibly fun! It was a great experience fro me to learn about MFCCs, Chroma, and Kurtosis, which were terms that I have never heard of before coming to this assignment. Some of the experiments are documented below.

Adding Kurtosis didn't really improve the scores as much as I expected.

fold 0 accuracy: 0.3657
fold 1 accuracy: 0.3216
fold 2 accuracy: 0.3289
fold 3 accuracy: 0.3525
fold 4 accuracy: 0.3691

Adding Chroma to the combo (and not Kurtosis) had mixed results: it is the only test that went above 0.4 in any category, but the general average was not very high.

fold 0 accuracy: 0.4005
fold 1 accuracy: 0.3328
fold 2 accuracy: 0.3426
fold 3 accuracy: 0.3412
fold 4 accuracy: 0.3794

Adding RollOff also stayed below 0.4, but one notable thing about this mix was that it remained relatively stable at around 3.65.

fold 0 accuracy: 0.3583
fold 1 accuracy: 0.3848
fold 2 accuracy: 0.3882
fold 3 accuracy: 0.3608
fold 4 accuracy: 0.3667

Adding ZeroX also stayed below 0.4, but was generally high except for fold0.

fold 0 accuracy: 0.3505
fold 1 accuracy: 0.3794
fold 2 accuracy: 0.3784
fold 3 accuracy: 0.3833
fold 4 accuracy: 0.3995

Adding everything together was as below.

fold 0 accuracy: 0.3667
fold 1 accuracy: 0.3225
fold 2 accuracy: 0.3657
fold 3 accuracy: 0.3696
fold 4 accuracy: 0.3725

Everythign minus MFCCs. Shows MFCCs are really crucial perhaps?

fold 0 accuracy: 0.3088
fold 1 accuracy: 0.3225
fold 2 accuracy: 0.3191
fold 3 accuracy: 0.3206
fold 4 accuracy: 0.3044