Presenting the newest advancement of ChatGPT - SingGPT! SingGPT will fulfill your wildest dreams and sing to you using the lyrics of your favorite singer. Enter any statement or question, and SingGPT will respond.
To create SingGPT, I used a combination of various machine learning and music tools. SingGPT uses the GPT-3 API to query the GPT-3 text-davinci-002 engine for it's responses using a hand-tuned custom input prompt that asks GPT-3. Once GPT-3 returns a response, it will speak it using a text-to-speech generator. This text response is also then used as input into a sentence transformer, MiniLM, which is used for semantic similarity. Using the HuggingFace MiniLM 6 interface, I search a database of lyrics for the most similar line. For the audio part I use ChucK to play the audio files. For each song, I downloaded the corresponding LRC file that encodes lyric time information. For the selected lyric, I used ChucK to play the corresponding segment of the song.
I found that the most difficult part of this project was figuring out what I wanted to create. At first, I was experimenting with controlling my musical mosaic through the video camera input by reading and using the RGB values of the pixels. This is what I did for my milestone. Afterwards, I continued to iterate of this and try to make the sounds more controllable. However, I felt uninspired and frustrated by this – I had difficulty making it responsive or fine-tuned to my actions and working on it and using it felt so boring. One day, I was watching some YouTube videos related to Taylor Swift and inspiration suddenly struck. What if I could use ChatGPT to sing in Taylor Swift lyrics? At the last minute, I changed my project. I ended up creating something that I love and that I was excited to work on and use. I had so much fun tuning the GPT-3 prompts and testing out different song lyrics and seeing what kind of responses it would come up with.
I think this entire process helped me realize the importance of creativity and fun. While this may seem like an obvious conclusion, I felt it very keenly for this project. Although both projects I made are equally whimsical and useless, I think using them is a very different experience. To me, my milestone felt aimless and directionless – I made it just to make something. However, with SingGPT, I wanted to make it great. There are still so many aspects I want to add to it such as taking in voice input through a mic, creating a fun chat interface for it, adding in more singers and lyrics, adjusting the GPT voice, and more. This was fun and silly to make, and I felt a great sense of satisfaction seeing it all come together. There’s so much that can be created in this space, and I think working on this project made me appreciate why I’m working in this space at all.
For SingGPT, I used the GPT-3 API and MiniLM 6 API to query for the responses. I used the site Lyricsify for the LRC files. I was inspired to work on this project by the videos by Omar Nova on YouTube.
For this milestone, I tested using video inputs to control my audio mosaic tool. I wanted to see how I could use data from video to control the sounds that I made. I worked with trying out different forms of visualization for the video as well as different parameters. For this iteration of the project, the input sound frequency is controlled by the brightness of the pixels.
For this milestone, I used the songs "Surge", "Bedtime Stories", and "Lost and Found" by Amon Tobin.