Assignment 2: Beatboxing Beats

By Danny Mottesi

In this project, I extracted 4 different features (centroid, flux, rms, and mfcc) from 15 different recordings of me making beatboxing sounds. I then mapped those feature extractions to wav files of real percussive instruments. This allowed me to turn my live beatboxing into actual drums playing in real time.

Relevant Chuck Code:

Download Mosaic Extract Download Mosaic Mic Download Feature Extraction from all the human sounds I made Download names extractor

Using the mosaic beatboxing with a synth!

Reflection

Throughout this project, I learned how much I want the things I work on to have practical use. Every time I had an idea, I would ask myself, in what context would that be useful? I kept modifying my idea to fit this mold, from an application that shows you the beats you are playing with visuals to a looping device that could be used for quick beat creation based on your voice. But when I took a break from learning Unreal Engine or trying to use MLP to more accurately map my inputs to outputs, and just played with my instrument, I realized it was super fun to mess around with. I experimented with it, modified it, and played with it for hours while jamming on my piano or guitar, and I had a great time. It was a super liberating feeling to be able to accept that this project wasn't going to have some commercial value, and I could just have fun with it. Also, the fact that it wasn't a perfect translator from sung beatboxing sounds to real instruments meant the resulting beat would be way more interesting, introducing instruments in random places I wouldn't have thought to add. After two weeks of constant ideating and trying to come up with the 'perfect' idea, I feel that I'm more aware of what I'm trying to get out of things, and appreciate having fun for the sake of it a lot more.

Experiments in Phase 2

In phase 2, I experimented with using different songs on the mosaic mic. I decided to collect a folder of different percussive instruments (Kicks, snares, closed and open hats, cymbals, toms, etc.) and sound effects (as well as a silent wav file and one of a normal room), and it ended up with a very cool end product. You can beatbox, and the program will match the sounds you make. In phase three, I would love to make a program where you can create different layers of beats, where after a certain amount of time/measures you can add onto the next layer.

Video of Milestone 1: Drum Machine