Research

Harmonic Neural Networks

PDF of the paper

Github link

I trained Recurrent Neural Networks to generate harmonies and compared it to Markov chain-generated harmonies!

Markov chain-generated harmonic accompaniment

Markov chain-generated harmonic sequence

RNN-generated harmonic accompaniment

RNN-generated harmonic sequence

Chord2Vec

Chord2Vec is a vector-based harmony generation system, based on Word2Vec. The vector-based encoding lends itself well to chords, and the temporal context that is included in the encoding works well for generating music in particular. I trained two different Chord2Vec models for this project, one using Bach chorales and one using Jazz standards. I chose these two styles in particular because the datasets were easily accessible for training, and because they are stylistically very different.
After training, I created this user interface that allows the user to customize how much of each style they want to incorporate into the music that is being generated. You first select a melody and then the harmony starts generating. Users can shift between chorale-style and jazz-style generated music by changing the instrumentataion, the rhythmic performance, as well as the harmony that was generated by the two models.

Projects

Foot Piano

Foot Piano is a VR piano inspired by the movie "Big". It uses the Vive VR headset to show the environment and the piano to the player, and Vive trackers strapped to the feet in order to play the piano. It uses similar game mechanics as video games such as Guitar Hero and Dance Dance Revolution, where the notes that need to be played are shown in space at the time they are supposed to be played. You can select from pre-set songs or play the piano freform.

Constellation Sequencer

Link to Full Explanation

DNA Visualizer