Pitch-Based Interactive Sound Source Separation

For my dissertation, I'm working on the problem of taking a single audio recording (e.g. a pop song) and separating it into its respective sound sources (e.g. drums, bass, vocals, etc.). In particular, I'm interested in the idea leveraging interactive user-feedback to inform source separation algorithms and improve separation quality.

To incorporate user-feedback, we allow end-users to roughly draw or paint on visualizations of sound. In the demo below, we perform separation by pitch.

Papers