Adaptive and interactive machine listening with minimal supervision

Abstract: Nowadays deep learning-based approaches have become popular tools and achieved promising results in machine listening. However, a deep model that generalizes well needs to be trained on a large amount of labeled data. Rare, fine-grained, or newly emerged classes (e.g. a rare musical instrument or a new sound effect) where large-scale data collection is hard or simply impossible are often considered out-of-vocabulary and unsupported by machine listening systems. In this thesis work, we aim to provide new perspectives and approaches to machine listening tasks with limited labeled data. Specifically, we focus on algorithms that are designed to work with few labeled data (e.g. few-shot learning) and incorporate human input to guide the machine. The goal is to develop flexible and customizable machine listening systems that can adapt to different tasks in a data-efficient way with the help of minimal human intervention.

Bio: Yu Wang is a Ph.D. candidate in Music Technology at the Music and Audio Research Laboratory (MARL) at New York University, advised by Prof. Juan Pablo Bello. Her research interests focus on machine learning and signal processing for music and general audio. She has interned with Adobe Research, Spotify, and Google Magenta. Before joining MARL in 2017, she was in the Music Recording and Production program at the Institute of Audio Research. She holds two M.S. degrees in Materials Science & Engineering from Massachusetts Institute of Technology (2015) and National Taiwan University (NTU) (2012), and a B.S. in Physics from NTU (2010). Yu is a guitar player and also enjoys sound engineering. Japanese math rock is her current favorite music genre.