Neural Nets for Music

Convolutional and Recurrent Neural Nets can be used for a wide variety of applications in music: to pick out notes, to recognize musical genre, to identify instruments or recognize songs, to name just a few popular examples. In this workshop, we will introduce you to machine learning in general and neural networks in specific to create music recognition applications. We will experiment with a variety of popular audio-based ML libraries and compare their viability for different applications. To make a working application, you will implement simple recognition using transfer learning on pre-trained models and integrate that into a more general purpose instrument or tool.

This workshop is intended for musicians, makers, engineers, computer scientists, etc. Basic programming capability is assumed, but the workshop is open to everyone who is game to try something new.

Participants will need to bring their own laptops for this workshop. We will provide further details on installing software prior to the workshop. Participants will be supplied with a docker container (or similar package installer) to run the neural network with the required software (Tensorflow/PyTorch) preinstalled.

About the Instructors

Professor Wendy Ju  is an Assistant Professor in the Dept. of Information Science at Cornell Tech's Jacobs Technion-Cornell Institute, where she leads the Future Autonomy Research (FAR) Lab. Her work focuses on ways that interactive devices like robots and autonomous vehicles can communicate with people without interrupting or intruding.

Professor Ju taught Physical Interaction Design at CCRMA from 2008-2012, and collaborated with Professor Edgar Berdahl to create CCRMA Satellite, a platform for building embedded musical instruments and embedded art installations.  She holds a PhD in Mechanical Engineering from Stanford, and a MS in Media Arts & Sciences from MIT. Her monograph, The Design of Implicit Interactions, was published in 2005.

David Goedicke is a PhD Student at Cornell Tech with Wendy Ju. Within the FAR Lab, David prototypes systems to design and test behaviors of autonomous systems. The results are then often used to test interaction ideas or to train basic Machine Learning algorithms. His work has involved a variety of technical fields, from realtime audio generation in Max and C++, over embedded systems development for interactive devices to Virtual and Augmented Reality. Besides this technical work, David has been playing, composing and experimenting with music since he was five.