Deep Learning for Music Information Retrieval I: How Neural Networks Learn Audio

This workshop will cover the industry-standard methods to develop deep neural network architectures for digital audio using PyTorch. Throughout five immersive days of study, we will cover theoretical and practical principles that deep learning researchers use everyday in the real world. Our schedule will be:

 
Day 1: Cross entropy and feedforward neural networks
Math - Linear algebra and differential calculus review. The mathematics of feedforward neural networks. Activation functions. Batch Norm.
Theory - How synaptic neuroplasticity inspired the backpropagation algorithm.
Practice - Automating differentiation in a neural network with PyTorch.

Day 2: Dimension reduction techniques for audio
Theory - Dimensionality reduction. Principal Component Analysis. Autoencoders.
Practice a) - Finding interpretable features in the Tinysol and EGFxSet datasets with PCA.
Practice b) - Writing an autoencoder to denoise audio in PyTorch.

Day 3: Convolutional neural networks
Theory - convolution, optimizers and momentum, Loss functions.
Practice - writing a CNN for music genre classification

Day 4: Temporal encoding with RNN, GRU, and WaveNet
Theory - Architecture and data flows on a Gated Recurrent Unit (GRU).
Practice a) - Writing an RNN and a GRU in PyTorch and using it for sound event classification.
Practice b) - Reading the seminal WaveNet paper

Day 5: Generative Models
Theory - Kulback-Leibler divergence. Probability review, Variational autoencoders. Self-attention.
Practice - writing a VAE to use its latent space to generate parameters for an audio synthesizer.


Enrollment Options:

In-person (CCRMA, Stanford) and online enrollment options available during registration (see red button above). Students will receive the same teaching materials and have access to the same tutorials in either format. In-person students will gain access to more in-depth, hands-on 1:1 instructor discussion and feedback when taking the course in-person. 

About the instructors:

Iran R. Roman is a theoretical neuroscientist and machine listening scientist at New York University’s Music and Audio Research Laboratory. Iran is a passionate instructor, with extensive experience teaching artificial intelligence and deep learning. His industry experience includes deep learning engineering internships at Plantronics in 2017, Apple in 2018 and 2019, Oscilloscape in 2020, and Tesla in 2021. Iran’s research has focused on using deep learning for speech recognition and auditory scene analysis. iranroman.github.io

https://docs.google.com/forms/d/e/1FAIpQLSdL4LWoX5EpYUEp0UMFUhhmgMWOHkd8VlF70G9BK8e3-AfX2w/viewform