MUS423 Research Seminars

Abstract: Classical DSP techniques have recently been overshadowed by deep learning for applications such as image recognition and audio generation. End-2-end learning has been key to this shift, enabling the optimization of high-dimensional nonlinear functions. However DSP techniques provide interpretability, modularity, and efficiency lacking from black-box deep networks. In this talk, I'll review the Differentiable Digital Signal Processing (DDSP) library, which enables direct integration of classic signal processing elements with deep learning methods.

Abstract: Conventional audio synthesis (TTS, voice conversion, enhancement, etc) often relies on acoustic feature representations (spectrogram, MFCC, F0, etc.) and a signal processing procedure that infers the waveform from these features. However, such procedures often introduce artifacts caused by insufficient information in the feature representation (e.g. iSTFT without the correct phase info) and/or an oversimplified synthesis process (e.g. a source-filter model). Recent advancements battle this problem using deep learning: WaveNet, for example, generates the waveform sample-by-sample based on acoustic features and previously generated samples using a dilated convolutional net.

Carmine Emanuele Cella, assistant professor in music and technology at CNMAT, will present work done in the last years in searching good signal representations that permit high-level manipulation of musical concepts. After the definition of a geometric approach to signal representation, the theory of sound-types and its application to music will be presented.  Finally, recent research on assisted orchestration will be shown and some possible musical applications will be proposed, with connections to deep learning methods.

CNMAT, University of California, Berkeley: http://www.carminecella.com/index.html

Over the past four years, Jatin Chowdhury developed a wide variety of audio DSP systems intended to be used by mixing engineers and instrumentalists, including non-linear digital audio effects, timbral converters, binaural renderers, physical models, and more. This presentation will give a brief overview of all his previous works as well as several still in progress, along with more in-depth discussion of recent projects including notGuitar (a real-time timbral conversion system), various distortion effects, and NewMixer (a non-traditional mixing tool).