Anna Huang on Deep Learning for Music Composition: Generation, Recommendation and Control
Date:
Fri, 04/19/2019 - 10:30am - 12:00pm
Location:
CCRMA Seminar Room
Event Type:
Hearing Seminar Anna Huang, a Google AI Resident and a Harvard PhD student, will be at CCRMA on Friday to talk about her efforts as part of Google's Magenta project, to understand musical compositions in a “deep” way. People like David Cope (UCSC) have tried to capture musical patterns and generate new compositions. While, more recently, people like Marcus Pearce (UCL) have built Markov models to capture the statistical predictability in music. The new DNN approaches take these ideas to a completely new level.
You might have already seen Anna’s work---It drove the Doodle on the Google home page to celebrate Bach’s birthday. Her model of Bach’s style harmonized more than 50 million(!!!) melodies entered by users on that day. That sounds like a nice birthday present. Come find out how it is done.
Who: Anna Huang (Google Brain and Harvard)
What: Deep Learning for Music Composition: Generation, Recommendation and Control
When: Friday, April 19th at 10:30AM
Where: CCRMA Seminar Room
Why: Can neural networks capture the essence of music?
For more details about her Bach work visit the Google doodle page, click on the play button to create your own, and read about the technical details: https://magenta.tensorflow.org/coconet.
Come to CCRMA on Friday morning to see what neural networks, both real and deep, can do to generate, recommend and control our music. This will be deep!
- Malcolm
P.S. I expect this will be popular, so come early to get a seat at the table.
Deep Learning for Music Composition: Generation, Recommendation and Control
Anna Huang (Google and Harvard)
Technology has always helped expand the range of musical expression, from the fortepiano to synthesizers to electronic sequencers. Could machine learning further extend human creativity? We explore three ways deep learning supports the creative process: generation, recommendation, and control. Generative models can synthesize stylistic idioms, enabling artists to explore a wider palette of possibilities. Recommendation tools can assist artists in curation. Better model control helps artists stay in the creative loop. Furthermore, this control could take place at one or more musically-meaningful levels -- the score, the performance, or timbre -- or on a non-musical level, such as a subjective quality like “scary.” I posit that deep learning models designed to better match the structure of music can generate, recommend and provide control in the creative process, making music composition more accessible. I describe four projects to support this statement. AdaptiveKnobs uses Gaussian Processes to capture the nonlinear multimodal relationship between low-level sound synthesis parameters and perceived sound qualities. By using active learning, we assist sound designers in defining their own intuitive knobs by querying them on sounds that the model expects to improve the controls most. ChordRipple uses Chord2Vec to learn chord embeddings for recommending creative substitutions and a Ripple mechanism to propagate changes, allowing novices to compose more adventurous chord progressions. Music Transformer uses self-attention mechanisms to capture the self-similarity structure of music, generating coherent expressive piano music from scratch. As the model processes composition and performance as one, improvisers can play an initial motif and have the model develop it in a coherent fashion. Coconet uses convolutions to capture pitch and temporal invariance. The generative model can fill in arbitrarily-partial musical scores, allowing it to perform a wide range of musical tasks. The model uses Gibbs sampling to approximate how human composers improve their music through rewriting. Recently, Coconet powered the Bach Doodle, harmonizing more than 50 million melodies composed by users. We hope machine learning can enable new ways of approaching the creative process for both novices and musicians.
Bio
Anna Huang is currently an AI Resident on the Magenta project at Google Brain, working on generative models for music. She will also be defending her PhD in computer science this May at Harvard University, with a secondary field in music composition. She designed the machine learning model Coconet that powered Google’s Bach doodle last month. She was a recipient of the NSF Graduate Research Fellowship and the Josephine de Karman Fellowship. She spent a year as a visiting research student in the Montreal Institute of Learning Algorithms (MILA) at the Université de Montréal. The summer before, she interned at a music startup called MuseAmi. She has won Best Hack at the Hack Audio and Music Research (HAMR) hackathon, lead workshop at Music Education Hackathon (as part of the Monthly Music Hackathon NYC series). Her compositions have won awards including first place in the San Francisco Choral Artists’ New Voice project. She holds a masters in media arts and sciences from the MIT Media Lab, a B.S. in computer science and B.M. in music composition, both from the University of Southern California. She grew up in Hong Kong, where she learned to play the guzheng.
FREE
Open to the Public