FluCoMa: Making Music using Machine Listening & Machine Learning
Description
This workshop will introduce and go in-depth on the Fluid Corpus Manipulation project (FluCoMa), a toolkit for making music with machine listening and machine learning within Max, SuperCollider, & Pure Data. FluCoMa (https://www.flucoma.org/) is a toolkit for making music with machine listening and machine learning within Max, SuperCollider, & Pure Data.
The toolkit offers audio decomposition tools to separate real-time and recorded audio into component elements, audio analysis tools to describe audio components as analytical and statistical representations, data analysis and machine learning algorithms for pattern detection and expressive corpus browsing, and audio morphing and hybridization algorithms for audio remixing, interpolating, and variation-making.
Participants will learn to use most aspects of the FluCoMa toolkit–with an emphasis on creative musical applications and outcomes. The math and theory behind these tools and algorithms will discussed when appropriate for building fluency and intuition towards artistic uses of FluCoMa. Mornings will be spent on lectures and code dissection while afternoons will be a combination of coding sessions and instructor support on individuals’ projects.
Participants may use either Max or SuperCollider. Pure Data users are welcome knowing that Pure Data won’t be represented in the course materials.
Who is this workshop for?
Musicians and researchers interested in incorporating machine listening and machine learning tools in their creative practice. No prior knowledge of these topics is necessary.
Prerequisites
Comfort using Max or SuperCollider for creative music projects. This course will not include introducing users to the basics of Max or SuperCollider.
Schedule
Day 1: Neural Networks & Supervised Learning
• Working with data in FluCoMa
• Timbral Classification with MFCCs
• Regression for manipulating high-dimensional control spaces
• Neural Network prediction using real-time audio descriptors
• Wavetable Autoencoder
Day 2: Temporal & Spectral Decomposition
• Slicing audio in time using: – amplitude – spectral onsets – transients – measures of “novelty”
• Sinusoidal Modelling 1
• Harmonic-Percussive Source Separation • Transients
• Spectral Decomposition with NMF (Non-negative Matrix Factorization)
Day 3: Audio Analysis & Plotting
• Audio descriptor analysis
• Statistical Analysis
• Plotting Data in FluCoMa • KDTree
• Concatenative Synthesis • Distance as a Measure of Similarity
• Scalers: Normalization, Standardization, & Robust Scaler
Day 4: Dimensionality Reduction & Unsupervised Learning
• Dimensionality Reduction with
– Principal Component Analysis (PCA)
– Multidimensional Scaling (MDS)
– Uniform Manifold Approximation and Projection (UMAP)
• Clustering with KMeans & SKMeans
Day 5: Misc., Presentations, & Next Steps
• Catch up on topics
• Presentations by participants sharing projects or progress from the week
• Next Steps and Future Directions
About the instructors:
Ted Moore (he / him) is a composer, improviser, intermedia artist, and educator based in New Haven, CT. He holds a PhD in Music Composition from the University of Chicago and recently served as a Research Fellow in Creative Coding at the University of Huddersfield (AY 2021-22), investigating the creative affordances of machine learning and data science algorithms as part of the FluCoMa project. His work creates dense, technologically-enriched musical experiences that fuse the sonic, visual, physical, and acoustic aspects of performance and sound. Ted’s work has been presented by leading cultural institutions such as MassMoCA, South by Southwest, and The Walker Art Center and presented by ensembles such as Talea, International Contemporary Ensemble, the [Switch~ Ensemble], and the JACK Quartet. Ted frequently performs on electronics using his laptop, modular synthesizer systems, resonant physical objects, lighting instruments, and video projection. Visit him at https://www.tedmooremusic.com/
Davor Vincze is a contemporary music composer whose artistic interest lies in meta-reality and musical mosaics. Inspired by technology and science fiction, Vincze searches for hidden acoustic spaces and unexplored properties of existing music. He holds a doctorate from Stanford University and has won numerous international awards such as the City of Stuttgart Composition Prize, the Noperas! Composition Prize and “Boris Papandopulo” prize for the best Croatian composer. His pieces have been performed all over the world by renowned ensembles such as Klagforum Wien, Talea and many others. In 2014, he founded a contemporary music festival, Novalis. His music is published by Maison ONA. Visit him at: www.db-vincze.com