CCRMA Open House 2017
On Friday March 3, we invite you to come see what we've been cooking up at the Knoll!
Join us for lectures, hands-on demonstrations, posters, installations, and musical performances of recent CCRMA research including digital signal processing, data-driven research in music cognition, and a musical instrument petting zoo.
10-12: Presentations throughout the Knoll
12-1:30: [lunch break]
1:30-2:20: Keynote lecture: Ryan Groves "Building Artificially Intelligent Musical Composers"
2-5: Presentations and open classes in the Knoll
We will continue to update the detailed schedule here, even during the event. Check back often!
Facilities on display:Neuromusic Lab
Max Lab and garage (maker / physical computing / fabrication)
Hands-on CCRMA history museum
JOS Lab - DSP research
CCRMA Stage - music and lectures
Partial List of Exhibits (lectures, performances, demos, posters, installations...)
This project aims to demonstrate the use of fundamental frequency estimation as a gateway to create a non-interactive system where computers communicate with each other using musical notes. Frequency estimation is carried out using the YIN algorithm implemented using the ofxAubio add-on in openFrameworks. This system is used to simulate a conversation between people at a round-table event, but using computers.
I Said it First: Topological Analysis of Lyrical Influence Networks
Jack Atherton, Blair Kaneshiro
We present an analysis of musical influence using intact lyrics of over 550,000 songs, extending existing research on lyrics through a novel approach using directed networks. We form networks of lyrical influence over time at the level of three-word phrases, weighted by tf-idf. An edge reduction analysis of strongly connected components suggests highly central artist, songwriter, and genre network topologies. Visualizations of the genre network based on multidimensional scaling confirm network centrality and provide insight into the most influential genres at the heart of the network. Next, we present metrics for influence and self-referential behavior, examining their interactions with network centrality and with the genre diversity of songwriters. Here, we uncover a negative correlation between songwriters’ genre diversity and the robustness of their connections. By examining trends among the data for top genres, songwriters, and artists, we address questions related to clustering, influence, and isolation of nodes in the networks. We conclude by discussing promising future applications of lyrical influence networks in music information retrieval research. The networks constructed in this study are made publicly available for research purposes.
Chapter 31, pages 415-926
Constantin Basica, Performed by the JACK Quartet & the Spektral Quartet
"Chapter 31, pages 415-926" is a piece for string octet and video, which was inspired by the shape of a circle and the concept of infinity. It is said that the number known as Pi, which is believed to have an infinite non-repeating sequence of digits, could include all possible combinations of numbers, and that one could find all the history of the universe in it. For example, looking through the infinite digits and converting the numbers to ASCII text or bitmap information, one could find all the texts ever written and all the images that ever existed. For this piece, specific strings from Pi’s digits were extracted and filtered algorithmically in Max, then applied in MaxScore to a series of one hundred chords in order to generate musical material. The same digits were used to determine the length of each video clip. All the video material was shot in California and represents a diversity of banal imagery, which reinforces the idea of Pi encompassing everything—even things that are usually overlooked.
Recorded 8-channel string octet performance with video.
The Contrenot is a musical interface designed to emulate the mechanics of a bowed upright bass. It has a form factor similar to that of an electric upright bass. The neck consists of a high-resolution linear softpot and FSR for monophonic pitch and aftertouch detection. The body houses a pull-string sensor, created from a gutted tape measure spring, a high resolution incremental rotary encoder, and custom 3D-printed parts. The pull-string sensor is able to detect velocity and motion very precisely, allowing for very nuanced control similar to bowing a bass. The Contrenot plugs into a computer to synthesize sound.
Demo - pettable instrument.
Automatic Music Chords Generation System
Ziheng Chen, Jie Qi, Yifei Zhou
This project applies machine learning techniques to generate music chords to follow a given melody. We compared the performance of different machine learning algorithms, and chose Hidden Markov Model for this project. Using MusicXML format lead sheets as training dataset, we calculated chord observation probabilities and chord transition probabilities. We then implemented the Viterbi algorithm to compute the most likely sequence of chords for a new melody.
Virtual Audio Player -- An Augmented Reality Project
Virtual Audio Player is an augmented reality project implemented in openFrameworks. Users can design and draw their very own audio player with markers, and play with it using fingers. A web camera is used to detect the finger position and audio player elements. The computer screen also shows waveform, slider value and other elements to interact with the drawing.
Seminal computer music composition with historical context.
Abe Davis, Justin Chen, Fredo Durand, Doug James
One of the most important ways that we experience our environment is by manipulating it: we push, pull, poke, and prod to test hypotheses about our surroundings. By observing how objects respond to forces that we control, we learn about their dynamics. Unfortunately, regular video does not afford this type of manipulation – it limits us to observing what was recorded. In this work we present algorithms for turning regular video of vibrating objects into dynamic videos that users can interact with. By analyzing subtle vibrations in video, we can extract plausible, image-space simulations of physical objects. We show how these simulations can be used for interaction, as well as for low cost special effects and structural analysis.
Ambisonic Mixing Bowl
Nick Gang, Wisam Reid
This project seeks to provide spatial audio artists and engineers with a tactile interface for real-time ambisonic panning. Users move and rotate physical magnetic objects around the surface of an acrylic dome. Positions and angles of rotation are tracked with an under-mounted camera and lighting system, processed with reacTIVision's open source computer vision software, and sent to Spat to control the spatialization. The dome's spherical shape mimics possible source locations surrounding the listener. The sound source symbols inform the user of the state of the system, and allow for immediate changes in three dimensions with one gesture.
Does anticipating a tempo change systematically modulate EEG beta-band power?
Emily Graber, Takako Fujioka
Neural oscillatory activities are known to relate to various sensorimotor and cognitive brain functions. Beta-band (13-30Hz) activities are specifically associated with the motor system; they are characterized by a decrease in power prior to and during movement (event-related desynchronization, ERD), and an increase in power after movement (event-related synchronization, ERS). Recent research has shown that listening to isochronous auditory beats without motor engagement also induces beta-band power fluctuations in synchrony with the beat; the rate of ERS is proportional to the beat rate suggesting that ERS may be related to temporal prediction. Top-down processes such as imagining march or waltz meters also induce modulations related to the temporal structure of the particular meter. Here we hypothesize that beta-band modulations may reflect top-down anticipation in other timing-related tasks that commonly occur in music performance. Specifically, we examine how the processes of actively anticipating two types of gradual tempo changes (accelerando and ritardando) are reflected in beta-band dynamics.
Building Artificially Intelligent Musical Composers
This talk will present some of the challenges faced when trying to build artistic machines. Through his work on Ditty, the automated musical messenger, as well as his work on creating adaptive musical scores for video games at Melodrive, Ryan will highlight the different musical components of the vast topic of automatic musical composition. Given his background of computational music theory, Ryan will emphasize the importance of building and validating machine-learning models that can perform particular musical tasks, and leveraging those to create artificially intelligent compositional agents.
Keynote Lecture (50 minutes).
Hzpiral is a polyphonic playable pitch-space in polar coordinates, developed with openFrameworks and Faust. Constant angle from the center is pitch chroma and radius is pitch height on a rotating playing area. Four custom-set chords are playable from a single touch. Control ADSR envelope, low-pass filter cutoff frequency and resonance, tremolo rate and depth, waveshape mix between sine, sawtooth, square, and triangle waveforms, and pitch-quantization, in addition to lockable sustain and sostenuto. Experience the visual mathematics of harmony!
Demo - pettable instrument.
Do violinists' gestures reflect voice entries in implied polyphony? A motion capture study
Madeline Huberth, Takako Fujioka
Implied polyphony is a musical structure in which a monophonic melody is perceived to outline multiple sources, or `voices'. Performers may express implied polyphony in their sound using dynamics and rubato, but it is currently unknown if musicians also embody the voices. In this motion capture study, we investigate if and how violinists' body motions indicate a change in implied voice at a specific moment in time, as well as characteristics of general motion during passages containing implied polyphony.
Music 251 Course Overview and Final Projects
Madeline Huberth, Nolan Lem, Tysen Dauer
Music 251, Psychophysics and Music Cognition, aims to introduce students to basic concepts and fundamentals of auditory perception, as well as music perception and cognition. The primary emphasis is placed upon experiencing all the processes of discovery-oriented human behavioural research and core issues in methodology as well as scientific discourse with colleagues. This talk will provide an overview of the course (happening this term), as well as present a selection of students’ proposed final projects.
Performers’ motions reflect their intention to express local or global structure in melody
Madeline Huberth, Takako Fujioka
Performers often must choose to emphasize short melodic groupings, or to more strongly integrate these groupings into a phrase. This study aimed to characterize the nature of motions associated with either choice. We filmed 12 cellists playing a musical excerpt in two conditions in which they were asked to think about either short or long melodic groupings, where groupings in both cases were specified. The results show that, overall, participants’ heads move more frequently when thinking about short groupings compared to long groupings. Our results illustrate that different melodic grouping interpretations by the same performer can be embodied.
Action-monitoring in piano duet performances
Madeline Huberth, Tysen Dauer, Iran Roman, Chryssie Nanou, Wisam Reid, Nick Gang, Matthew Wright, Takako Fujioka
Music ensembles involve complex social interactions between players in which coordination of actions and monitoring of outcomes are crucial in achieving joint goals. Previous event-related potential (ERP) studies of joint-action tasks have shown that the feedback-related negativity (FRN) is elicited when the outcome of one’s own and another’s actions is different from what is expected. One of the Neuromusic lab’s present interests is to examine if the FRN and oFRN differ depending on the strength of the joint goal in a piano duet task. This demonstration shows how our lab elicits the FRN in participants by altering the pitch feedback of piano keyboards.
Advances in physics-based sound synthesis
Prof. Doug James
Decades of advances in computer graphics have made it possible to convincingly animate a wide range of physical phenomena, such as fracturing solids and splashing water. Unfortunately, our visual simulations are essentially "silent movies" with sound added as an afterthought. In this talk, I will describe recent progress on physics-based sound synthesis algorithms that can help simulate rich multi-sensory experiences where graphics, motion, and sound are synchronized and highly engaging.
Using synchrony of cortical, physiological, and behavioral responses to index listener engagement with naturalistic music
Blair Kaneshiro, Jacek P. Dmochowski, Duc T. Nguyen, Anthony M. Norcia, Jonathan Berger
Temporal reliability of cortical responses, indexed by inter-subject correlation (ISC), has been shown to predict audience engagement with narrative works. Here we present results from two EEG-ISC experiments assessing listener engagement with naturalistic music. Intact musical excerpts produce more consistent cortical components and higher ISC than phase-scrambled controls. Moreover, ISC analyses can also be applied to physiological and continuous behavioral responses. In sum, results suggest that EEG-ISC may be a promising approach to studying responses to full-length musical works in a single-listen experimental paradigm.
The Music Engagement Research Initiative
Blair Kaneshiro, Jonathan Berger
The Music Engagement Research Initiative is a multidisciplinary research group at CCRMA headed by professor Jonathan Berger. Through various approaches -- including laboratory studies, analysis of large-scale industrial data, and development of open-source research software and datasets -- we explore how and why people engage with music. In this talk we will present an overview of the group and selected current research projects. More information is available at https://ccrma.stanford.edu/groups/meri/
Large-Scale Music Discovery Behavior: Effects of Genre and Geography
Blair Kaneshiro, Lewis Kaneshiro, Casey W. Baker, Jonathan Berger
Music discovery has become a prevalent pastime, yet generalizations of this behavior remain elusive. Here we investigate music discovery on a large scale using data from the audio identification service Shazam. We aggregated geo-tagged United States Shazam queries corresponding to Billboard Top-20 Pop, Country, and R&B/Hip-Hop songs from October 2014 to March 2015. Query locations were labeled by Nielsen Designated Market Areas (DMAs). We visualize chart performance and Shazam query volume over time, as well as examples of geotemporal dynamics of Shazam queries; impact of special events on query volume; and the relationship between query volume and Billboard chart position.
The SpHEAR Project, a family of 3D surround microphone arrays
The *SpHEAR (Spherical Harmonics Ear) project is an evolving family of 3D printed, GNU Public License/Community Commons licensed soundfield microphones that include hardware designs for the microphone array and interface electronics, and all the software needed to perform an automated and accurate calibration of the finished microphone array. Ambisonics surround recording and processing technology, which the SpHEAR family embodies, is now widely recognized as being a perfect fit for the requirements of the exploding field of Virtual Reality. Come and look at the current finished prototypes and listen to examples of real recordings.
Classification of Auditory Stimuli Using Auditory Evoked Potentials
Steven Losorelli, Blair Kaneshiro, Gabriella Musacchia, Matthew Fitzgerald, Nikolas Blevins
An open question in cochlear implant research is whether users receive auditory signals needed to discriminate sound, but have not yet learned to interpret them; or whether they do not receive the auditory cues necessary to discriminate certain acoustical features. With children and difficult cases, this makes the implant fitting process challenging for audiologists, potentially limiting patient performance. Patients may therefore benefit from an objective method of auditory discrimination.
In this study, we demonstrate that classification of scalp-recorded auditory evoked potentials can provide an objective measure of hearing discrimination in normal hearing individuals. EEG classification performed on single-trial cortical responses and group-averaged envelope frequency following responses (FFR) recorded at the auditory brainstem yielded statistically significant above-chance classification accuracy.
Representation of Musical Beat in Scalp Recorded EEG Responses: A Comparison of Spatial Filtering Techniques
Steven Losorelli, Blair Kaneshiro, Jonathan Berger
Humans have the innate ability to perceive an underlying beat in complex musical signals. Despite the ease and speed with which our brains are able to accomplish this task, the neural mechanisms underlying the cognitive processing of beat and meter are unknown, and computational extraction of beat and other acoustical features from audio remain open topics of research.
Steady-state evoked potentials (SS-EPs) are natural brain responses to visual or auditory information at specific frequencies. In the realm of music, recent studies have shown that neural entrainment to beat and meter can manifest through a steady-state evoked potential captured in scalp-recorded EEG data.
Here we extend this research and take a spatial-filtering approach to analyzing EEG responses to musical stimuli with a steady beat. Spatial filtering techniques derive linear weightings of electrodes according to specific criteria. The present analysis focuses on two such techniques: Principal Components Analysis (PCA), which maximizes variance, and Reliable Components Analysis (RCA), which maximizes mutual correlation. Using these techniques, it may thus be possible to consolidate beat- related cortical activity into lower-dimensional subspaces of the data.
SmartKeyboard iOS and Android App Generator
Romain Michon, Julius O. Smith, Yann Orlarey, Chris Chafe
SmartKeyboard is a tool to generate musical Android and iOS applications using the Faust programming language. Standard Faust UI elements are replaced by a highly customizable interface that can be used to capture a wide range of gestures on a touch-screen. Polyphony MIDI support as well as built-in sensor control can also be easily added to the generated apps.
Passively Augmenting Mobile Devices Towards Hybrid Musical Instrument Design
Romain Michon, Julius O. Smith, Matthew Wright, Chris Chafe, John Granzow, Ge Wang
Mobile devices constitute a generic platform to make standalone musical instruments for live performance. However, they were not designed for such use and have multiple limitations when compared to other types of instruments. We present a framework to quickly design and prototype passive mobile device augmentations to leverage existing features of the device for the end goal of mobile musical instruments.
faust2api: a Comprehensive API Generator for Android and iOS
Romain Michon, Julius Smith, Chris Chafe, Stéphane Letz, Yann Orlarey
Faust2api is a tool to generate custom DSP engines for Android and iOS using the Faust programming language. Faust DSP objects can easily be turned into MIDI-controllable polyphonic synthesizers or audio effects with built-in sensors support, etc. The various elements of the DSP engine can be accessed through a high-level API, made uniform across platforms and languages.
Faust Physical Modeling Library and ToolBox
Romain Michon, Sara Martin, Yann Orlarey, Julius O. Smith, and Chris Chafe
We present a series of tools to simplify the design of physical models of musical instruments in the Faust programming language: the Faust physical modeling library, stl2faust, and imp2faust. The Faust physical modeling library allows to construct modular multidimensional diagrams with bidirectional connections to implement various types of physical models. It also contains a wide range of pre-made elements (e.g., strings, resonators, tubes, etc.) that can be used to build virtual musical instruments at a high level. stl2faust is a tool to convert STL CAD files into Faust modal physical models compatible with the Faust physical modeling library. imp2faust turns an impulse response into a Faust modal physical models compatible with the Faust physical modeling library.
Romain Michon’s Menagerie for the Petting Zoo
A wide range of musical instruments and controllers: the BladeAxe, the PlateAxe, the ModAxe, Nuance, and the Chanforgnophone.
Demo - pettable instruments.
Duet for One Pianist, Eight Sketches for midi piano and computer by Jean-Claude Risset
In 1989, composer and researcher Jean-Claude Risset‘s series of interactive sketches for piano and Disklavier entitled Duet for One Pianist explored the performative possibilities made available to pianists through the augmentation of emotive human musical gesture with the precise reactive and computational capabilities afforded by computer-based musical systems.
The composer explored simple compositional relations between the pianist's part and the computer's part: translations or symmetries in the time-frequency space (pitch transpositions or interval inversions); triggering by the pianist of specific patterns (arpeggios) or stored musical sequences influenced by certain performance parameters (tempo or loudness of the notes played); canon-like imitation. (https://chrysinanou.wordpress.com/portfolio/duet-for-one-pianist)
Performance or Musical Offering.
Verovio Humdrum Viewer
Craig Stuart Sapp
Verovio Humdrum Viewer (http://verovio.humdrum.org) is an online open-source music notation editor that is under development for digital editions of music, such as the Josquin Research Project (http://josquin.stanford.edu) and an edition of Chopin's music at The Fryderyk Chopin Institute (http://en.chopin.nifc.pl/institute) in Warsaw, Poland. The editor dynamically displays notation for musical data in the Humdrum format (http://www.humdrum.org), as well as importing MusicXML files and exporting MEI files (http://www.music-encoding.org). Animated playback of scores with MIDI and audio recordings will be demonstrated. Here is a sample set of Mozart piano sonatas being prepared with the editor: http://verovio.humdrum.org/?file=mozart/sonatas .
Resent JOS Research at CCRMA
Julius O. Smith, Jonathan Abel, Ericka Amborn, Chris Chafe, Weinong Chen, Zoran Cvetkovic, Enzo De Sena, Craig Doolittle, Ross Dunkel, Huseyin Hacihabiboglu, Christoph Hohnerlein, James Johnston, Hyung-Suk Kim, Esteban Maestre, Sara Martin, Aidan Meacham, Romain Michon, Vaibhuv Nangia, Michael Olsen, Julian Parker, Nick Porcaro, Maximilian Rest, Jordan Rudess, Lauri Savioja, Pat Scandalis, Gary Scavone, Jan Slechta, Harrison F. Smith, Peter Svensson, Vesa Valimaki, Ge Wang, Kurt Werner, and Matthew Wright
These posters will briefly summarize the following research publications on which I collaborated over the past year:
- ``Closed Form Fractional Integration and Differentiation via Real Exponentially Spaced Pole-Zero Pairs''
- ``More Than 50 Years of Artificial Reverberation''
- ``Digital Waveguide Network Reverberation in Non-Convex Rectilinear Spaces''
- ``A hybrid method combining the edge source integral equation and the boundary element method for scattering problems''
- ``A general and explicit formulation for wave digital filters with multiple/multiport nonlinearities and complicated topologies''
- ``The Fender Bassman 5F6-A Family of Preamplifier Circuits---A Wave Digital Filter Case Study,''
- ``Nuance: Adding Multi-Touch Force Detection to the iPad''
- ``Continuous Order Polygonal Waveform Synthesis''
- ``Design of Recursive Digital Filters in Parallel Form by Linearly Constrained Pole Optimization''
- ``Experimental Modeling of Bridge Admittance and Body Radiativity for Efficient Synthesis of String Sound by Digital Waveguides''
- ``Perceptually-motivated Spatial Audio Recording, Simulation, and Rendering''
- ``Development of Digital Filter Methodology for Hopkinson Bar Dispersion Correction''
- ``Real Time Performable Instruments on iPad and iPhone''
Lab full of posters.
FaucK!! Hybridizing the Faust and ChucK Audio Programming Languages
Ge Wang, Romain Michon
FaucK is a hybrid audio programming environment which combines the powerful, succinct Functional AUdio STream (Faust) language with the strongly-timed ChucK audio programming language. FaucK allows programmers to on-the-fly evaluate Faust code directly from ChucK code and control Faust signal processors using ChucK’s sample-precise timing and concurrency mechanisms. The goal is to create an amalgam that plays to the strengths of each language, giving rise to new possibilities for rapid prototyping, interaction design and controller mapping, pedagogy, and new ways of working with both Faust and ChucK. We present our motivations, approach, implementation, and preliminary evaluation.
Integrated all-frequency physics-based sound synthesis for physics-based animation
Jui-Hsien Wang, Doug James
In this project, we aim to build a general finite-difference time-domain wave solver for high-quality sound synthesis for physics-based animation. We will introduce novel algorithms used in our system to support animated objects with various sound source models, including linear modal model, acceleration sound (clicks) model, and water bubble sound model. We will discuss how to optimize the integration of these models onto our system, and challenges in mapping the current system onto state-of-the-art parallel computer architecture, such as GPUs.
Effects of Musical Roles on Coordinated Timing Asymmetries in Piano Duet Performance
Aury Washburn, Matt Wright, Takako Fujioka
Recent work in human behavioral dynamics has begun to identify conditions under which functionally asymmetric behavioral interaction between individuals arises and results in sustainable coordinative patterns, which allow for the successful completion of joint goals. Prior investigation of the temporal relationships between members of a string quartet has demonstrated that even though performers within this kind of ensemble already have distinct roles that are generally related in a hierarchical fashion, different patterns of following and mutual adaptation occur at different timescales during performance. In order to better understand and model the evolution of such asymmetrical behavior during successful ensemble performance, the current study examined the influence of asymmetries in factors such as the complexity, and pitch range of the two musical parts on the collective temporal stability and relative adaptability exhibited by pianists during duet performance. Preliminary results indicate that temporal asynchronies, as measured at each point where temporal synchrony would be expected based on the musical score, are quantitatively greater when the performers’ parts included some degree of asymmetry to begin with.
Feedback Network Ensemble
Matthew Wright, Alex Chechile, Julie Herndon, Mark Hertensteiner, Christopher Jette, Justin Yang
A structured musical improvisation in which an ensemble of players "plug our instruments into each other" to excite and control a sparsely-connected feedback delay network. Each player's instrument receives an audio input as well as generating an audio output, allowing to connect all of them to a matrix mixer so we can effect changes in the graph topology and adapt to them in realtime. See https://nime2015.lsu.edu/proceedings/329/index.html
Live Musical Performance.