Automatic Accompaniment for Electric Guitar

Michael Horgan

Music 220c

Spring 2008

Click here for my 220D project, which is a composition using this software.

Weekly Log:

4/3/08

I have my narrowed my project idea down to two different extensions for the electric guitar. The first one is a patch, most likely written in pd, that will that automatically synthesize some sort of accompaniment for the electric guitar signal being fed into it. My goal is create something that will generate sound to both support and color/enhance the notes that the guitarist is actually playing.

My other idea is to use the guitar as a midi input instrument. This would involve both pitch and amplitude detection algorithms to map every monophonic event from the guitar to a particular midi note and velocity. I am aware that this technology already exists, but I believe most midi guitars use fret sensors rather than pitch detection.

My ultimate goal for both of these projects is to extract the human-instrument interaction components of the guitar (i.e. the physical gestures required to play the guitar) from the actual sound of the guitar. In this way, the guitar can be used as the interface to an infinite number of software instruments. This is of particular relevance to me since the guitar is the only instrument I can play well.

4/10/08

I have decided to go with the automatic accompaniment patch idea. The core of the patch will be a pitch detection algorithm, which I will use to synthesize new musical material that is logically (according to my personal musical logic) connected to what the guitarist is playing. I have also been looking into some wave-shaping techniques for guitar, which are described in one of Miller Puckette's papers on a similar electric guitar patch he has been developing (http://crca.ucsd.edu/~msp/Publications/pd07-reprint.dir/). For the next couple of weeks, however, I will be focusing on creating a new pd pitch detection object, in addition to the fiddle~ object.

4/17/08

After meeting with Gautham and Chris, I have decided to create a pitch detection object in pd for the well-known Yin algorithm (http://recherche.ircam.fr/equipes/pcm/cheveign/pss/2002_JASA_YIN.pdf). This week I will be disecting the paper to determine exactly what equations I will need to implement in C and how they will be optimized for real-time performance. Gautham has also sent me a matlab implementation of the algorithm, which I will use for guidance.

The core of the algorithm is a difference function based on the Autocorrelation Function (ACF). Essentially, the difference function measures the correlation between a guessed period length and a signal segment. Doing this for every period within a frequency range and choosing the largest period with the lowest difference coefficient yields an accurate estimation of the fundamental period of the signal. There are a number of error-correcting additions to this function that help reduce the error rate to below 1%.

4/25/08

This week I created my own implementation of the yin algorithm in matlab. The paper discusses the core difference function plus four additional error-reducing steps—a cumulative norm function, an absolute threshold, parabolic interpolation, and a phase-correction function. The error reduction for each step is approximately 0.5 – 1%, in other words, not terribly significant.

I have chosen to implement only the cumulative norm function. I would include the absolute threshold (which sets a threshold (typically 0.1) below which the maximum value from the difference function is chosen), but in my testing, the correct period length often did not correspond with this value, instead more often corresponding to the global minimum value of the difference function. I'm not sure why this is, but it may have something to do with the fact the the paper used only speech samples for testing, rather than music.

The parabolic interpolation step is helpful for interpolating between integer period lengths, but of all the steps, this step apparently gave the smallest improvement. Since I want my pd object to be as efficient as possible for real-time playback, I am trying to avoid adding additional processing time where possible.

The phase correction step involves choosing a cluster of blocks that overlap near the same starting point in order to provide multiple phase-shifted versions of the same block of frequencies. This step would also increase processing time significantly, so I have chosen not to include. Also, the paper does not go into the details of this step, so I am not confident of exactly how to implement it.

5/2/08

Gautham has directed me to the flext c++ library (http://grrrr.org/ext/flext/) for generating pd externs. It's quite a bit easier than doing the standard c implementation for externs, so I think I will definitely use it. This week I will be looking into the details of the flext library.

5/9/08

I began porting my matlab implementation to c++ this week. The extern will consist of four functions: the ACF implemented using fftw.h, the difference function, which can be derived from the ACF, the cumulative norm function, and the find_max_period function which searches for the index of the minimum value computed by the previous three functions. Dividing the sample rate by this index number yields the algorithm's best estimate of the fundamental frequency of the signal block.

The biggest difference between the matlab code and the c++ code is computing the fft. I still need to figure out how to link in the fftw.h library with the flext makefile. Computing the ACF in the frequency domain, rather than the time domain, is absolutely necessary for real-time execution. Since the processing time of the time-domain version of the ACF grows as n^2, while the fftw processing time grows as n*log(n), fftw is much faster.

5/16/08

I finally figured out how to link the fftw shared library with the makefile for my yin~.cpp extern. Now my yin object is working in pd. The main drawback is that since yin measures the period length of the fundamental frequency in samples, the block size determines the minimum frequency yin is capable of identifying. The paper also suggests using a block size twice the length of the minimum frequency you wish to search for. This means that at a sampling rate of 44.1 kHz, I will need to use a block size of 2048 samples to accurate measure frequencies above 44100/1024 = 43 Hz. Since the low E string of a guitar is about 82 Hz, this seems like a necessary compromise.

5/23/08

This week I have been working on finishing up my pd patch for the Open House concert next week. Since I now know how to use the flext library, I went ahead and creating a few more pd objects to help in the guitar patch.

I have made a collection of pitch-rounding objects for specific musical scales. Since my yin~ object is frequently a few Hz off in its estimates, it helps to be able to round the frequency estimate up or down to the closest pitch of the key I wish to play in. I have made e_major, e_minor, and e_gypsy objects, which rounds estimates to the pitches of e minor, e major, and the gypsy scale in e (the gypsy scale consists of a major scale with a flatted second and sixth).

I also needed some way of detecting when I pluck my guitar, so I first created a transient~ obect, based on the AC-2A codec's transient detection algorithm, which I learned about last quarter in Music 422. But for some reason, it wasn't functioning quite accurately enough, so I created a simplified threshold detector object. This way, whenever the input level rises and exceeds -30 dB, which I defined arbitrarily, the synthesizer envelopes are triggered. This seems to be more effective than the transient detector.

5/30/08

My final project is now complete. All in all, I am a little disappointed with the latency issue involving the large block sizes necessary for the yin~ object. But I guess there are no cheap ways to do real-time pitch estimation yet. There is about 50 milliseconds of delay between the guitar note and the synthesizers' response, which makes playing in rhythm pretty challenging. While this bit of software may not be ready for live performance, it can still be a useful compositional tool for recording music, which I will demonstrate for my 220D project in a couple of weeks.

For a more comprehensive discussion of the patch's usefulness as a composition tool, please visit my 220D web page.

Here are the links to the executables and source code for the pd externs I wrote: