GuitarFace

Goals

To identify virtuosity in rock guitar solos by analyzing MIDI/Guitar Pro data with supervised learning (SVM).

Motivation

As guitar players ourselves, we can appreciate a masterful guitar solo. Fast, dynamic solos comprise difficulty to some extent, but there's also "extra-musical" things like stage presence, emotion, and cool.

Solving the difference between good and bad guitar solos is a monstrous task, but we believe that there are a number of MVPs in the same vein that would serve guitarists and other musicians well. A tool that tells you if you're playing on the beat, for example, is very useful for solo guitarists and other "bedroom musicians" who don't have that kind of feedback from a band, or perhaps can't even tell until playback. Personal experience with lining up tracks in a DAW points to a need for excellent rhythm when recording and overdubbing. Other enhancements to the soloing experience could include knowing how close your sound is to Jimmy Page's, or transcribing a live improvisation of a song you want to remember later, or fine tuning your intonation and technique.

We choose rock music because of the spotlight that the guitar is frequently given in a rock band. Rock music is one of the largest genres of music, but there are individual artists who stand out as titans in the genre. Led Zeppelin, The Doors, and Jimi Hendrix are just a few examples of bands that truly convey masterfulness in the genre. Solos like these will make up our data set of "good" solos; we have yet to identify our data set of "bad" solos, but in general, the more extreme bad/good, the better.

Now, which features to include?

To give a quick example, rock and roll has 4 core instruments: the electric guitar, bass, drums, and voice. We could compare it to other genres on a basis of instrumentation, and find some differentiation. For example, jazz would have a noisier distribution of instruments: perhaps a peak over drums and the trumpet, but would we see the same over the piano and guitar?

Similarly, there are other features besides instrumentation that are core to rock, and furthermore rock guitar soloing:

• scale / key / intonation
• timing
• dynamic range (loudness)
• pitch range
• repetitiveness
• non-pitched decorations
• vibrato, bend
• stage presence (physical movements, facial expression)
• use of pedals / FX
• music theory of the context, expectation (build-up and violation)
• creativity

All of these features, when performed well or if they fall flat, can completely make or break a song. Violating these are clear violations of expectation. Naturally, we are interested in the measure of quality: what makes a good guitar solo? By "good", we mean retains our interest, impresses, and even inspires. We want to minimize the amount of subjectivity in our definition of quality, and we can do this by choosing features that we feel best contribute to "quality". 'The pitch content (melody) of the solo is a convincing example, knowing what we do about major and minor keys and other scales. Timing is another: in general, things should fall on the beat or integer divisions thereof.

User

Musicians, specifically guitar players, and more specifically improvisers / soloists. This could and should be able to be scaled to any instrument capable of producing MIDI output.

The Product

The product will use a support vector machine (SVM) to input MIDI data (tablature -> Guitar Pro -> MIDI) and feature vectors and output a classification. The product should have a few modes: practice and test, for example. During practice, one could see the raw data, and evaluate things in the feature space, such as timing, moments of vibrato, intonation, and more. At this stage, they could also scrub their data and extract a musical score.

Test would output a health meter as the guitarist is playing, i.e., the software is analyzing the solo in real time. This idea points to a gaming context, where 2 friends could duel and see who comes out on top, over a range of different categories (who has better technique? timing? pitch ranges/jumps? etc.).

Libraries and Pre-existing Code

• Liblinear SVM (C++): http://www.csie.ntu.edu.tw/~cjlin/liblinear/
• HMMLib (C++): http://www.cs.au.dk/~asand/?page_id=152
• Tablature -> MIDI: http://www.guitar-pro.com/en/index.php
• JGuido (C++) graphical rendering of raw MIDI -> score: http://csl.sony.fr/downloads/papers/2013/fober-13a.pdf

Relevant papers

• "Machine Learning Techniques for Real-time Improvisational Solo Trading": http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=2CC1E4C5C751E6D66E54855B2995F797?doi=10.1.1.129.8045&rep=rep1&type=pdf
• "Using Machine-Learning Methods for Musical Style Modeling": http://musicweb.ucsd.edu/~sdubnov/Papers/CM.pdf

Sensors / Accessories

• Roland G3 MIDI guiar pickup: http://www.rolandus.com/products/details/651
• Computer camera to detect guitar face?
• Accelerometer / Game-Track to track hip gyration?

Team

• Roshan Vidyashankar
• Gina Collecchia

Milestones

• Week 1: data acquisition and machine learning research
• Week 2: feature vector design, data scrubbing
• Week 3: App architecture, UI/UX
• Week 4: Code code code

Scratchwork

Ultimately, we hope to implement our model in the form of a game. It makes sense that actually playing the game should inform us further, since our players will be providing quality data. The game would provide feedback for your soloing, by rewarding good solos and punishing bad ones. If played by 2 people, this could send messages from a partner's superior solo to the opponents in the form of damage or further musical obstacles. The opponents could be dueling asynchronously, or essentially jamming together.