Difference between revisions of "GuitarFace"

From CCRMA Wiki
Jump to: navigation, search
Line 1: Line 1:
 +
Welcome to the GuitarFace Wiki! Check out the Github repository for GuitarFace here: https://github.com/ginacollecchia/GuitarFace
 +
 +
Why is this project called "guitar face"? It started with a feeling:
 +
 
[[File:Brianmay_guitar_face.jpg]]
 
[[File:Brianmay_guitar_face.jpg]]
 +
 +
Oh hey there Brian May! I bet you're playing some sweet jams. Here are some other amazing guitar faces. Try to notice similarities between them.
  
  

Revision as of 17:03, 13 November 2013

Welcome to the GuitarFace Wiki! Check out the Github repository for GuitarFace here: https://github.com/ginacollecchia/GuitarFace

Why is this project called "guitar face"? It started with a feeling:

Brianmay guitar face.jpg

Oh hey there Brian May! I bet you're playing some sweet jams. Here are some other amazing guitar faces. Try to notice similarities between them.


Goals

  • To detect the facial features of the "guitar face" using OpenCV
  • To make a visualizer of musical MIDI data that records and rewards user input in real time

Motivation

Guitar faces are hilarious. Funny faces in general are a fairly unexplored territory of facial recognition, mostly existing in the domain of mood detection.

Extracting musical features, even from MIDI, is an intriguing domain for players who want more feedback about their playing.

Storyboard

MIDI data is input in real time from a MIDI guitar pedal unit, the Roland GR55. The data isn't perfect, but the software is designed specifically with guitarists in mind.

Similarly, there are other features besides instrumentation that are core to rock, and furthermore rock guitar soloing:

  • scale / key / intonation
  • timing
  • dynamic range (loudness)
  • pitch range
  • repetitiveness
  • non-pitched decorations
  • vibrato, bend
  • stage presence (physical movements, facial expression)
  • use of pedals / FX
  • music theory of the context, expectation (build-up and violation)
  • creativity

All of these features, when performed well or if they fall flat, can completely make or break a song. Violating these are clear violations of expectation. Naturally, we are interested in the measure of quality: what makes a good guitar solo? By "good", we mean retains our interest, impresses, and even inspires. We want to minimize the amount of subjectivity in our definition of quality, and we can do this by choosing features that we feel best contribute to "quality". 'The pitch content (melody) of the solo is a convincing example, knowing what we do about major and minor keys and other scales. Timing is another: in general, things should fall on the beat or integer divisions thereof.

User

Musicians, specifically guitar players, interested in knowing more about their practice sessions. This can be used by any instrument capable of producing MIDI output. The output can serve as a score, though we don't really think of it primarily that way; we're more interested in the statistics and progress one makes toward set goals in their practice time, such as the change in duration from session to session, or pitch and volume content.

The Product

The product will use a support vector machine (SVM) to input MIDI data (tablature -> Guitar Pro -> MIDI) and feature vectors and output a classification. The product should have a few modes: practice and test, for example. During practice, one could see the raw data, and evaluate things in the feature space, such as timing, moments of vibrato, intonation, and more. At this stage, they could also scrub their data and extract a musical score.

Test would output a health meter as the guitarist is playing, i.e., the software is analyzing the solo in real time. This idea points to a gaming context, where 2 friends could duel and see who comes out on top, over a range of different categories (who has better technique? timing? pitch ranges/jumps? etc.).

Libraries and Previous Work

Relevant papers

Sensors / Accessories

  • Godin MIDI guitar
  • Roland GR55 guitar MIDI interface / pedals
  • Computer camera to detect guitar face
  • Lighting for the face?
  • Accelerometer / Game-Track to track hip gyration?

Design

Testing

Team

  • Roshan Vidyashankar
  • Gina Collecchia

Milestones

  • Week 1: OpenCV compilation, research; most of MIDI code; graphics setup
  • Week 2: OpenCV feature detection progress; MIDI + graphics integration and design; make the thing work, basically
  • Week 3: UI/UX tweaks
  • Week 4: Code code code

Scratchwork