Gesture Signal Processing and Music

From CCRMA Wiki
Revision as of 00:37, 12 October 2009 by Lukedahl (Talk | contribs) (Interlude #2: Filtering of Audio Signals)

Jump to: navigation, search

Gesture and Measuring Movement

  • What is a gesture? Physical. Bodily: hands, face, posture. Non-verbal. What do gestures communicate? Is a gesture more like a button or a handle?
  • What is a musical gesture?

Techniques for measuring human movement

Really basic physics

  • Position: x
  • Velocity: v = dx/dt
  • Acceleration: a = dv/dt
    • Proportional to force (Newton's 2nd law of motion: F = ma )
  • Jerk: j = da/dt

Interlude #1: Open Sound Control

Interlude #2: Filtering of Audio Signals

  • Can view any sound as being composed of sinusiodal waves at different frequencies
  • A filter removes energy at only certain frequencies:
    • A low-pass filter removes high freqs.
    • A high-pass filter removes low freqs.
    • A band-pass only lets certain middle frequencies through.
  • Is differentiation a low-pass or high-pass filter? What about integration?

Back to movement and music

Accelerometers: How to analyze data

  • How to distinguish orientation vs movement?
  • Can get jerk via difference (as approximation of differentiation)
  • How to get velocity from acceleration (or position from velocity)?
    • True integrator (will eventually overflow):
      • y = y_prev + x.
    • Leaky integrator as approximation of integration:
      • y = a*y_prev + (1-a)*x
  • Thresholding - detecting specific events
    • max: past. pd: threshold
  • Classification through training of Machine Learning algorithms

Mapping to Sound