# Gesture Signal Processing and Music

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

## Gesture and Measuring Movement

• What is a gesture? Physical. Bodily: hands, face, posture. Non-verbal. What do gestures communicate? Is a gesture more like a button or a handle?
• What is a musical gesture?

### Really basic physics

• Position: x
• Velocity: v = dx/dt
• Acceleration: a = dv/dt
• Proportional to force (Newton's 2nd law of motion: F = ma )
• Jerk: j = da/dt

## Interlude #2: Filtering of Audio Signals

• Can view any sound as being composed of sinusiodal waves at different frequencies
• A filter removes energy at only certain frequencies:
• A low-pass filter removes high freqs.
• A high-pass filter removes low freqs.
• A band-pass only lets certain middle frequencies through.
• Is differentiation a low-pass or high-pass filter? What about integration?

## Back to movement and music

### Accelerometers: How to analyze data

• How to distinguish orientation vs movement?
• Can get jerk via difference (as approximation of differentiation)
• How to get velocity from acceleration (or position from velocity)?
• True integrator (will eventually overflow):
`y = y_prev + x.`
• Leaky integrator as approximation of integration:
`y = a*y_prev + (1-a)*x`
• Thresholding - detecting specific events
• max: past. pd: threshold
• Classification through training of Machine Learning algorithms

### Mapping to Sound

• How do we create a musically expressive instrument from sensor data?
• We need to map the data from one domain to another appropriately:
• linear: to map [a b] to [c d] use
`z = (s-a)*(d-c)/(b-a) + c`
• exponential: more appropriate in many situations since perceptual qualities are often logarthmic.
• any arbitrary function
• max: scale, pd: ???
• How do we decide the best mapping? (i.e. How do we select what physical parameter controls which musical parameter?)
• this is an art, not a science!