Gesture Signal Processing and Music
From CCRMA Wiki
Gesture and Measuring Movement
- What is a gesture? Physical. Bodily: hands, face, posture. Non-verbal. What do gestures communicate? Is a gesture more like a button or a handle?
- What is a musical gesture?
Techniques for measuring human movement
- 2D image processing
- 3D Motion Capture using many cameras
- Goniometry: measuring joint angles with physically attached sensors
- Laetitia Sonami's Lady's Glove: http://www.youtube.com/watch?v=HYTrNOmSRSo&feature=player_embedded
- Electromyography: measures muscle activation by measuring electrical activity
- Pamela Z's controller (which I believe uses emg): http://www.pamelaz.com/VociMov.html
- With accelerometers attached to the body (e.g. wiimote, iphone)
Really basic physics
- Position: x
- Velocity: v = dx/dt
- Acceleration: a = dv/dt
- Proportional to force (Newton's 2nd law of motion: F = ma )
- Jerk: j = da/dt
Interlude #1: Open Sound Control
- Open Sound Control: http://opensoundcontrol.org/
- iPod Touch has 3 axis accelerometer
- We will use TouchOSC to transmit accelerometer data as OSC messages over wifi to Max/Pd
Interlude #2: Filtering of Audio Signals
- Can view any sound as being composed of sinusiodal waves at different frequencies
- A filter removes energy at only certain frequencies:
- A low-pass filter removes high freqs.
- A high-pass filter removes low freqs.
- A band-pass only lets certain middle frequencies through.