Music 256 Final Project

ThreeDee : 3D Sound Processing Environment

Project Vision

• Realize 3-D sound processing in real-time system using measured HRTFs
• Localizing a sound source in 3-D graphical environment
• Furthermore, provided as a tool for creative musical works or fun

Binaural Synthesis using measure HRTFs

• Transform a monural sound source into binaural sound
• Use Head-Related Transfer functions
• Measured HRTFs: azimuth resolution = 15 degree, elevation resolution = 10 between -40 and 40 degree.
• Interaural Time Difference (ITD) and position-dependent filters for left and right

Specification

• Minimal essential system
• Audio
• play wave files in loop
• time-varying delay for position-dependent delay
• overlap-and-add convolution by FFT for position-dependent filtering


• Visual/Control
• localize a sound source under a user's control: azimuth and elevation
• visualize a sound source in 3-D graphics


• More features
• define dynamic movement of a sound source
• spatial cue / guide : a sphere grid or 3-D bitmap image, etc
• effecient filter design and implmentaion
• control listener's viewing angle

Milestone

• tabularize measure HRTF data into C-format
• get binuaral processing work for given directions : time-varying delay and FFT
• design several types of moving visual objects to produce positional info
• add more dynamic movements of a sound source
• finish visualization : background and direction information, etc

Software Design

• Binaural class
  • Time-varying delay: DelayL in STK
  • overlap-add FFT: FFTW library
  • interpolation: ITD and HRTF data table
  
  
• SoundObject class
• include x, y, z postion and distance, azimuth and elevation, also conversion between them
• define dynamic modes and calculate position every audio frame

Control / Demo

• Mode
• User control (f key) : azimuth (j, k key), elevation(i, m key)
• Rotation (c key) : elevation(i, m key)
• Mosquito (r key) : moves randomly
• Comet (e key) : special fun mode


• Other keyboard functions
• 1 - 9 keys : adjust speed


• Demos

Future work

• Add distance control and more dynamic modes
• Flexible and intuitive control interface: joystick and OSC
• Support multiple sound sources
• Efficient filter implementation using frequency warping, Prony's method, etc
• Improve spatial cues with image files