Jump Around, Sound: Musical Geometry in 3 Dimensions
For Stanford University Music 220c 2006
John Stoecker
Project Log Screenshots Ray Tracing Walkthrough Code Links Impossible Extensions
Jump around is a project that treats sound as a passive object. Beams representing sound travel within a space, and we perceive the sound through a beam's interaction with other objects in the space. I wanted to create a game and an instrument fun to play, and I also wanted to learn how to integrate audio and graphics in a computer program.
The program allows the user to place objects (spheres, cylinders, etc.) in a cube world and shoot beams that bounce around in the world. As a beam crashes into the objects, different sounds are created depending on the crash position and object type. I mapped frequency, note duration, and a reverb filter to the x, y, and z coordinates, and different Stk instruments to the objects.
How To Play:
Run ShapeWriter to set up the world; enter the coordinates, shape, and instrument of each object.
Run direckdraw and wait for the world to render. After you see all your objects, click the mouse where you want your first beam to go.
To add another beam, press 'n'. The program will wait for you to shoot the beam by clicking the mouse. Press escape anytime to exit.
My programming work involved creating the world renderer, determining the (mostly accurate) geometry for the beam movement and interaction, and using the CCRMA Synthesis ToolKit to create an instrument. There exist 3D packages for the type of graphics I used, but I've been wanting to code a ray tracer for a while now, and this proved a great opportunity to do so. (See the ray tracing walkthrough for a background.) Although a tracer renders slowly, it turned out to be a good graphics choice for my program because much of what I had to code for the tracer I could reuse in the geometry of the beam movement.
Integrating graphics and sound was tough. I used Visual Studio 2003; Microsoft is very picky about project settings and installing libraries and C++ packages. I planned to use the GrWin graphics library and the flext external sound library (to link up with PureData) in the beginning, but GrWin was too slow for me and flext was a nightmare to set up. I switched to DirectX (a great Windows graphics library) and Stk. Stk may not have been the best choice; I needed efficiency in computing samples, and the Stk instruments in my program did not respond well to heavy real-time use.