Visualization of Sound & Sonification of Images

Painting Noise Sound

Zune Lee, CCRMA, Stanford University, 2003

An example image of Painting Noise Sound Program

This project is an approach to the integration of sound and image motivated from one of my artistic mottos, "Painting Sound and Playing Image”. The goal is to find reasonable mapping between sound and visual by connecting sound elements to visual elements, thereby achieving a new audio-visual composition. I mapped sound pitch to visual point in a two dimensional space.

Here is the Demo Video (mpeg/13.5M).

Those mapping between sound and image is based on these factors.

  How can pitch (point) be arranged on a 2D space?
    random order, ascending order, descending order, combined order on a X-Y plane
  How dense can the pitches (points) be placed on the 2D plane?
  How long (short, regular, combined) are the intervals among pitches (points)?
  Here are the example images of the mapping.

To implement it, I wrote a simple noise painting program with generating MIDI signals using MS Visual C++ 6.0, and a simple noise-generated PD patch. By dragging a mouse point on the canvas, a user can paint noise sound.

Here is a full description (html) on this project.


(C) 2003~2004 Zune Lee all right reserved.