From CCRMA Wiki
Revision as of 05:20, 10 November 2009 by Shiweis (Talk | contribs) (Created page with '= Pictone = Music 256A Fall 2009 Project Proposal. Shiwei Song (shiweis) == Idea / Premise == Some of the traditional instruments that people are familiar with have been around …')

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search


Music 256A Fall 2009 Project Proposal. Shiwei Song (shiweis)

Idea / Premise

Some of the traditional instruments that people are familiar with have been around for hundreds of years. The advancements made in computing technology have enabled development of unconventional musical instruments that uses sensors such as accelerometer, track pad, and microphone as inputs. The goal of my project is to build an instrument where the inputs are pictures drawn with pen and paper. A simple hieroglyph music language will be created. The instrument takes in "sentences" written in this language and generates the corresponding music. The audience will be able to see whats drawn/written by the performer and perhaps a visualization of the music itself.


My initial inspiration came from the Birds on the Wires video. The creator converted a picture of birds on wires to musical notes and produced a short piece of music that is both simple and beautiful. I feel that creating sound from pictures/drawings has several advantages:

  • Most people can write or draw simple shapes. This system will hopefully allow anyone to try and make some music/noise (and have fun!).
  • The music language will be new so the audience will be constantly surprised or left wondering what the next output will be.
  • (At some point) The characters/pictures in the music language itself has some sort of meaning (perhaps when combined together) so there will be both visual and sonic messages conveyed.
  • "Live coding" by drawing is fairly unconventional and this itself may be an interesting experiment to perform or observe.

The Thing (What is it)

Although there are a lot of room to develop/explore, the goal of this project is to create a basic working system that consists of:

  • A simple music language that maps to sound and other controls (such as loops).
  • A program that can recognize music written in the simple language and synthesize sound/give some visual feedback.

The user will interact with the system by drawing "sentences" using the simple language (or use pre-drawn ones), hold it up to the camera to get digitalized, repeat. The system will then output the sound of the music it has digitalized so far, incorporating loops and perhaps several tracks. The audience will see what the performer has drawn through live feed from the camera and visualization of what is being played.


The project will consists of two parts: the music language and the software system.

Picture Music Language

This project will construct a simple form of the picture music language. The primary concern is to have a good mapping of the pictures to representations and also have it easily recognizable by the software (so perhaps sacrificing picture meaning/aesthetics for ease of recognition.

The language will control two aspect of the music:

  • The controls/flow of the sound produced. Such as indicating loops, tracks, speed, etc.
  • Actual notes/instruments and pauses (lack of notes).