Difference between revisions of "Sound Explorer"

From CCRMA Wiki
Jump to: navigation, search
Line 5: Line 5:
  
 
== Motivation ==
 
== Motivation ==
As I looked at the waterfall plots generated by sndpeek and previous assignments in this class, it was fascinating to see so much information about the audio being played in one glance. Correlating the audio with the visual display gave me insight into time and frequency domain properties of the audio. The purpose of the project is to allow the user to further explore properties of audio by allowing them to manipulate waveforms and listen to the results in real time.
+
 
 +
During my studies at CCRMA I'm exposed to many new audio / DSP concepts. Sometimes, however, it's hard to get a good intuition as to how things sound. I would like to have a tool that allows me to shape waveforms in various ways and hear the results. From my experience using sndpeek and assignment 3, there's a strong motivation to make the tool real-time since that helps make the correlation between the manipulation and the resulting sound most obvious.
  
 
== Product Description ==
 
== Product Description ==
Sound Explorer will allow the user to interactively construct and shape sets of waveforms. The results will be displayed on the waterfall display and the audio played back in real time. The ways in which the waveform can be shaped are:
+
Sound Explorer allows the user to interactively construct and shape sets of waveforms. The results are displayed using a waterfall plot and the audio played back in real time. The ways in which the waveform can be shaped are:
 
* Frequency domain:
 
* Frequency domain:
 
:* Generate a harmonic series starting at a given frequency
 
:* Generate a harmonic series starting at a given frequency
 
::* Control the amount of non-harmonicity (i.e. how much the partials deviate from multiples of the the base frequency).
 
::* Control the amount of non-harmonicity (i.e. how much the partials deviate from multiples of the the base frequency).
 
:* Generate white noise
 
:* Generate white noise
:* Draw and apply spectral envelope
+
:* Draw a spectral envelope
 
* Time domain:
 
* Time domain:
:* Draw and apply a time-domain envelope
+
:* Draw a time-domain envelope
 
+
In addition, for most of the above shaping methods, there will be a way to control which part of the waveform to apply them to. For example, the it will be possible to apply a time-domain envelope to a subset of the spectrum.
+
  
 +
In addition, for most of the above shaping methods, there will be a way to control which part of the waveform to apply them to. For example, it will be possible to apply a time-domain envelope to a subset frequencies.
  
 
== Design ==
 
== Design ==
* Interface:
+
'''* Interface:'''
 
:The interface is made out of three elements: graphical display, keyboard input, and mouse input.
 
:The interface is made out of three elements: graphical display, keyboard input, and mouse input.
 
:* Graphical display
 
:* Graphical display
:: The screen is divided into three parts
+
:: The screen is divided into three parts:
 
::* Waterfall display: a waterfall plot of the audio currently being played
 
::* Waterfall display: a waterfall plot of the audio currently being played
 
::* Edit window: this is where the user manipulates the waveform. At any time, this window is either in "additive mode" or "envelope mode". Each of those modes be in the frequency or time domain.
 
::* Edit window: this is where the user manipulates the waveform. At any time, this window is either in "additive mode" or "envelope mode". Each of those modes be in the frequency or time domain.
Line 31: Line 31:
 
:* Keyboard input: used to control modes and various parameters.
 
:* Keyboard input: used to control modes and various parameters.
  
* Software
+
'''* Software'''
 
:* The program will use OpenGL for graphics, RtAudio for audio, and FFT routines from the Chuck.
 
:* The program will use OpenGL for graphics, RtAudio for audio, and FFT routines from the Chuck.
:* I will attempt to construct the program using the model / view / controller design patten. The model, for example, will contain the current (and next) set of waveforms, the current envelope values, and the range and domain(s) to which the envelope(s) is applied.
+
:* I will attempt to construct the program using the model / view / controller design patten. The model, for example, contains the samples for the current (and next) set of waveforms, the current envelope values, and the range and domain(s) to which the envelope(s) is applied.
 
* Real Time interaction
 
* Real Time interaction
 
**  The end goal is to have the user's interactions reflected in audio and graphics immediately. Initially, however, there may be two steps involved: 1) edit the wave:form 2) apply the changes and hear/see them.
 
**  The end goal is to have the user's interactions reflected in audio and graphics immediately. Initially, however, there may be two steps involved: 1) edit the wave:form 2) apply the changes and hear/see them.
Line 40: Line 40:
 
The software will be tested by letting a user try it out and evaluate the:
 
The software will be tested by letting a user try it out and evaluate the:
 
* flexibility / expressiveness
 
* flexibility / expressiveness
 +
* usability
 
* sound quality
 
* sound quality
* sound-annoyingness level
+
* can interesting sounds be created?
  
 
== Team ==
 
== Team ==

Revision as of 18:43, 11 November 2009

Sound Explorer

Idea / Premise

Sound Explorer is an environment for exploring and shaping sounds in real time.

Motivation

During my studies at CCRMA I'm exposed to many new audio / DSP concepts. Sometimes, however, it's hard to get a good intuition as to how things sound. I would like to have a tool that allows me to shape waveforms in various ways and hear the results. From my experience using sndpeek and assignment 3, there's a strong motivation to make the tool real-time since that helps make the correlation between the manipulation and the resulting sound most obvious.

Product Description

Sound Explorer allows the user to interactively construct and shape sets of waveforms. The results are displayed using a waterfall plot and the audio played back in real time. The ways in which the waveform can be shaped are:

  • Frequency domain:
  • Generate a harmonic series starting at a given frequency
  • Control the amount of non-harmonicity (i.e. how much the partials deviate from multiples of the the base frequency).
  • Generate white noise
  • Draw a spectral envelope
  • Time domain:
  • Draw a time-domain envelope

In addition, for most of the above shaping methods, there will be a way to control which part of the waveform to apply them to. For example, it will be possible to apply a time-domain envelope to a subset frequencies.

Design

* Interface:

The interface is made out of three elements: graphical display, keyboard input, and mouse input.
  • Graphical display
The screen is divided into three parts:
  • Waterfall display: a waterfall plot of the audio currently being played
  • Edit window: this is where the user manipulates the waveform. At any time, this window is either in "additive mode" or "envelope mode". Each of those modes be in the frequency or time domain.
  • Apply-to window: in this window, the user highlights which portion of the audio to apply the edit to. The window can be in the time or frequency domain.
  • Mouse input: is used to draw envelopes and select ranges (in the apply-to window).
  • Keyboard input: used to control modes and various parameters.

* Software

  • The program will use OpenGL for graphics, RtAudio for audio, and FFT routines from the Chuck.
  • I will attempt to construct the program using the model / view / controller design patten. The model, for example, contains the samples for the current (and next) set of waveforms, the current envelope values, and the range and domain(s) to which the envelope(s) is applied.
  • Real Time interaction
    • The end goal is to have the user's interactions reflected in audio and graphics immediately. Initially, however, there may be two steps involved: 1) edit the wave:form 2) apply the changes and hear/see them.

Testing

The software will be tested by letting a user try it out and evaluate the:

  • flexibility / expressiveness
  • usability
  • sound quality
  • can interesting sounds be created?

Team

Roy Fejgin

Milestones

  • 11/16/09:
  • Waterfall window; "Time domain edit" window; audio rendering.
  • 11/23/09:
  • Apply-to window
  • 12/07/09:
  • Frequency domain processing:
  • "Frequency-domain edit" window
  • More sophisticated DSP: overlap add
  • Add harmonic series