Difference between revisions of "Sound Explorer"
From CCRMA Wiki
| Line 6: | Line 6: | ||
== Motivation == | == Motivation == | ||
| − | During my studies at CCRMA I'm exposed to many new audio / DSP concepts. Sometimes, however, it's hard to get a good intuition as to how things sound. | + | During my studies at CCRMA I'm exposed to many new audio / DSP concepts. Sometimes, however, it's hard to get a good intuition as to how things sound. It would be useful to have a tool that allows me to shape waveforms in various ways and hear the results. Also, from my experience using sndpeek and assignment 3, there's a strong motivation to make the tool real-time since that helps make the correlation between the manipulation and the resulting sound most obvious. |
== Product Description == | == Product Description == | ||
Sound Explorer allows the user to interactively construct and shape sets of waveforms. The results are displayed using a waterfall plot and the audio played back in real time. The ways in which the waveform can be shaped are: | Sound Explorer allows the user to interactively construct and shape sets of waveforms. The results are displayed using a waterfall plot and the audio played back in real time. The ways in which the waveform can be shaped are: | ||
* Frequency domain: | * Frequency domain: | ||
| − | :* Generate a harmonic series starting at a given frequency | + | :* Generate a harmonic series starting at a given frequency. |
::* Control the amount of non-harmonicity (i.e. how much the partials deviate from multiples of the the base frequency). | ::* Control the amount of non-harmonicity (i.e. how much the partials deviate from multiples of the the base frequency). | ||
| − | :* Generate white noise | + | :* Generate white noise. |
| − | :* Draw a spectral envelope | + | :* Draw a spectral envelope. |
* Time domain: | * Time domain: | ||
:* Draw a time-domain envelope | :* Draw a time-domain envelope | ||
| Line 23: | Line 23: | ||
'''* Interface:''' | '''* Interface:''' | ||
:The interface is made out of three elements: graphical display, keyboard input, and mouse input. | :The interface is made out of three elements: graphical display, keyboard input, and mouse input. | ||
| − | :* Graphical display | + | :* Graphical display: |
:: The screen is divided into three parts: | :: The screen is divided into three parts: | ||
::* Waterfall display: a waterfall plot of the audio currently being played | ::* Waterfall display: a waterfall plot of the audio currently being played | ||
| Line 33: | Line 33: | ||
'''* Software''' | '''* Software''' | ||
:* The program will use OpenGL for graphics, RtAudio for audio, and FFT routines from the Chuck. | :* The program will use OpenGL for graphics, RtAudio for audio, and FFT routines from the Chuck. | ||
| − | :* I will attempt to construct the program using the model / view / controller design patten. The model, for example, contains the samples for the current (and next) set of waveforms, the current envelope values, and the range and domain(s) to which the envelope(s) is applied. | + | :* I will attempt to construct the program using the model / view / controller design patten. The model, for example, contains the samples for the current (and next) set of waveforms, the current envelope values, and the range and domain(s) to which the envelope(s) is applied. I'm not sure though how easy it will be to separate the controller from the display. |
| − | * Real Time interaction | + | '''* Real Time interaction''' |
** The end goal is to have the user's interactions reflected in audio and graphics immediately. Initially, however, there may be two steps involved: 1) edit the wave:form 2) apply the changes and hear/see them. | ** The end goal is to have the user's interactions reflected in audio and graphics immediately. Initially, however, there may be two steps involved: 1) edit the wave:form 2) apply the changes and hear/see them. | ||
Revision as of 18:47, 11 November 2009
Contents
Sound Explorer
Idea / Premise
Sound Explorer is an environment for exploring and shaping sounds in real time.
Motivation
During my studies at CCRMA I'm exposed to many new audio / DSP concepts. Sometimes, however, it's hard to get a good intuition as to how things sound. It would be useful to have a tool that allows me to shape waveforms in various ways and hear the results. Also, from my experience using sndpeek and assignment 3, there's a strong motivation to make the tool real-time since that helps make the correlation between the manipulation and the resulting sound most obvious.
Product Description
Sound Explorer allows the user to interactively construct and shape sets of waveforms. The results are displayed using a waterfall plot and the audio played back in real time. The ways in which the waveform can be shaped are:
- Frequency domain:
- Generate a harmonic series starting at a given frequency.
- Control the amount of non-harmonicity (i.e. how much the partials deviate from multiples of the the base frequency).
- Generate white noise.
- Draw a spectral envelope.
- Time domain:
- Draw a time-domain envelope
In addition, for most of the above shaping methods, there will be a way to control which part of the waveform to apply them to. For example, it will be possible to apply a time-domain envelope to a subset frequencies.
Design
* Interface:
- The interface is made out of three elements: graphical display, keyboard input, and mouse input.
- Graphical display:
- The screen is divided into three parts:
- Waterfall display: a waterfall plot of the audio currently being played
- Edit window: this is where the user manipulates the waveform. At any time, this window is either in "additive mode" or "envelope mode". Each of those modes be in the frequency or time domain.
- Apply-to window: in this window, the user highlights which portion of the audio to apply the edit to. The window can be in the time or frequency domain.
- Mouse input: is used to draw envelopes and select ranges (in the apply-to window).
- Keyboard input: used to control modes and various parameters.
* Software
- The program will use OpenGL for graphics, RtAudio for audio, and FFT routines from the Chuck.
- I will attempt to construct the program using the model / view / controller design patten. The model, for example, contains the samples for the current (and next) set of waveforms, the current envelope values, and the range and domain(s) to which the envelope(s) is applied. I'm not sure though how easy it will be to separate the controller from the display.
* Real Time interaction
- The end goal is to have the user's interactions reflected in audio and graphics immediately. Initially, however, there may be two steps involved: 1) edit the wave:form 2) apply the changes and hear/see them.
Testing
The software will be tested by letting a user try it out and evaluate the:
- flexibility / expressiveness
- usability
- sound quality
- can interesting sounds be created?
Team
Roy Fejgin
Milestones
- 11/16/09:
- Waterfall window; "Time domain edit" window; audio rendering.
- 11/23/09:
- Apply-to window
- 12/07/09:
- Frequency domain processing:
- "Frequency-domain edit" window
- More sophisticated DSP: overlap add
- Add harmonic series
