THE AUDIO SKETCHBOOK
Daniel Schlessinger
Music
256 Final Project
12/12/2008
Vision
Specification
Milestones
Implementation
Supporting
Documentation
Screenshot
Code
Have you ever gotten a great idea for a song, but lost it because you couldn't write it down or record it fast enough? And if you are like me and your ideas come in groove-melody pairs, the absence of a quick way to hear everything in tandem can make brainstorming a very frustrating process. The idea behind the Audio Sketchpad is to allow people to get their multi-track ideas from their head to recorded form as quickly as possible.
The Audio Sketchbook is a
combination of a looping pedal and a mulitrack recorder, without any
of those distracting extra features.
On any device with only one
mic, people should be able to:
Record a track immediately upon startup
Loop tracks and record new ones for multitrack playback
Quickly be able to A-B different combinations of looping tracks for comparison, as well as to see how certain ideas sound next to other ones
Save text notes and lyrics associated with various tracks
Save files for future use
Tracks should be like
sticky notes – you should be able to move them around. This
is important for brainstorming
SAMPLE USER INTERACTION
My personal workflow would
go something like this: I sing a small 2-bar vocal 'drum' line into
the recorder. Then, this loops indefinitely and I sing an 8-bar
baseline. Then, I sing a few melodies, perhaps with dummy lyrics,
over the drum and bass that is currently playing. Then, I duplicate
the grouping of tracks, mute the bassline, and sing another
bassline (perhaps for the next section) and try a different melody.
Then, I click back and forth from the first and second grouping, to
decide how well I like the ideas I've come up with. Then I type in
some lyric ideas. Repeat indefinitely.
MILESTONE 2: Record clips and have them show up as windows (and waveform drawn) COMPLETED 12/9/08
MILESTONE 3: Basic Grouping Functionality COMPLETED 12/11/08
MILESTONE 4: Adding mutexes to fix all the crashing.
MILESTONE 5: Saving/loading/working with files
MILESTONE 6: Adding text, both as extra windows as well as being able to add names to clips
MILESTONE 7: Redesign everything from the ground up. For some reason I always want to do this just about the time everything is working.
MILESTONE 8: Port to iphone or other portable device. This is where it would actually be useful.
Currently implementation is on a Windows platform. Here is a simplified class diagram:
This idea was born from
the many, many times I have lost what felt like a great song idea
because I could not get to a multitrack recorder quickly enough. I
have tried many things – pen and paper are invaluable for
certain types of writing, but for anything that involves groove,
nuance, or simply requires a couple of iterations of improvising to
refine, it is just not fast enough. And while a straight up
multitrack recorder on an iphone would certainly do the trick, as
long as we're reinventing the wheel I must say that what would be
ideal would be an extremely simplified, default-looping recorder.
And thus, the audio sketchpad was born. If I can actually get this
ported to more, er, portable devices, I am confident it would be a
huge boon to anyone who's writing process is similar to mine. Which
might be a population of one.
Usage of the software goes
something like this:
click 'r' to begin recording a track. Click spacebar to stop.
Mute a clip by clicking on the body of the clip window.
Move a clip in the normal way you move a window, put it wherever you want.
Select a clip by clicking on the top bar, and hit 'delete' to delete that clip.
Zoom in and out with 'z' and 'x'
The blue windows are groupings, which are basically just presets about which tracks to mute. To duplicate a grouping, click 'd'. Then click on a grouping to select it and deselect the others.
Some known bugs at current milestone:
At the moment, you can only record up to 20 seconds of audio. This obviously needs to be unhacked to allow for unlimited recording.
Threading bugs – every so often the program crashes occasionally when clicking on clips. This is because I was too lazy to figure out how to implement mutices (kind of ugly I know, but I'm convinced this is the plural of 'mutex') in windows, and there are a bunch of global lists which are accessed and altered by disparate threads.
Sometimes the audio does not line up exactly to where you intended it to. This seems related to the current speed of the processor. I need to look into a better way of synchronizing. Or rather, perhaps I should, actually implement synchronization, since now it is pretty much nonexistent. :)
This program currently uses about 65% of my CPU. This has got to be improved. Premature optimization may be the root of all evil, but I think it is no longer premature.
When you select an object, it really ought to deselect any others that are currently selected (unless, perhaps, you hold down the shift key). Since there is no undo function, it is waaaaay too easy to accidentally delete stuff.
Other design issues I need to tackle, and soon:
You know, in retrospect it was totally unnecessary for me to reinvent the concept of a 'window', calling it an SkObject. This was a big pain, and still doesn't work flawlessly. If/when I port this to a more portable platform, that's the first thing I will do. Or maybe sooner. It was good to go through the process, but it would also be nice to have some experience making guis in a more efficient way.
For one, I need to take a good, hard look at the data structures I'm using. Right now the names for the SkObjects (for use with picking) are assigned when they are created as the index into the vector containing all objects. This makes deleting them a big pain in the ass. I really should implement a map for use with the picking.
On a similar note, I think I need to simplify the model for clip groupings. Really, there ought to be a class called state, which just contains the status of every object, and a clip grouping could house a state, and there could be a current state, etc. This would simplify things.
Comments? Questions?
Email me at dannymo at stanford dot edu