Rearrange Project
Sean Bratnober
MUS 220c
Spring 2007

main rearrange pic


Introduction

Operation

Rearrange File Format

Time Scaling

Project Status/Plans

Project Files/Installation


Introduction

For my project I built a (semi) interactive sequencer for Max/MSP.  My goal is to make a sequencer environment that bridges the arrangement paradigm of Max/MSP/Jitter (and possibly Pd, in the future) with the arrangment paradigm found in digital audio workstations (DAWs) like ProTools, Logic, and Ableton Live.  Essentially, I want to bring the functionality of a Music V type programming environment (like Max/MSP and Pd) to the realm of DAWs, to give composers and performers new flexible, interactive ways to bring algorithmic composition methods to the

I am designing and implementing an arrangement interface that expands the two dimensions of arrangment found in DAWs (horizontal axis == time, vertical axis == tracks/instruments) to three dimensions, where the third dimension represents every possible piece of music that can be peformed by a track at any point in time.  Unfortunately, the scope of this project was too big for me to be able to get that far this quarter, but I was able to implement a basic sequencer that can read and play a song with MIDI using a file format that I developed.  I called the sequencer Rearrange because I want it to be a platform for flexible musical arrangements, that can be arranged and rearranged in a real-time,

Most of the functionality of the patch is contained within the jit.rearrange~ external object which I programmed.  The user interface/control/display stuff was implemented with Max/MSP/Jitter objects.  The whole functionality of the patch essentially involves using these Max objects to send control messages to the
jit.rearrange external, which in turn sends control messages to display objects and MIDI output, and so on...

Operation

The operation of the sequencer is straightforward.  It opens specially formatted text files in the format I defined for the sequencer (explained 
here).  Files can be opened using the absolute path file open box or the relative path file open box in the patch.  To open a file with one of these, simply type the path you want in the appropriate box, and click the bang next to that box (make sure the text in the box has ONLY the text that you want, without any whitespace).  The relative file path is hardcoded in the first line of the function readfileRelativePath within the file rearrange_ui.js, which is inside the patches directory:


function readfileRelativePath(s)
{
    var f = new File("/Users/seanbratnober/CCRMA/220c/Rearrange/midi_files/" + s);

...

}


Simply change the line within quotes to the path where your song file (and its associated MIDI files, again explained
here) is located.  When this has been changed, you can simply enter the name of the song file in the relative path open file box, and it will open the fie within that path.  Make sure the absolute path is in the same format as the absolute path in the function above (ie, make sure that you have all the forward slashes you need, from beginning to end).

Once the song file is loaded, you're ready to play!  The operation here is very straightforward - at this point Rearrange is a really simple DAW type playback interface, without all the cool editing capabilities.  The Play/Pause button (it changes from "Play" to "Pause", depending on whether the sequencer is paused or playing.  You can move the timeline of the sequencer clock (the light blue line) to different parts of the song by moving the "Clock Time MS" number box.  This number box scales the time according to the number of milliseconds in one measure of a song in 4/4 time at 120 bpm (the time scaling is explained in more detail 
here).

The "Loop Begin" and "Loop End" number boxes set the start and end points of the loop, which are represented on the display by the pink lines.  The "Loop" checkbox turns looping on and off (turning looping on will make those pink lines brighter).  The looping works as you would expect looping to work in any standard DAW sequencer.

The "BPM" number box changes the BPM for playback, and the solo/mute buttons solo and mute the channels accordingly.

Rearrange File Format

This is the default Rearrange song file, "debug.txt" found in the midi_files folder in the Rearrange folder:

1 0 debug_piano4.txt 2000;
1 2000 debug_piano4.txt 2000;
1 4000 debug_piano4.txt 2000;
1 6000 debug_piano4.txt 2000;
1 8000 debug_piano4.txt 2000;
1 10000 debug_piano4.txt 2000;
1 12000 debug_piano4.txt 2000;
1 14000 debug_piano4.txt 2000;
1 16000 debug_piano4.txt 2000;
1 18000 debug_piano4.txt 2000;
1 20000 debug_piano4.txt 2000;
1 22000 debug_piano4.txt 2000;
1 24000 debug_piano4.txt 2000;
1 26000 debug_piano4.txt 2000;
1 28000 debug_piano4.txt 2000;
1 30000 debug_piano4.txt 2000;
2 8000 debug_piano1.txt 2000;
2 10000 debug_piano1.txt 2000;
2 12000 debug_piano1.txt 2000;
2 14000 debug_piano1.txt 2000;
2 16000 debug_piano1.txt 2000;
2 18000 debug_piano1.txt 2000;
2 20000 debug_piano1.txt 2000;
2 22000 debug_piano1.txt 2000;
2 24000 debug_piano1.txt 2000;
2 26000 debug_piano1.txt 2000;
2 28000 debug_piano1.txt 2000;
2 30000 debug_piano1.txt 2000;
3 16000 debug_piano2.txt 2000;
3 18000 debug_piano2.txt 2000;
3 20000 debug_piano2.txt 2000;
3 22000 debug_piano2.txt 2000;
3 24000 debug_piano2.txt 2000;
3 26000 debug_piano2.txt 2000;
3 28000 debug_piano2.txt 2000;
3 30000 debug_piano2.txt 2000;
4 24000 debug_piano3.txt 2000;
4 26000 debug_piano3.txt 2000;
4 28000 debug_piano3.txt 2000;
4 30000 debug_piano3.txt 2000;


Each line of this file has this format:

<channel> <start_time_ms> <midi_file> <midi_file_length_ms>;

Essentially, the file format specifies midi files that must be played at each channel in different points of time.  When these files are read, Rearrange also reads every midi file and stores the event lists from each file into a hash table (this table is used to save memory if users want to use multiple copies of a midi file in a song).  These midi files are specially formatted text files that are obtained from using midi files with the coll object in Max.  This object can be used to read midi files, and it allows users the option to save these midi files as text files.  In order to get your midi files to work with rearrange, you must make sure they fit this format.  You must also make sure that they reside in the same directory as the Rearrange song file.

The time scaling for the file described
here.  According to this scaling, a midi_file at start_time_ms == 0 would occur at the first measure, while a midi_file at start_time_ms == 2000 would start at the second measure.  A length of 2000 means that the midi_file is one measure long.

The data structure is set up so as a system of multiple doubly-linked lists.  Each line in the rearrange file represents a data object known as an arrangement in the Rearrange program.  Each arrangement has a pointer to the midi event list (doubly-linked) associated with it.  Rearrange builds ordered arrangement lists for each channel, and Rearrange song files are played with iterators that move through each arrangement list for each channel.  Each channel iterator also has an event list iterator that moves through the midi events over time.  The doubly-linked lists are used so that the clock can be moved backwards in time if needed (when this happends, the iterators just move back to the appropriate point in time in all of the respective lists).

For a future iteration of Rearrange, this file format/data structure will be extended so that multiple midi files can be set for any channel at any point in time.  This would be implemented so that each of these midi files is associated with a rule system, where the decision to play a particular file would be based on whatever rule is satisfied at that point in time.  This would allow users to perform interactively with Rearrange, while giving them the ability to change the midi file that gets played at any point in time.  It would also allow users ways to control song performances algorithmically with external patches and externals in Max.


Time Scaling

At this point, Rearrange only plays songs in 4/4 time, but the tempo can be adjusted accurately.

Originally I tried to implement the timing of this sequencer from the Max event clock (ie, using stuff like the "tempo" and "metro" objects), but the timing is not rock-solid, especially with big, nasty patches like mine (which was much bigger and nastier before I gave up on the Max event clock and let my external do all the timing/event scheduling).  Here is the method I used to get rock-solid timing.

To implement my own event scheduling clock, I decided to center it around the number of milliseconds per measure (2000) in a 4/4 song at 120 BPM (the default tempo of the sequencer).  My event clock is set up so that it will tick 2000 times for every measure, regardles of the BPM.  I decided to use the clock for the digital-to-analog converter (at a default sampling rate of 44100 Hz) to drive this event clock, because it is accurate, stable, and reliable.  I set up an algorithm that determines the number of DAC ticks required to tick the event clock.  This formula is used to obtain that value, where BPM is an adjustable variable:

sample_rate_clock_ticks_per_event_clock_tick = 44100 / ( 1000*(BPM / 120) )

With BPM = 120, the number of sample rate ticks need for each event clock tick is 44.1.

The jit.rearrange external is set up to work like an MSP object, so it can use the ticks at the sampling rate to drive the event clock.  The event clock ticking algorithm counts each sample rate tick, and when this count surpasses the 
sample_rate_clock_ticks_per_event_clock_tick value, the event clock is ticked, and the sample tick counter is reset as follows:

new_sampling_rate_tick_count =
sample_rate_clock_ticks_per_event_clock_tick - current_sampling_rate_tick_count

When playing a song at 120 BPM, the first event clock tick will occur when the sample rate tick count reaches 45.  This formula will set
new_sampling_rate_tick_count to -0.9.  This formula is used to prevent the timing from drifting, because it allows us to eliminate the effects of the clock overflow that takes place when ticking a discrete clock and comparing it against floating point values (ie, we can't tick the event clock at 44 sample rate clock ticks, but we can tick it when it reaches 45, although it is technically later.  Without the above formula, this "lateness" of the event clock ticks would cause the timing accuracy to drift over time.


Project Status/Plans

-Make Rearrange more like a full-functioning, DAW environment with editing capabilities.

-Expand connectivity to Max/MSP, to allow Max/MSP objects to have greater interaction with Rearrange.

-Implement rule based arrangement selection in real and non-real time.

-Implement the 3D arrangement perspective described above.

Project Files/Installation

In order to build/run this external/patch, Max/MSP/Jitter is required.

Rearrange.zip
Contains all the Max/MSP/Jitter patches needed for Rearrange, as well as a folder containing the "debug.txt" Rearrange song file, along with its associated midi text files.  This folder can be placed and used anywhere on your computer, but you must make sure to hard code the absolute file path as described 
here.  To run Rearrange, simply go to the Rearrange/patches directory and open the Rearrange patch.

rearrange~.zip
Contains the source (an Xcode project) for building the jit.rearrange~ external.  Right now, I can only guarantee support for Max/MSP 4.6 and Jitter 1.6, running on an Intel Mac, although I believe that intrepid users should have little trouble porting this to Windows.  I'm not as sure about support on earlier versions of Max/MSP/Jitter, but it seems possible.