Overview

“Jnana” is a generative musical accompaniment system integrated into Ableton Live. It has the ability to analyze MIDI input and generate new material in a similar style. It can analyze input in real-time or from desired clips within Ableton and can populate Ableton clips with new material.

Jnana dynamically learns a musical style based on input melodies (or just rhythms) and uses this model to generate unique material in a similar style. Jnana is currently in the form of two plugins, “Jnana Live” and “Jnana Clips”. “Jnana Live” can listen to a real-time MIDI input and generate “response” melodies on demand or automatically when the input stops momentarily. “Jnana Clips” has the ability to analyze preexisting clips in Ableton and populate other clips with generated content. Jnana will never delete any notes in your Ableton Live session.

Here is a demo:

Contents

Installation and Setup

To run Jnana within Ableton Live, you’ll need the following software:

• Ableton Live 8.3.x
• Max for Live

For information on installing on OS X, see the setup guide: ccrma.stanford.edu/~colinsul/projects/jnana/osx-setup-guide/.

Installation on Windows will be very similar, but I have not tried. Feel free to contact me if you wish to contribute a setup guide for Windows.

Functionality

Jnana comes in the form of two plug-ins.

Jnana Live

The Jnana Live plug-in will take live input from the Ableton track and integrate it into the analysis each time a phrase has “ended”. The end of a phrase is determined simply by noticing when the input stops for a given amount of time.

When an input phrase has completed, the plug-in has the ability to auto-generate a “response” to this phrase based on all the phrases seen thus far.

• The input analysis section of the plug-in contains options and controls for how the analysis of the input MIDI stream is analyzed:
  • When auto analyze is enabled, the input stream will be analyzed and incorporated into the model so the system can learn your style. This must be enabled for the system to learn. Optionally, you can hold down the “hold to analyze” button which will toggle the analysis only when the button is held.
  • The phrase detection timeout determines how much silence is required for an input “phrase” to have occurred. This can be lengthened if you wish to incorporate long pauses into your phrases, or shortened if your input phrases will be very close together.
  • When a phrase is analyzed, it will be analyzed using the options under the how section, described below in Common functionality.
  • The reset training button will reset any learning that has occurred thus far.

• The accompaniment generation section contains controls for how the response will be generated:
  • When auto response is selected, the system will automatically respond when an input phrase is detected as described above. The clip on the track with a name ending in -auto will be populated with notes and triggered. See the Jnana Live Tutorial section of the setup guide for more information on required setup within Ableton Live.
  • Optionally, if the hold to generate button is held, the accompaniment will generate endlessly until released. In this case, the clips on the track with names ending in -manual1 and -manual2 will be populated on each play and triggered alternately. This requires their “follow mode” to be set properly in Ableton, see the Jnana Live Tutorial section of the setup guide for more information.
• The midi passthrough always toggle button enables MIDI to pass through the plug-in at all times. When disabled, MIDI will only pass through the plug-in when one of the generative clips is playing (i.e. a response is occurring).

Jnana Clips

The Jnana Clips plug-in has the ability to analyze preexisting clips within an Ableton Live session and generate new ones based on this analysis.

The controls for Jnana Clips are much more simple than in Jnana Live.

• The Clip analysis section contains controls for modifying how and when the clips in the track are analyzed:
  • In the when section, there is an analyze track now button which will analyze all of the clips with names ending in numbers like myriff-01 or myriff-5.
• Once the track is analyzed, the controls under Clip generation modify when any clips ending in -auto are generated with a new phrase.
  • The generate clips button simply forces all the -auto clips to generate immediately. Be careful about pressing this too often, or while the clips are playing.
  • The auto generate button allows the clips to self-generate when they are done playing, after being played the amount of times specified in the plays between generations box. For example, if plays between generations is set to 1 then each time one of the -auto clip plays, it will auto-generate. If set to 2, the generated riff will play twice before changing. It is best to set up the follow mode between two or more -auto clips to facilitate this behavior. This is probably the safest way to use this plugin. See the Jnana Clips Tutorial section of the setup guide for more information.

Common functionality

Both plug-ins have the same options for tweaking the analysis: use starting statistics and assume circular. These options can be useful to enable or disable in various scenarios. They can be disabled or enabled at any time without losing any information, and will only affect phrases generated in the future.

• Use starting statistics
  • When this is toggled, the start of each phrase in the analysis will be considered when generating new phrases.
• Assume circular
  • When this is toggled, the phrase will assume to loop from the end to the beginning. This is useful when analyzing loop-like melodies, or phrases that are meant to resolve back to the starting note.

Troubleshooting

Integrating JavaScript, Max/MSP, the Max for Live API, and Ableton has presented significant technical challenges. I have worked out many of the problems, but I suspect there will be more. Please use the following discussion group for any issues you encounter, or to tell me how much you love / hate my software:

groups.google.com/forum/?fromgroups#!forum/jnanaapp

Deficiencies and Future

For the internal analysis, the system assumes the input is monophonic. If two notes are played simultaneously, it assumes they were played individually. The system does not understand chords, and the output of the system will be monophonic.

Some tweaking of the analysis could be desired. This is why I implemented the “Assume Circular” and “Use Starting Analysis” controls because they may sound better in different situations. I think once these situations become apparent, the controls can be re-labeled to something like “Improv Mode vs. Echo Mode” for example.