Overview

Jnana” is a generative musical accompaniment system integrated into Ableton Live. It has the ability to analyze MIDI input and generate new material in a similar style. It can analyze input in real-time or from desired clips within Ableton and can populate Ableton clips with new material.

Jnana dynamically learns a musical style based on input melodies (or just rhythms) and uses this model to generate unique material in a similar style. Jnana is currently in the form of two plugins, “Jnana Live” and “Jnana Clips”. “Jnana Live” can listen to a real-time MIDI input and generate “response” melodies on demand or automatically when the input stops momentarily. “Jnana Clips” has the ability to analyze preexisting clips in Ableton and populate other clips with generated content. Jnana will never delete any notes in your Ableton Live session.

Here is a demo:

Demo of Jnana v0.1

Jnana is free and open source.

github.com/colinsullivan/Jnana

Contents

Installation and Setup

To run Jnana within Ableton Live, you’ll need the following software:

For information on installing on OS X, see the setup guide: ccrma.stanford.edu/~colinsul/projects/jnana/osx-setup-guide/.

Installation on Windows will be very similar, but I have not tried. Feel free to contact me if you wish to contribute a setup guide for Windows.

Functionality

Jnana comes in the form of two plug-ins.

Jnana Live

The Jnana Live plug-in will take live input from the Ableton track and integrate it into the analysis each time a phrase has “ended”. The end of a phrase is determined simply by noticing when the input stops for a given amount of time.

When an input phrase has completed, the plug-in has the ability to auto-generate a “response” to this phrase based on all the phrases seen thus far.

Screenshot of the Jnana Live plug-in within Ableton.

Screenshot of the clip setup for a track using the Jnana Live plug-in. The `-auto` clip will be used when Jnana automatically responds, and the two numbered clips will be used when generating for an arbitrary duration.

Jnana Clips

The Jnana Clips plug-in has the ability to analyze preexisting clips within an Ableton Live session and generate new ones based on this analysis.

Screenshot of the Jnana Clips plug-in within Ableton.

The controls for Jnana Clips are much more simple than in Jnana Live.

Screenshot of the clip setup for a track using the Jnana Clip plug-in. The auto clips will be populated based on an analysis of the numbered clips.

Common functionality

Both plug-ins have the same options for tweaking the analysis: use starting statistics and assume circular. These options can be useful to enable or disable in various scenarios. They can be disabled or enabled at any time without losing any information, and will only affect phrases generated in the future.

Both Jnana Clips and Jnana Live have these common controls

Troubleshooting

Integrating JavaScript, Max/MSP, the Max for Live API, and Ableton has presented significant technical challenges. I have worked out many of the problems, but I suspect there will be more. Please use the following discussion group for any issues you encounter, or to tell me how much you love / hate my software:

groups.google.com/forum/?fromgroups#!forum/jnanaapp

Deficiencies and Future

For the internal analysis, the system assumes the input is monophonic. If two notes are played simultaneously, it assumes they were played individually. The system does not understand chords, and the output of the system will be monophonic.

Some tweaking of the analysis could be desired. This is why I implemented the “Assume Circular” and “Use Starting Analysis” controls because they may sound better in different situations. I think once these situations become apparent, the controls can be re-labeled to something like “Improv Mode vs. Echo Mode” for example.