Ravi

Brief

This is the wiki page for Ravi Parikh and Keegan Poppen's Music 256A final project, Fall 09-10.

Introduction

We wish to extend assignment 2 in order to create a MIDI-controlled vocoder/harmonizer/pitch correction software. The user will be able to play MIDI notes and sing into a mic simultaneously, and the output will be audio that is either pitch corrected or vocoded to the MIDI notes being played, depending on the mode. There will be a GUI to control parameters.

Motivation

Neither of us are very good singers, and in raw form, our voices are one instrument that we can't use in compositions. Software already exists that vocodes and auto-tunes voices, but we want to have a greater understanding of how this software works at the lowest level. This way, we'll have as much control as possible on how our voices can be processed. Our goal is not to create an Antares clone; rather, we want to cultivate our own sound and use this in future musical creations.

Software Architecture

Our project will be built in C++, leverage the RtAudio and RtMidi libraries in order to give us (significantly more) pain-free real-time audio and midi. Generally speaking, there will be a basic class hierarchy such that we can abstract out the various means of pitch detection and pitch modification that we will be experimenting with throughout the course of the project. In terms of development, this structure allows us to work iteratively on different parts of the system (once there is one in place) so that we can improve and/or add features to the final application. This structure also allows us to more easily implement the interface, as we will be able to change only what we need to change in order to create all the different interface elements necessary for the final project.

The interface for interacting with the application (to allow real-time control of the various parameters in the application) will be implemented in OpenGL.

The control process will essentially be:

application owns: 1 MIDI/audio track object

user selects track object type (vocoder, harmonizer, etc)

audio/MIDI is handled by callback function in the track class

instance variables in track class are set via OpenGL interface

User Experience

Fundamentally, this begins as an extension of assignment 2. Then, we will be able to implement a vocoder track-type and a harmonizer/pitch correction track-type. These types of tracks both take two inputs simultaneously (MIDI and audio). We will remove the multi-track functionality, since this is an instrument and not a DAW. Then, we will put a basic GUI on top of this, which will be for real-time parameter control.

We will then have an extensible architecture on top of which we can place any sort of voice/midi track. Essentially, the ideal case scenario is to have a generic "track" type and subclass it with various track types such as "harmonizer" or "vocoder." Depending on what track type is selected, a different GUI should show and allow the user to adjust parameters. We will experiment with different pitch detection, pitch correction, and vocoding algorithms to allow for maximum flexibility and creative control over audio output.

As a result of this, the user will see a default setting when they launch: it will automatically be in Vocoder mode (or whatever the default is). They can immediately start singing/playing notes and getting output. They can select a different track mode via some selection control (Drop-down, radio button, etc). Then, they can adjust parameters in real-time via sliders, dials, etc. We will emulate some hardware-type controls into the app, such as dials, and some software-type dials when they make more sense. For example, a 2-d coordinate system where the user can set two interrelated parameters at once is the type of control that makes sense in software but would be unwieldy in hardware.