Additive Synthesis Projects by Matt Wright

March, 2004

I wrote an additive synthesizer for SDIF data. It should work to synthesize the analysis results of the ICMC2000 Analysis/Synthesis Comparison panel session. The project resulted in an ICMC 2005 paper Open-source matlab tools for interpolation of SDIF sinusoidal synthesis parameters

SDIF in Matlab

The Analysis/Synthesis Team at IRCAM created an SDIF extension for Matlab which I used.

My first project with this, to make sure I understood how to use it, was to port my existing "spew-sdif" program (written in C) to Matlab: spewsdif.m

A Framework for Different Frequency/Phase/Amplitude Interpolation Schemes

Here's the code to my synthesizer: additive_synth.m

My code is designed to make it very easy to plug in different methods for interpolating sinusoidal track parameters between SDIF frames. (Of course Julius Smith has a web page about this.) My original goal was to test the audibility of the difference between linear frequency interpolation ignoring all but initial phase versus cubic phase interpolation that gets the phases and frequencies right at the synthesis frame boundaries.

Therefore there are two parts of my code:

• The general-purpose part:
  • reading SDIF sinusoidal track models
  • selecting a particular SDIF stream (or taking the first one it sees)
  • selecting a subset of the frames
  • lots of error checking
  • making sure every partial track begins and ends with zero amplitude
  • matching partial tracks from frame to frame (handling births and deaths, etc.)
  • an array of current phase values for all partial tracks
  • an array for output time-domain samples
  • dispatching to the appropriate synthesis subroutine
• The interpolation-method-specific part: A collection of synthesis "inner loop" subroutines.
• Given the starting and ending frequency, amplitude, and phase for each partial alive in this synthesis frame,
• Accumulate synthesized output into the output array.
• There's a different subroutine for each form of interpolation

The Synthesis To-Do List

These two parts of my code communicate with a data structure I call a "Synthesis To-Do List". This is a 2D matrix whose rows represent the partials to be synthesized in the current synthesis frame and with these columns:

1. Partial track index (to uniquely identify this sinusoidal track, specifically, an index for accessing this track's entry in the array of the current phase of each partial) Because of Matlab's 1-based array indexing, this is one more than the SDIF partial track index (which may be zero).
2. "Old" frequency (frequency at beginning of synthesis frame)
3. "New" frequency (frequency at end of synthesis frame)
4. "Old" amplitude
5. "New" amplitude
6. "Old" phase
7. "New" phase

• Starting sample number to synthesize
• Ending sample number to synthesize
• T (= 1/fs), the time sampling interval of the synthesis sampling rate

• output: the array of output samples
• phases: the array of phases

Analysis Frames and Synthesis Frames

SDIF is totally general about each frame having a float64 time to give the instant when its data applies. Generally the data in one frame of SDIF data comes from a windowed segment of original time-domain input signal, but by the time it comes through SDIF it's a matrix of sinusoidal track parameters that apply to a given instant. Therefore I say that each frame of SDIF input data represents one analysis frame .

Generally, a synthesizer must interpolate sinusoidal track parameters to avoid discontinuties. From this point of view, the frames of the SDIF file represent points to which smooth time-sampled curves must be fit to produce instantaneous values for each sinusoid. Therefore a synthesis frame is the time between two adjacent-time SDIF frames.

Analysis Frame: one SDIF frame. Synthesis Frame: two adjacent SDIF frames.

Note that my synthesis frames have starting and ending times in samples. (I round from the full-resoultuion float64 SDIF times to the nearest discrete sample time for the given synthesis sampling rate so that all synthesis frames are an integer number of samples long.)

The last sample of the Nth synthesis frame is the first sample of the N+1st synthesis frame, so by convention my synthesis subroutines write to output[fromSample+1:toSample].

This architecture also supports OLA methods such as IFFT.

Future Work

Eventually I'd like to implement every flavor of additive synthesis represented in the Analysis/Synthesis Comparison session:

• Linear interpolation of freq+amp between frames (SNDAN's pvan:) (done.)
• Cubic phase interpolation (Allows freq and phase continuity at each frame point SNDAN's mqan, IRCAM)
• Spline interpolation (PartialBench)
• Linear interpolation of amp on dB scale (SMS)
• Fudging frequency trajectory to meet a partial's stated instantaneous phase by the time of the next frame (IRCAM) (Insert new breakpoint halfway between two frames and use linear interpolation.)
• "Bandwidth enhancement" of partials (Loris)
• SMS noise filter coefficients
• SMS noise spectral magnitudes and phases
• IRCAM's spectral envelope of white noise
• Track-by-track temporal reassignment (Loris)
• Interpreting Modal Distribution time/frequency surface as other than sinusoidal tracks (???)

Please let me know if you find this to be useful, and please send me back any additions or improvements that you make.