Source Separation by Score Synthesis (ICMC 2010)
Download the paper here.
In this paper, we present a method based for source separation based on the PLCA algorithm, using synthesized music scores as prior information to guide the analysis and give meaning to the results. Basically, we use an approach in 2 stages. In a first stage, 'ideal' components are learned from the synthesized instruments separately using standard PLCA. We assume that these components resemble what is present in the real mix, so we use them as prior information in the second stage. There, we perform a PLCA decomposition on the mix, with all the previously learned components as prior distributions.
If we then want to resynthesize only a single instrument, we only need to resynthesize those components that were learned from its synthesized counterpart, since in the analysis of the mix, those components will almost certainly have converged to represent that instrument.
We tested our approach first on J.S. Bach's Air (BWV1068), a well known work for baroque chamber orchestra, that is in the recording we used performed by 2 violins, viola and cello. Note that the instruments are all spectrally relatively similar to each other, and play harmonically related notes, which makes a decomposition of the spectrogram an inherently difficult task due to similarities between the components and overlapping harmonics respectively.
For your convenience, you can also download all these files in a zip file . Notice there is some phasiness in the results, this is due to the method only decomposing the magnitude spectrogram. The phase is ignored for now, and is in every extracted source just copied from the mix. In future work we plan to work on this issue. Nevertheless, listening to only the first few seconds, we do get a decomposition of the chords, with the notes in the vast majority of cases assigned to the correct instruments.