An interactive interface for predominant pitch extraction, and its applications in singing evaluation, source separation and cover-version generation.
Dr. Vishweshwara Rao,
SensiBol Audio Technologies
Abstract
A graphical interface for predominant pitch extraction from polyphonic music is described and demoed for various music clips including Indian classical and western pop. The high-resolution pitch contour of the predominant instrument is useful for applications such as evaluation of singing proficiency, predominant source separation, and cover-version creation. For the former, a mid-level melodic information layer consisting of notes, modulation such as vibrato, glides and rapid note transitions, is first generated. Perceptual thresholds and pitch-curve stylization are applied for evaluating singing proficiency of another singer using the original singer's melody as the benchmark. The predominant pitch is also used as an input to a source separation algorithm which suppresses and isolates the original singer's voice from the background accompaniment using a combination of sinusoidal modeling and binary masks. This algorithm was submitted to the MIREX 2014 Singing voice separation task. Suppression of the original singer's voice allows cover version creation by adding another singer's voice intelligently to the background music. Finally, an interactive singing platform, which uses all of the above signal processing solutions, and also vocal effects, is demonstrated.
Dr. Vishweshwara Rao received his Masters in Music Engineering from the University of Miami in 2004 and Ph. D. from the Digital Audio Processing Lab, Department of Electrical Engineering, at IIT Bombay in 2011. Dr. Rao is now the CEO of a technology start-up - SensiBol Audio Technologies - which successfully licensed and commercialized his Ph. D. research (among other audio processing technology). SensiBol recently won the award for Best Startup under the TIDE scheme 2014, from the Minister of MCIT, Govt. of India, and was also identified by the Economic times as one of the hot 15 start-ups for 2015. When not occupied in his start-up (which is rare), Dr. Rao is an avid singer and Tabla player.