My research

  • My research project here in CCRMA have been advised by Dr. Julius Orion Smith III (jos@ccrma.stanford.edu), Associate Professor of CCRMA/Music Department.

    A System for Timbre Modeling in Real Time

    1) The doctoral research

    This research is part of my EE PhD course at the Electric Engineering Faculty of Campinas University, Brazil. It is entitled: "SMT - Sistema de Modelamento de Timbres," or: Timbre Modeling System. This project is preceeded by the work developed in my EE MS, entitled: "Transformacoes Sonoras Atraves de Operacoes Timbrais", or: Sound Transformations by Timbral Operations, where a real time DSP system was projected to perform sound transformations based on the Fast Fourier Transformation, known as FFT. SMT utilizes the Short-Time FFT, or STFT, to achieve Timbre Manipulations of any sound in real time, whithout necessarely changing other sound features such as loudness and pitch. Therefore SMT aim to turn the Timbre Composition feasible.

    The architecture of SMT is based on the works of Xavier Serra and Julius Smith in Spectral Modeling. The software system developed by them called as: Spectral Modeling System, or SMS, splits a original input sound in its two components: Deterministic and Stochastic, that are separatedly edited. The output sound resulting from the SMS is the synthesis and mixing of the deterministic and stochastic components. SMT uses and develop this concept to speed up the processing time for a real time modeling. Instead of being a software that does sound transformations in off-line, SMS is a firmware system dedicated for real time timbre modelings.

    The SMS is compounded of two parts: a sincronous analysis-synthesis for both deterministic and stochastic parts; a assincronous part that apply operators over the deterministic and stochastic arrays. The sincronous part is in charge of the heavy real time processing involving the STFT computation, where frames not larger than 30 msec of sound are analysed and synthesized in FIFO mode (first in first out). The assincronous part does the timbre modeling through the modifications of the sound information b etween analysis and sythesis. This is done using a group of simple operators that will modify the frames of sound deterministic and/or stochastic. These operators are used to build more complex timbre modelings.

    The SMT was initially projected and simulated in the Matlab and in the Simulink. Now the project is being translated to the DSP56002. Using its evaluation module; DSP56002EVM, It's expected to separately simulate each branch (deterministic and stochastic) of the sound analisys-synthesis. After that it's planned to implement the whole system in a PCI DSP audio board, where the system will be fully operational and ready to be used in any PC with a Windows enviromental.