The Short-Time Fourier Transform

The Short-Time Fourier Transform (STFT) [8,10,9] is a powerful general-purpose tool for audio signal processing. It defines a particularly useful class of time-frequency distributions [38] which specify complex amplitude versus time and frequency for any signal. We are primarily concerned here with tuning the STFT parameters for the following applications:

1. Approximating the time-frequency analysis performed by the ear for purposes of spectral display.
2. Measuring model parameters in a short-time spectrum.

In the first case, applications of audio spectral display go beyond merely looking at the spectrum. They also provide a basis for audio signal processing tasks intended to imitate human perception, such as auditory scene recognition [25,192] or automatic transcription of music [110].

Examples of the second case include estimating the decay-time-versus-frequency for vibrating strings [265] and body resonances [104], or measuring as precisely as possible the fundamental frequency of a periodic signal [92] based on tracking its many harmonics in the STFT [56].

An interesting example for which cases 1 and 2 normally coincide is pitch detection (case 1) and fundamental frequency estimation (case 2). Here, ``fundamental frequency'' is defined as the lowest frequency present in a series of harmonic overtones, while ``pitch'' is defined as the perceived fundamental frequency; perceived pitch can be measured, for example, by comparing to a harmonic reference tone such as a sawtooth waveform. (Thus, by definition, the pitch of a sawtooth waveform is its fundamental frequency.) When harmonics are stretched so that they become slightly inharmonic, pitch perception corresponds to a (possibly non-existent) compromise fundamental frequency, the harmonics of which ``best fit'' the most audible overtones in some sense. The topic of ``pitch detection'' in the signal processing literature is often really about fundamental frequency estimation [92], and this distinction is lost. This is not a problem for strictly periodic signals.