Randal Leistikow, Ph.D.
I am currently managing AV development and MIR research at Smule, a company working to bring the world together through music.
- Probabilistic modeling of musically informed machine listeners
- Audio source separation
- Real-time collaborative music over IP networks
- Digital audio effects
- Virtual acoustic environments
"Bayesian Modeling of Musical Expectations via Maximum Entropy Stochastic Grammars"
When presented with musical sounds, humans take advantage of prior knowledge of
acoustic and musical context to accomplish an impressive array of cognitive listening
tasks, such as meter tracking, transcription, style classification, instrument identification,
harmonic analysis, and melody prediction. This dissertation presents a dynamic
Bayesian framework for modeling listeners with differing musical expectations.
Although a simulated listener with specific experience may simply be created by
learning prior distributions directly from a given musical corpus, a more interesting
approach is to construct a listener whose expectations are governed by rules of music
theory. Such rules are often expressed as statements involving musical tendencies,
e.g., "A large upward melodic interval is typically followed by a smaller downward
interval." This dissertation focuses on a novel method of transforming music-theoretic
rule sets into parameterized, maximum entropy rate distributions suitable for use in
dynamic Bayesian networks. Encoding rule-based expectations allows the system to
infer which rules are most responsible for predicting musical attributes at each time
in a piece, and to identify which rules are violated at points of musical surprise.
In addition to enabling a wide variety of interesting musical tasks to be performed
using symbolic data as input, our framework can also be integrated into compatible
probabilistic models that use recorded audio signals as input. The signal processing
layers encode acoustic expectations by modeling the spectrotemporal evolution of instrument
tones, and segment the signal into a sequence of note events. A system
in which signal and symbolic layers inform one another is desirable because musical
expectations can help the system compensate for corrupted signals, and the ability to
predict musical sequences suggests a future sequential Monte Carlo inference implementation
in which sampling distributions concentrate on the most likely transitions,
thereby avoiding the computational cost of evaluating all points in the potentially
vast space of possible transitions.
Download: PDF
- Thornburg, H., R. Leistikow, and J. Berger, "Melody extraction
and musical onset detection via probabilistic models of STFT peak data."
IEEE Transactions on Audio, Speech and Language Processing 15(4): 1257-1272,
May 2007
- Thornburg, H., D. Swaminathan, T. Ingalls, and R. Leistikow,
"Joint segmentation and temporal structure inference for partially-observed
event sequences." in Proc. 8th IEEE International Workshop on
Multimedia Signal Processing (MMSP-06), Victoria, BC, Canada.
- Leistikow, R., H. Thornburg, J.O. Smith III, and J. Berger,
"Bayesian Identification of Closely-Spaced Chords from Single-Frame STFT Peaks,"
in Proc. 7th Intl. Conference on Digital Audio Effects (DAFx'04),
Naples, Italy, October 2004.
- Thornburg, H. and R. Leistikow,
"A New Probabilistic Spectral Pitch Estimator: Exact and MCMC-Approximate Strategies,"
in Proc. Computer Music Modeling and Retrieval 2004 (CMMR 2004),
Esbjerg, Denmark, May 2004 (to be published by Springer Verlag as a book in
the Lecture Notes in Computer Science Series).
- Thornburg, H. and R. Leistikow,
"An Iterative Filterbank Approach for Extracting Sinusoidal Parameters From Quasi-Harmonic Sounds,"
in Proc. 2003 IEEE Workshop on Applications of Signal Processing to
Audio and Acoustics (WASPAA'03), New Paltz, New York, November 2003.
- Thornburg, H. and R. Leistikow,
"Analysis and Resynthesis of Quasi-Harmonic Sounds: an Iterative Filterbank Approach,"
in Proc. 6th Intl. Conference on Digital Audio Effects (DAFx'03),
London, September 2003.
- Chafe, C. and R. Leistikow,
"Levels of Temporal Resolution in Sonification of Network Performance,"
in Proc. 7th Intl. Conference on Auditory Display (ICAD 2001), Espoo, Finland,
July 2001.
- Chafe, C., S. Wilson, R. Leistikow, D. Chisholm, and G. Scavone,
"A Simplified Approach to High Quality Music and Sound over IP",
in Proc. COST-G6 Conference on Digital Audio Effects (DAFx-00), Verona, Italy, December
2000.