Making a Synthesizer Sing Like Pavarotti
The Wall Street Journal, Friday May 6, 1994; page B1
By Don Clark, staff reporter of The Wall Street Journal, PALO ALTO,
Calif. -- Meet Sheila, a soprano who exists only on a computer.
Her bell-like voice rings out from two stereo speakers, never
pausing to take a breath. Perry Cook, her creator, tweaks a mouse on his
workstation to widen an animated diagram of her vocal passages and modify
timbre or vibrato. Though she normally sings just vowels, the researcher
plays an eerie recording of Sheila jamming with some Greek folk
musicians.
Sheila is one of the many colorful creations of Karma, a Stanford
University laboratory where researchers are trying to develop the
ultimate musical synthesizer. Such a device would replicate not only the
precise sound of any instrument but also the unique style of any
instrumentalist or singer. Synthesizers, which in recent years have
dropped in price, have become a standard tool for many composers and
performing musicians, as well as nonprofessionals.
"Let's say we have a good model of Pavarotti after he's long gone,"
says John Chowning, Karma's director. "With certain permissions, a
[synthesized] singer could sing works that had not been composed in his
or her lifetime."
Karma, the familiar pronunciation of the Center for Computer
Research in Music and Acoustics, has transformed the music business
once -- with technology used in millions of personal-computer sound
boards -- and stands a chance of doing it again.
The lab, in a Spanish-style mansion overlooking the campus, has
developed a radically different approach to electronic synthesizers. The
software technology, know as "waveguides," mathematically recreates the
expressive signatures of instruments -- the flutter of a flute, a
saxophone's squawk or the scrape of bow on string.
Waveguides are already being used in a $4,995 Yamaha Corp.
synthesizer designed for playing solo passages. Karma researchers are
now trying to use waveguides to create a universal keyboard, one that can
emulate virtually any musical sound and provide equivalents to the subtle
action of instruments' keys.
In what was once a pantry, research assistant Brent Gillespie uses
motors and computers to emulate the action of a single piano key. He is
trying to develop a mechanism that can be adjusted to emulate the feel of
a clavichord, harpsichord or even a particular make of piano.
"We'll never have really good synthesizers until we can also
replicate the physical sensations that a good performer feels," Prof.
Chowning says.
Man Who Coined 'Transistor'
Karma's pioneers include John Pierce, the former head of Bell
Laboratories who developed the first communications satellite and coined
the word "transistor"; Max Mathews, widely regarded as the father of
computer music; and Prof. Chowning, a composer who studied electronic
music in Paris before coming to Stanford in 1963.
In 1967, Prof. Chowning discovered that at a certain frequency,
patterns of vibrations from two oscillating devices generated harmonic
tones that resembled those of musical instruments. The formula became
known as FM synthesis.
Yamaha took an exclusive license to Stanford's 1977 FM synthesis
patent and seven years later introduced the DX7 synthesizer, one of the
most popular musical instruments ever made. Yamaha's FM synthesis chips
have powered about 20 million personal-computer sound boards.
Prof. Chowning's patent ran out on April 19. But not before it
generated some $20 million in royalties to Stanford and Karma, which set
up an endowment that's still its main funding source.
Karma hopes waveguides will become just as important. They were
invented by Julius Smith III, a musician turned electrical engineer who
in the early 1980s became consumed with a big deficiency in
synthesizers -- the lack of a realistic violin sound.
A technique called sampling is one option. Sampling is based on
digital recordings of instrument sounds. But sampling lacks expression
since it is based on pre-existing recordings, musicians have limited
ability to modify those sounds in authentic-sounding ways when they are
playing them.
Famous Equation
Dr. Smith created a mathematical model of the physical action of
drawing a bow across a string. Jean Baptiste d'Alembert, a French
scientist of the 18th century, developed a famous equation describing the
way waves travel down the ideal string or inside a wind instrument. The
traveling-wave equation includes such variables as the tension and
density of a string and how hard it is struck.
Most attempts to use the equation require extensive measurements to
account for vibration losses as the wave moves down a string, a
gargantuan computing task. In 1986, Dr. Smith perfected a way to
simplify the process; it started by ignoring the losses and adds them
back later with a digital filtering device that uses a sample of the
original sound as a guide.
Yamaha's new VL1 synthesizer, which began appearing in stores in
February, shows that startling effect. Players can modify 128 waveguide
sounds using three wheels that control vibrato, pitch variation and the
force with which a note is blown or plucked. The VL1 also offers a more
unusual accessory, a special breath-control tube that allows players to
add subtle changes of expression blowing as they play notes on the
keyboard. On the flute setting, for example, a reduction in air pressure
causes the pitch to drop mournfully. Blowing too hard can cause an
instrument to jump an octave in pitch, or causes a growling flutter.
"It's pretty much changed my life," says Alan Pasqua, a veteran jazz
and rock keyboard player in Santa Fe, N.M., who mainly uses the VL1 as a
composing tool. "Now I have an instrument that really will express how I
hear a melody, just as if I hired one of the great saxophonists."
Waveguides also can be used to create hypothetical instruments. The
VL1 has settings such as a bowed flute, of the combination of a flute and
oboe, a trumpet and a sax.
But there are limitations. Because of the heavy computing involved,
the first-generation VL-1 can produce just two tones at a time. And its
acoustic strings are less impressive than its wind instruments and
electric guitars. That's partly because string instruments also generate
tones from vibrations on their wooden bodies, the next frontier for
waveguides.
Other companies have also licensed Dr. Smith's patent from Stanford,
which has been criticized for its exclusive FM license to a Japanese
company. Media Vision Technology Inc. expects to deliver PC sound boards
that combine waveguides, sampling and FM synthesis. Dr. Cook now works
for the Fremont, Calif., company, and Prof. Chowning is on its board.
Dr. Smith, meanwhile, is moving toward a goal of his research -- to
use waveguide simulations of actual performers, not just instruments. "I
know it's possible in principle," Dr. Smith says. "But it's never been
demonstrated."