This project is an adaptation of Xavier Serra's Spectral Modeling Synthesis (SMS) technique for compositional purposes. It provides an SMS sound composition environment integrating several tools: First, the sound is analyzed from a Unix shell using Serra's C programs. A graphical interface (SMSEditor, from an Objective C prototype by Serra) has been greatly enhanced in order to display the resulting files in a three-dimensional waterfall plot. After the analysis is done, several routines support reading and writing of SMS files from inside MatLab (cmex files) and the post-processing and normalization of these files. Once analysis and post-processing are done, a series of routines and instruments integrating Lisp, CLM (Bill Schottstaedt) and C, are used for the resynthesis of the sound. The resynthesis employs the Inverse FFT algorithm (Xavier Rodet) which Xavier Serra and I programmed in the '94 Summer Workshop at CCRMA. The resynthesis programs run in real-time.
The MusiCloth project is a study in the design and implementation of a performance environment for computer music that utilizes a graphical display along with a physical interface. The conceptual model for the MusiCloth is a large tapestry which the performer manipulates through large hand and arm motions and which produces MIDI output based on the performer's actions. Ultimately, the visual display should be implemented on a large, high-definition display so that the performer can stand before it, as if standing in front of a tapestry. The performer would then manipulate areas of the tapestry through hand and arm motions. This design would allow for both large, sweeping motions in addition to smaller, more precise control over smaller sections of the display. As a measure of flexibility, the graphical design of the display, along with its corresponding mappings to performance input and MIDI output, can be implemented using custom-designed overlays created by the composer. Currently, this project exists as a simple prototype to be run on a Power Macintosh G3. Eventually, however, the project should be ported to a large display system such as the Information Mural in the Stanford Computer Science department.
Samply Great, a standalone Windows application with a user-friendly graphic interface, is a track-based Sampling/Mixing programme with DSP features. Basic concepts of computer music, such as additive, subtractive and granular synthesis can be explored in a WYSIWYG manner.
The programme uses sound samples, envelopes for additive synthesis (which can be derived from the analysis of an existing sound), and noise as sound sources. Several effects, for instance volume changes, waveshaping or transposition can be applied to the whole score or each track, and also to each note of a track. The effects, as well as the sources, can be varied dynamically over the range of the score and/or each note.
All parameter curves/envelopes can be drawn with the mouse, providing an extremely intuitive working environment. If the computational load is not too great, the output can be heard in realtime (using the Windows Direct Sound API). An output file (WAVE format) is additionally created during each rendering process. The projects can be saved and loaded to and from disk. The option of exporting the whole project as ANSI C code provides the possibility of porting and compiling the project on a platform other than Windows, as well allowing post-processing and fine-tuning of the project.
Voice teachers/pedagogues usually lack an in-depth understanding of the concepts used to analyze the singing voice, a fact which is a considerable obstacle to efficiently putting them into practice. Singsing, a Windows application with a simple graphical user interface, provides basic tools to introduce a nevertheless profound analysis of the singing voice into the process of teaching.
For pitch detection and calculation of the residual signal, Singsing uses the programme Praat and its shell script (as developed by Paul Boersma - http://www.fon.hum.uva.nl/praat) as an underlying process. The programme offers the following features: Plots of Pitch Tier, Second Order Perturbation, average wavecycle and error signal, and time-varying spectral plots, as well as spectrogrammes of the input, the residual and the vibrato tier. To be developed is an estimation of the vocal track shape.
The analysis results of each sound file are automatically written or appended to an ASCII output file, which can then be imported into other applications to calculate statistics.
New releases (V5.0+) are now made by Leigh Smith of tomandandy and Stephen Brandon at the University of Glasgow, who are porting the Music Kit to OPENSTEP, Apple's MacOsX and MacOsX-Server, Windows98, and Linux. Latest releases and progress can be found at http://www.tomandandy.com/MusicKit.
The 4.2 version of the Music Kit was released in 1997 and is available free of charge via FTP at ftp://ccrma-ftp.stanford.edu/pub/NeXT/MusicKit/. This release is compatible with NEXTSTEP software releases 3.2 and later on NeXT and Intel-based hardware. Also, Music Kit programs that are compiled under NEXTSTEP can run on OPENSTEP for Intel and NeXT hardware.
Release 4.2 is an incremental release with several significant additions:
The 4.2 Music Kit includes a new driver and support for the Turtle Beach Fiji and Pinnacle DSP cards. These cards provide the best price/performance of any currently-available Music Kit-compatible DSP cards (as of July 1997). They have a DSP56002, 32K of fast static RAM, and both digital and high-quality analog I/O. The Pinnacle also has an MPU401-compatible Kurzweil synthesizer that will work with the Music Kit MIDI driver. In addition, the Music Kit driver for the Turtle Beach Multisound, Tahiti and Monterrey has been upgraded to support the full Turtle Beach DSP memory space.
The Intel implementation has been optimized. Support for writing soundfiles from the DSP is now supported on Intel hardware. This functionality was previously available only on NeXT hardware.
Two Music Kit applications of note are available separately:
There are a variety of new UnitGenerator classes. For example, rock-solid real-time envelopes are now available with AsympenvUG, which down-loads its envelope to the DSP, instead of feeding the break-points down one at a time (as does AsympUG.)
For more details on these items, as well as other new features, please see the Music Kit release notes, which are available via ftp at ftp://ccrma-ftp.stanford.edu/pub/NeXT/MusicKit/ReleaseNotes.rtf.
Other Music Kit News
Until recently, we were making extensive use of the ``Frankenstein'' cards (in various forms), home-brewed DSP cards based on the Motorola EVMs. However, with the advent of the Turtle Beach Fiji and Pinnacle cards, we no longer feel it is necessary (or worth the trouble) to pursue the ``Frankenstein'' direction.
We have been planning to provide a combined sound/MIDI driver for SoundBlaster-compatible cards. We negotiated with NeXT to do this (because we needed permission to use their sound driver code) and everything was ready to happen, but then there were some legal complications that held things up, so we weren't able to get this done for the 4.2 release.
Music Kit Background
The Music Kit is an object-oriented software system for building music, sound, signal processing, and MIDI applications in the NEXTSTEP programming environment. It has been used in such diverse commercial applications as music sequencers, computer games, and document processors. Professors and students have used the Music Kit in a host of areas, including music performance, scientific experiments, computer-aided instruction, and physical modeling. The Music Kit is the first to unify the MIDI and Music V paradigms, thus combining interaction with generality. (Music V, written by Max Mathews and others at Bell Labs three decades ago, was the first widely available ``computer music compiler.'')
The NeXT Music Kit was first demonstrated at the 1988 NeXT product introduction and was bundled in NeXT software releases 1.0 and 2.0. Since the NEXTSTEP 3.0 release, the Music Kit has been distributed by CCRMA. Questions regarding the Music Kit can be sent to musickit@ccrma.stanford.edu.
The CCRMA Music Kit and DSP Tools Distribution (or ``Music Kit'' for short) is a comprehensive package that includes on-line documentation, programming examples, utilities, applications and sample score documents. It also comes with Bug56 (black hardware only), a full featured, window-oriented, symbolic debugger by Ariel Corp. for the Motorola DSP5600x signal processing chip family.
The Music Kit is an object-oriented software system for building music, sound, signal processing, and MIDI applications in the NEXTSTEP programming environment. It has been used in such diverse commercial applications as music sequencers, computer games, and document processors. Professors and students have used the Music Kit in a host of areas, including music performance, scientific experiments, computer-aided instruction, and physical modeling. The Music Kit is the first to unify the MIDI and Music V paradigms, thus combining interaction with generality. (Music V, written by Max Mathews and others at Bell Labs three decades ago, was the first widely available ``computer music compiler.'')
The NeXT Music Kit was first demonstrated at the 1988 NeXT product introduction and was bundled in NeXT software releases 1.0 and 2.0. Since the NEXTSTEP 3.0 release, the Music Kit has been distributed by CCRMA. Questions regarding the Music Kit can be sent to musickit@ccrma.stanford.edu.
The CCRMA Music Kit and DSP Tools Distribution (or ``Music Kit'' for short) is a comprehensive package that includes on-line documentation, programming examples, utilities, applications and sample score documents. The package also comes with Bug56, a full featured, window-oriented, symbolic debugger by Ariel Corp. for the Motorola DSP5600x signal processing chip family.
Source code is available for everything except Bug56. (The low-level DSP and MIDI drivers are available only for NEXTSTEP-Intel.) This means researchers and developers may study the source or even customize the Music Kit and DSP Tools to suit their needs. Enhancements can be sent to musickit@ccrma.stanford.edu to have them considered for future CCRMA releases. Commercial NeXT software developers may freely incorporate and adapt the software to accelerate development of NEXTSTEP software products. (Free commercial use of files copyrighted by NeXT Inc. are restricted to NEXTSTEP platforms.)
People who answered the Music Kit survey sent around last year will notice that many of the most requested items on the survey have been included in the 4.0 release. Please send your future Music Kit requests to musickit@ccrma.stanford.edu. To subscribe to the Music Kit mailing list, send email to ``listproc@ccrma.Stanford.EDU''. The body of the message (not the Subject line) should contain the text ``subscribe mkdist <your name>'' (You don't type the '<' and '>'). To unsubscribe, send an email with ``unsubscribe mkdist'' in the body of the message.
See the Music Kit Release Notes for further details.
The Music Kit was designed by David Jaffe and Julius Smith, with input from James A. Moorer and Roger Dannenberg. The Objective-C portion of the Music Kit was written by David A. Jaffe, while the signal processing and synthesis portion was written by Julius Smith. The Ensemble application and much of the SynthPatch library were written by Michael McNabb. Douglas Fulton had primary responsibility for the documentation. Others who contributed to the project included Dana Massie, Lee Boynton, Greg Kellogg, Douglas Keislar, Michael Minnick, Perry Cook, John Strawn and Rob Poor.
Music Kit References
Highlights of the Music Kit 4.0 Release
Highlights of the Music Kit 4.1 Release
The Music Kit 4.1 release is essentially Release 4.0 plus support for NEXTSTEP 486/Pentium machines. It uses one or more plug-in DSP cards to support music synthesis and digital audio processing. MIDI is similarly provided by plug-in cards. The release is ``fat'' so there is only one package that works on both NeXT and Intel-processor computers.
For music synthesis and digital audio processing on Intel hardware, the 4.1 Music Kit provides drivers for three DSP sound cards, the Ariel PC-56D, the Turtle Beach Multisound and the i*link i56.
For MIDI on Intel hardware, the Music Kit provides a driver for MPU-401 cards (such as the MusicQuest family and the SoundBlaster-16), emulating the functionality of NeXT's MIDI driver, including synch to MIDI time code. Source to all the drivers is included in the Music Kit Source Package.
While only one DSP card is required, the power of a system can be scaled up by the use of multiple cards. An application built with the Music Kit can simultaneously use multiple DSP and MIDI cards by the same or different manufacturers, with details of DSP resource allocation handled automatically. In addition, the drivers provide automatic sensing so that applications can be moved between machines with different hardware configuration with no re-configuration necessary.
NeXT hardware has not been left behind. The Music Kit now supports the 192K DSP extension memory board (available from S.F.S.U.) with automatic sensing.
Other new features include a MusicKit panel for the Preferences application for setting various defaults and managing multiple DSP cards.
See the Music Kit 4.1 Announcement for further details regarding the supported DSP cards.
For further inquiries regarding the Music Kit or DSP tools, send email to musickit@ccrma.stanford.edu. To join the Music Kit email list, send a subscribe message to mkdist-request@ccrma.stanford.edu.
Capella is an object-oriented graphical interface for algorithmic composition in Common Music. It defines classes of browsers and worksheets that implement a consistent set of visualization tools and serve as a graphical front end for the system. The interface currently runs on the Macintosh under Macintosh Common Lisp.
Algorithmic composition is a complex activity in which both musical and technological issues must be addressed in parallel. This, in turn, places special requirements on a graphical interface that supports the process. Object-oriented composition environments such as Common Music (Taube 1994), DMix (Oppenheim 1993), and Mode (Pope 1992) place additional demands on graphical tools due to the breadth of representation and functionality that these kind of systems implement. Smalltalk environments are able to take advantage of a powerful windowing system provided by Smalltalk itself. Since Common Music was designed to be as portable as possible, without the aid of a native windowing system, almost no attempt to address visualization issues was made until recently. Until now, visual output in Common Music was completely text-based, similar to the type of display one sees when working, for example, in a Unix shell window. Common Music's command-line driven interpreter, Stella, connects to the system's toolbox similar to the manner in which a shell connects to Unix. Although it allows powerful input expressions to be formulated, Stella does not allow the inner processes to be easily understood. Capella is a response to some of the communication limitations in Stella, while keeping in mind that graphic representation and mouse based gestures are not always the best or most expedient models to choose for interacting with a complex system. Capella has been designed to be a complement, not a replacement, for the two other modes of interaction supported by the system: command processing from Stella and procedure invocation from Lisp. Common Music simply runs all three modes ``in parallel'' and the composer is free to choose whatever is most appropriate to a particular situation.
Capella is still in the early stages of development. Its primary goal is to allow a set of flexible visualization tools to be developed, but it also makes interacting with the system as a whole easier and more transparent. The need for transparency is particularly acute in algorithmic composition workshops, where participants must quickly absorb not just new theoretical concepts, but a specific implementation of them as well.
References
Mi_D is a multi-platform shared library that offers clients a simple and unified, yet unique set of MIDI services not commonly found in existing driver interfaces. Its main design goal was to allow clients to add sophisticated MIDI support to their applications at minimal cost.
See also the Mi_D Home Page at: http://ccrma-www.stanford.edu/CCRMA/Software/mi_d/doc/
Highly structured music composition systems such as Common Music raise the need for data visualization tools which are general and flexible enough to adapt seamlessly to the--at times very unique--criteria composers employ when working with musical data. These criteria typically involve multiple levels of data abstraction and interpretation. A ``passing note'', for instance, is a fairly complex, compound musical predicate, which is based on properties of several other, lower-level musical predicates such as the degree of consonance, metric position, or melodic direction, all of which are of different complexity, draw upon different primitives, and apply only to a limited set of data types, that is, ``notes''. Visualizing compound musical predicates then translates to a mapping of a set of criteria--predicates and properties--on a set of display parameters.
The SEE visualization tool provides graphical and programming
interfaces for these two tasks and consists of an abstracting program
layer to allow for the construction of custom musical predicates out
of a possibly heterogenous set of data and a separate program module
which controls their mapping onto a wide variety of display
parameters. As large screens and full color support become more and
more standard for most computer systems as well as to account for the
complexity that comes with visualizing musical predicates in general,
the display parameters make consequent use of both, color and the 3D
visualization paradigm. Thus, object position as well as extension along the 
, 
, and 
axes, object representation (model), and color (position of its color along
the coordinate axes of the current color model) may be assigned up to
ten or more predicates.
Although SEE may be used as a standalone tool, it is highly integrated and primarily intended to be used with Capella, Common Music's graphical user interface. The application framework itself and the programming interfaces are implemented in Common Lisp, and thus run on a variety of platforms.
The current version is being developed on a SGI workstation using the X11 windowing system and the OpenGL and OpenInventor graphics standards, but portability is highly desired and upcoming ports will most probably start out with the Apple Macintosh platform.
PadMaster is a a real-time performance / improvisation environment currently running under the NextStep operating system. The system primarily uses the Mathews/Boie Radio Drum as a three dimensional controller for interaction with the performer, although that is no longer the only option. The Radio Drum communicates with the computer through MIDI and sends x-y position and velocity information when either of the batons hits the surface of the drum. The Drum is also polled by the computer to determine the absolute position of the batons. This information is used to split the surface of the drum into up to 30 virtual pads of variable size, each one independently programmable to react in a specific way to a hit and to the position information stream of one or more axes of control. Pads can be grouped into Scenes and the screen of the computer displays the virtual surface and gives visual feedback to the performer. Performance Pads can control MIDI sequences, playback of soundfiles, algorithms and real time DSP synthesis. The velocity of the hits and the position information can be mapped to different parameters through transfer functions. Control Pads are used to trigger actions that globally affect the performance.
The architecture of the system has been opened and it is now possible to create interfaces to other MIDI controllers such as keyboards, pedals, percussion controllers, the Lightning controller and so on. More than one interface controller can be active at the same time listening to one or more MIDI streams and each one can map gestures to the triggering and control of virtual pads. The problem of how to map different simultaneous controllers to the same visible surface has not been completely resolved at the time of this writing (having just one controller makes it easy to get simple visual feedback of the result of the gestures, something that is essential in controlling an improvisation environment). Another interface that is being currently developed does not depend on MIDI and controls the system through a standard computer graphics tablet. The surface of the tablet behaves in virtually the same way as the surface of the Radio Drum, and tablets that have pressure sensitivity open the way to three dimensional continuous control similar to that of the Radio Drum (but of course not as flexible). The advantage of this interface is the fact that it does not use MIDI bandwidth and it relies on hardware that is standard and easy to get.
Performance Pads will have a new category: Algorithmic Pads. These pads can store algorithms that can be triggered and controlled by gestures of the performer. While a graphical programming interface has not yet been developed at the time of this writing, the composer can create algorithms easily by programming them in Objective C within the constraints of a built in set of classes and objects that should be enough for most musical purposes. Any parameter of an algorithm can be linked through a transfer function to the movement of one of the axes of control. Multiple algorithms can be active at the same time and can respond in different ways to the same control information making it easy to transform simple gestures into complicated musical responses. An algorithm can also be the source of control information that can be used by other algorithms to affect their behavior.
I collaborated with professor Jonathan Harvey for the sound design of his piece Ashes Dance Back, for choir and electronic sounds. This collaboration was four quarters long, covering fall/winter 95-96, and fall/winter 96-97. At the request of professor Harvey I used my ATS system (see ``Current Research Activites'' section) for spectral modeling to generate the electronic sounds of the piece based on the analysis and transformation of a single vocal sound: a B flat sample of my own singing.
During the composition of this piece, many improvements and additions were done to ATS. Here is a list of the most prominent ones:
The equalization, montage, and final mix of all the electronic materials was done using CLM. For the performance of the electronic sounds of the piece we used the following strategy: long sequences (most of them backgrounds) were stored on CD and triggered by the sound engineer in the concert. Medium to short materials (1 to 20 second long) where transferred to two Emu E64 samplers and interpreted by a keyboard player during performance. Ashes Dance Back was premiered at the Strasbourg Musica Festival on September 27, 1997.
ATS is a library of Lisp functions for spectral Analysis, Transformation, and Synthesis of sounds. The Analysis section of ATS implements different partial tracking algorithms. This allows the user to decide which strategy is the best suited for a particular sound to be analyzed. Analysis data is stored as a Lisp abstraction called ``sound''. A sound in ATS is a symbolic object representing a spectral model that can be sculpted using a wide variety of transformation functions. ATS sounds can be synthesized using different target algorithms, including additive, subtractive, granular, and hybrid synthesis techniques. The synthesis engine of ATS is implemented using the CLM (Common Lisp Music) synthesis and sound processing language, and runs in real-time in many different platforms. ATS together with CLM provide an environment for sound design and composition that allows the user to explore the possibilities of Spectral Modeling in a very flexible way. The use of a high level language like Lisp presents the advantage of a symbolic representation of spectral qualities. For instance, high level traits of a sound, such as global spectral envelopes, frequency centroids, formants, vibrato patterns, etc., can be treated as symbolic objects and used to create abstract sound structures called ``spectral classes''. In a higher layer of abstraction, the concept of spectral class is used to implement predicates and procedures, conforming spectral logic operators. In terms of this this logic, sound morphing becomes a ``union'' (a dynamic interchange of features) of spectral clases that generates a particular hybrid sound instance.
For more information about ATS see http://www-ccrma.stanford.edu/~juan/ATS.html.
Karlheinz Stockhausen's piece Mantra (1970), for two pianos and live-electronics, marked an important point in terms of real-time electronic music. Stockhausen's piece presents a whole network of interactions both in terms of instrumental actions and sound processing. The performers are required to control not only the intricate interplay between the two instruments but also to control the way the sound of their pianos is transformed by means of ring modulation. A noticeable gap in terms of ``musical'' interpretation arises here; while the players can control to a great extent the piano gestures carefully notated by the composer on the score; adjusting the parameters of ring modulation using a ``dial'' provided with the original analog equipment (designed by Stockhausen back in the 1970) becomes really awkward and complicated for them. The motivation for this project was to create a new interface for the dynamic control of the ring-modulators aiming both to keep the expression of the original setup, that obviously represents an important part of the piece (i.e. the ``continuous'' character of the dial, the grid of fixed frequencies/pitches, etc.), and to create an homogeneous interface for the pianists. The project was achieved in four stages:
Conclusions: This computer implementation of Mantra not only opens the door for more performances of the piece without depending on its original analog gear (there are just a few working analog units that can be rented from the composer's editor), but it also allows for a new musical interpretation of the piece. The sound processing parameters are controlled from an homogeneous user interface that allows the pianists to "play" the modulation frequencies as notes on a keyboard and use wheels and sliders for coarse and fine tuning. Taking advantage of the digital implementation of the sound processing modules new features such as the dc-blocking filters were incorporated helping for better sonic results. Using the MIDI protocol new expression subtleties were introduced, expanding further more the musical interaction of the piece and integrating sound processing controls with the piano gestures.
Spectral transformations had became an important tool for electronic music composers in the last few years. While working with spectral models composers usually want to evaluate how wide a range of new sounds is available by spectral transformations of a particular source. Usually these kind of explorations have to be done step by step out of real-time due to their complexity, limiting the composer to a gradual approximation to the results. This kind of approach tends to constrain the composer's ability to combine transformations and to explore different regions of the spectral structure, finally limiting his creative work in this domain. ATS provides a Spectral User Interface (SUI) for real-time spectral transformations. Using real-time CLM capabilities, the SUI provides the user with a set of sliders that control different transformation parameters during resynthesis. In its present version the SUI provides the following spectral controllers:
, where is
the frequency of a partial, its rank and 
the frequency value of the
fundamental, spectral distortion can be expressed with the following
equation: 
, where 
is the distortion factor. If the value of is
1.0 we obtain a harmonic structure, if we increase its value we get a non
linear frequency structure that is perceived as inharmonic (by default the
user can adjust the distortion factor within a range of 0.0 to 1.0)
Conclusions: Using ATS's SUI the composer can explore many ways of transforming spectral data during resynthesis. Transformations can not only be dynamic but can also be limited to a particular region of the spectrum by means of the TimeScale slider. Transformations can be compounded to create complex spectral results that the user can explore in real-time. On SGI platforms sliders can be controlled through MIDI so the user can use more ergonomic controllers (like fader boxes, wheels, etc.) to synchronically control several sliders.
SynthBuilder is a user-extensible, object-oriented, NEXTSTEP Music Kit application for interactive real-time design of synthesizer patches. Patches are represented by networks of digital signal processing elements called ``unit generators'' and MIDI event elements called ``note filters'' and ``note generators''. SynthBuilder is based on Eric Jordan's GraSP application, created at Princeton University in 1992, and the NeXT Draw example. The graphical interface enables construction of complex patches without having to write a single line of code, and the underlying Music Kit software provides support for real-time DSP synthesis and MIDI. This means there is no ``compute, then listen'' cycle to slow down the process of developing a patch. It can be tried out immediately on a MIDI keyboard, and unit generator and note filter parameters can be adjusted in real time while a note is still sounding. Sixteen bit stereo sound is synthesized immediately on one or more DSP56001 signal processing chips, and can be controlled from the user interface with software-simulated or physical MIDI devices.
In addition to synthesis, the system supports configurations for sound processing via the DSP serial port which is also used for sound output to DACs and other digital I/O devices. MIDI messages can be mapped to unit generator object control methods, permitting high-level control of patch parameters. For example, a MIDI key number can be readily mapped into frequency, and then mapped into a delay line length via a graphically constructed lookup table. A single MIDI event can be fed to (or through) multiple note filters, each of which can modify the event stream and/or control one or more unit generator parameters. Polyphony is handled in SynthBuilder by graphically specifying a voice allocation scheme. Optionally, a Music Kit SynthPatch can be generated (in high-level source-code form) and used in another application. Dynamically loadable custom ``inspectors'' (user interfaces) can be created for patch elements. Dynamic loading promotes easy distribution and sharing of inspector modules, and promotes a fast, efficient development cycle. The process of creating a custom inspector is facilitated by a default-inspector-generator which takes a DSP assembly macro and a signal-flow/parameter list specification as input, and creates working interface code which can then be customized.
As of this writing, SynthBuilder has more than 50 graphical custom inspectors, including an envelope editor, digital filter response curves, and a MIDI lookup table. SynthBuilder is currently being used by researchers at CCRMA to explore new synthesis techniques. SynthBuilder is now in the alpha release stage on both NeXT and Intel Pentium systems. Supported DSP cards for Intel systems include the Ariel PC56D, the Turtle Beach Multisound or Monterrey, and the i*link i56.
The continuing rise in processor speed in today's computers makes software synthesis ever more viable, even for real-time applications. Music, however, tends to contain high levels of polyphony and complexity, and can easily surpass the processing ability of any single processor to keep up with real time. This problem is expected to exist for at least a few more generations of processors, simply because of the sheer complexity of current projects. Therefore some sort of parallel processing is necessary.
This project started with the design and construction of the Frankenstein Box Multiple-DSP Processing Engine for MusicKit. The Frankenstein Box consists of 8 EVM56002 evaluation modules (chosen because of inexpensiveness and compatibility with the current MusicKit architecture), along with glue logic and sound hardware.
The project continues with the specification of the Franken-II system, which places all 8 56002 chips on a single PCI card and improves the inter-DSP communication and audio routing.
As the MusicKit and other real-time software synthesis systems at CCRMA move to using general-purpose microprocessors for calculation, this project will try to address concerns relating to the ability to easily scale the systems beyond single processors. Primary considerations are: (1) the portability of code between main processors and peripheral processors which, because of economics and other factors, are often different types of processors than the main processor; (2) the ability to communicate easily between processors and to move processing tasks between processors in as transparent a manner as possible; and (3) the ease of further scaling to any level. These considerations effect the design of the system at many levels, from the design of the add-on processor systems up to the architecture of the software synthesis system itself.
Planet CCRMA is an HTML document for the purpose of illustrating and informing new CCRMA users and visitors about the computer resources, the Linux environment, and applications which might be helpful for doing research and compositional work at CCRMA. It also briefly describes the meaning of ``open source'' as a part of a laboratory and community philosophy at CCRMA. It is also a brief history of hardware at CCRMA and descriptions of Linux as an operating system, the Unix environment, useful shell commands and many X windows applications in addition to Gnome and KDE desktops. In the applications section there are descriptions of programs and information provided by the developers' documentation and direct links to the application web page. Planet CCRMA focus is as the first stepping stone for a particular command, program or application but nevertheless the reader is encouraged to find more in-depth information on the Unix manual pages, on the web or in the links to home pages which will are also provided. During the 2001 autumn quarter at Stanford the web page was visited by more than 80% of new and old users of the CCRMA network and community. Planet CCRMA is also updated on a regular basis as per users suggestions and because of new software, upgrades, or updates to the system.
Resonant filters can be fine tuned to a very narrow frequency band thereby isolating a tone even from a non-pitched sound source. ``Maxf.ins'' is Max Mathews' new filter (2002) described as a high-Q, 2-integrator filter with two poles, and one zero at the origin. This Common Lisp Music (CLM) implementation renders equal tempered frequencies, integer and just scales out of a wide-band input signal. The filter might be used for Modal Synthesis but also might be Additive Synthesis in which a resonator is initialized to generate the exponentially decaying sinusoids at the desired phase. Different filters which are bound in parallel are defined in a structure which contains various frequencies and tunings for resonant modes. In this algorithm the filter is recurrent over the source signal by iterating the number of desired frequencies in a state. States can be defined as containing at least one frequency and can go up to the CPU processing power.
``Strad.ins'' is a Common Lisp Music (CLM) instrument implementation of the bowed string physical model with stiffness based on previous research by Serafin, Smith, Woodhouse, and others. It is specially suited for algorithmic composition and expression modeling of stringed instrument gestures because of its modular qualities inside the Lisp environment.
The instrument features non-real time rendering of bowed string sounds with variables such as string stiffness, bow force and friction interaction between the bow and the string. It also accounts for the effect of torsional waves on the bridge side and on the finger side and dispersion simulation plus Helmholtz motion. The algorithm is based on recent research done by Serafin, et al. using Matlab, Pd and STK implementations. The instrument is optimized for frequencies or rather tones inside a 100Hz and 600Hz range and are function of the ratio of its parameters. Its design allows for timed envelope style manipulation of most of CLM instrument parameters.
Strad.ins is part of the CLM-2 distribution at the CCRMA software ftp site.
Reference:
Sig++ is a set of C++ classes intended for use in writing sound generating/filtering programs by direct coding of flowgraphs schematics for signal processing filters as well as for traditional computer-music unit-generator flowgraphs. The paradigm for generating sound is similar to other Music V-style synthesis programs, such as Csound.
An intent of sig++ is to have very portable code. As a result, example programs using the sig++ library have been compiled on several computer configurations: Linux, Windows 95/NT, OpenStep, NextStep, Sun SPARCStations, HP-UX, and SGI IRIX.
See the main webpage for sig++ at http://www-ccrma.stanford.edu/~craig/sig which contains an overview, example binaries and sources, example sounds created by the example programs, documentation for the classes included in the sig++ library, as well as the source code for those classes.
Future additions to sig++ will be real-time sound input/output in Windows 95/NT and Linux as well as linking control of sound generation to MIDI using Improv.
A command-line program, line2sine, was written to interpret graphic lines in a CAD-like drawing program as sinwaves. Documents created by the NEXTSTEP program Diagram.app are read by the line2sine program and any lines in that document are converted into freqency and amplitude envelopes which are then fed into oscillator unit-generators. The line2sine program can be downloaded from ftp://ftp.peanuts.org/NEXTSTEP/audio/programs/line2sine.1.0.NI.bs.tar.gz or http://www.peak.org/next/apps/LighthouseDesign/Diagram/line2sine.1.0.NI.bs.tar.gz. These two files contain the program, documentation, and examples. On-line documentation as well as example conversions between graphics and sound can be found at http://hummer.stanford.edu/sig/doc/examples/line2sine.
The Webpage http://www-ccrma.stanford.edu/CCRMA/Software/SCMP contains links to various Mathematica packages dealing with computer music topics. The main package, SCMTheory, contains visualization and manipulation tools dealing with the fundamentals of digital signal processing, such as complex numbers, plotting complex domains and ranges, and modulo sequences and manipulations. The Windows package contains the definitions of various analysis windows used in short-time fourier transform analysis. The FMPlot package contains functions for plotting simple FM-synthesis spectra.
All packages run with Mathematica version 2.0 or greater, except the Windows package which requires Mathematica 3.0. Included on the SCMP main webpage are Mathematica notebooks which demonstrate various aspects of the SCMP set of packages. Also included on the SCMP main webpage are these notebooks in PDF format for viewing by those people who do not have Mathematica.
RtAudio is a C++ class recently added to the STK which provides a common API (Application Programming Interface) for realtime audio input/output across Linux (native ALSA, JACK, and OSS), Macintosh OS X, SGI, and Windows (DirectSound and ASIO) operating systems. RtAudio significantly simplifies the process of interacting with computer audio hardware. It was designed with the following goals:
RtAudio incorporates the concept of audio streams, which represent audio output (playback) and/or input (recording). Available audio devices and their capabilities can be enumerated and then specified when opening a stream. Where applicable, multiple API support can be compiled and a particular API specified when creating an RtAudio instance. See the API Notes section for information specific to each of the supported audio APIs
The RtAudio API provides both blocking (synchronous) and callback (asyncronous) functionality. Callbacks are typically used in conjunction with graphical user interfaces (GUI). Blocking functionality is often necessary for explicit control of multiple input/output stream synchronization or when audio must be synchronized with other system events.
Reference:
The nature of computer support for digital signal processing (DSP) research is an ongoing issue. Initially, the Fortran programming language was the standard ``high level'' representation, and hardware and horizontal microcode served as ``low level'' representations for mass-produced products. While special purpose hardware (e.g., ASICs) and DSP microcode continue to thrive, still giving the lowest asymptotic cost in mass production, the higher level tools have changed considerably: Fortran is all but obsolete in favor of C, and C is rapidly giving way to its object-oriented extension, C++. For faster research prototyping at the expense of slower execution, interactive programming environments such as MatLab are being used in place of classical software development. These programming environments offer extensive display capabilities and a high-level, interpreted language with easy to use syntactic support of common signal processing operations, both in the time and frequency domains. At a still higher level of abstraction, development tools supporting the direct manipulation of block diagrams are becoming more common. Examples include SimuLink (MatLab), LabView (National Instruments), Ptolemy and Gabriel (Berkeley), Max and TurboSynth (Opcode), SynthKit (Korg R& D), ComDisco, Star, and other CAD tools related to signal processing.
In a well designed rapid prototyping system, it is possible to work at all levels in a variety of alternative representations such as block diagrams, matlab, object-oriented C, or assembly language.
In typical music synthesis and audio signal processing applications, it is not necessary to sacrifice more than a few percent of theoretical maximum DSP performance, in terms of both speed and code size, in return for the use of a high-level, block-diagram oriented development tool. This is because a small number of primitive modules can implement the vast majority of existing synthesis and processing techniques, and they account for the vast majority of the computational expense. These modules can be fully optimized in advance so that simple, drag-and-drop programming can provide both a real-time simulation and well structured code generation which are very close to the efficiency of a special-purpose, hand-coded, DSP assembly language program. As a result, block-diagram based programming tools are fundamental to good signal processing support in music synthesis and digital audio development systems.
For rapid research prototyping in music and audio applications, there remains an unfulfilled need for a full-featured, available, open, and well structured development system supporting MIDI and digital audio synthesis and signal processing. CCRMA is presently supporting in part the development of SynthBuilder, a block-diagram based rapid prototyping tool for these purposes. SynthBuilder leverages very heavily off of the advanced capabilities of the Music Kit and NEXTSTEP.
References
The SynthBuilder, SynthScript, and SynthServer projects have been spun out from CCRMA to a new company Staccato Systems, Inc. The tools are currently being ported to ``all major platforms'' and focused into specific software products. Watch the Staccato website for latest details.
Extending existing software at CCRMA to incorporate tactile manipulation of software. My work at the Harvard Computer Music Center involved adapting computer music software to emulate analog studio techniques. I plan to adapt digital signal processing programs to accept MIDI or other external controller information to change program parameters. For example, an on-screen digital filtering program would have its frequencies, bandwidth, and attenuation set by MIDI sliders, so a composer could manipulate parameters in a tactile way, emulating analog graphic equalizers. By setting up external controllers, the composer would then be able to manipulate several parameters at once, as opposed to typing single parameters, or adjusting one parameter at a time with the mouse. I then plan to use this type of interactive control in live performance.
| © Copyright 2005 CCRMA, Stanford University. All rights reserved. |