Lab 3: “Hybrid” Instruments

Download this lab’s code here.

The goal of this lab is to make a musical instrument combining acoustic sound excitations with digital elements. Sounds excitations are fed into a digital resonator (e.g., a waveguide, a comb filter, a reverb, etc.). The parameters of the resonator can be changed to modify the properties of the generated sound.

Connecting a Microphone or a Piezo to Faust

• First, make sure that the audio worklet mode is enabled in the Faust web editor (click on the wrench icon and then choose “AudioWorklet” instead of “ScriptProcessor”). For this to work, your version of Google Chrome must be up-to-date!
• Audio inputs are accessible from the process line of a Faust program. For example, the following program connects the first input (i.e., microphone in the web browser) to the first output (built-in speakers):

process = _;

• The following connects the first input to the first two outputs:

process = _ <: _,_;

• Hence, the audio input can be connected to any argument of a function by placing it on the process line:

mult2(x) = x*2;
process = mult2;

or even:

mult(y,x) = x*y;
process = mult(2);

• Be careful with that when prototyping with headphones as it might create feedback!
• Microphones can be used to capture sound in the air and piezos in solids.
• To be connected to a computer, the signal of microphones and piezos must be digitized. This can be done by using an audio interface. A bunch of them are available in the MaxLab for this week’s lab.
• Take the piezo from your kit and solder a coaxial cable to it as such:

• Note that we removed the red and black wires that were pre-soldered to the piezo.
• Make sure the 2 pins of the cable are not touching each other! You might want to use a multimeter to check that.
• Solder the 1/4" male jack plug of your kit to the other side of the cable:

• Your piezo + cable + jack should now look like this (in your case, the cable should be a bit longer):

• Plug one of the USB audio interface in the MaxLab to your computer. You can find them in the lower drawer of the cabinet next to Romain’s desk:

• There are headphones in that drawer that you can use too. In case there are not enough headphones in the drawer, use the ones in the Rubbermaid bin.
• You should make sure that the audio interface works by setting it as the default audio input and output in the audio preferences of your system. Since this works differently on each operating system, feel free to ask if you have no idea of how to do that.
• Plug your piezo to the first mic input of the audio interface. Adjust the gain of the preamp by tapping on it and monitoring picks. Once again, if you’ve never done that before, feel free to ask Doga or Romain for help.
• Try your system in Faust by running the following simple program:

process = _ <: _,_;

• You should be able to hear the sound captured by the mic or the piezo on the left and right channels of your headphones.
• Let’s now use this sound to excite virtual resonators running in Faust. Resonators can be implemented in many different ways using comb filters, modal and waveguide physical models or even with a simple reverb.
• Try to run the starter code files. Don’t forget to drive them with acoustic excitations captured with your piezo or your mic. Experiment with the different parameters of the resonators and see how they impact the generated sound.
• A particularly interesting example is modalModel.dsp. Each mode of resonance of the model is implemented with a resonant bandpass filter. The center frequency of the filter determines the frequency of the mode, the gain of the filter corresponds to the gain of the mode, the bandwidth of the filter controls the resonance duration of the mode (here set at T60 in seconds).
• Modal synthesis can be used to make a physical model of any linear time-invariant system. Hence each parameter of the model might greatly affect the properties of the generated sound. Feel free to play with the mode parameters to see how they impact the synthesized sound. You might also want to add new modes to the system.

Note About Audio Latency

Using the web browser as our main prototyping platform has some drawbacks. One them is audio latency. Indeed, you might have noticed that it takes some time to process a sound and then play it back. You might be able to reduce this effect by decreasing the buffer size in the Faust web editor configuration panel. Be aware that this might introduce some glitches in the generated sound.

A more efficient solution is to generate a native application, but for that, you’d have to install Faust on your computer. If you’re feeling adventurous and want to give it a try, feel free to get in touch with Romain (solutions will vary depending on your target platform).

For this lab, it is likely that latency might not be an issue for the type of instrument that you will make. If your final project for this class involves the use of an audio input, we’ll dig things a bit more but for now you should not think too much about it.

Exercise: Adding Sensor Control to the Previous Instruments

• Take any of the starter code and control one or several of its parameters with external sensor(s).
• Lab 2 should contain all the material needed for you to reach this goal.
• Make sure that your instrument is expressive and playable.
• Try different sound sources, materials, etc. to drive it. Feel free to use anything available in the MaxLab for that (e.g., wood/metal pieces, mugs, tables, etc.).
• Feel free to incorporate some Faust audio effects to your instrument.

Assignment (Due on Jan. 30, 2019)

• Make an instrument combining acoustic excitations captured in real-time with piezo(s) and/or microphone(s) to a virtual resonator and audio effect(s).
• Your instrument should use sensors too.
• Feel free to explore different materials and mediums to create different types of sound excitations.
• Make sure that your instrument is expressive, artful, beautiful, playable, etc.
• Make a video of yourself playing it and post it on YouTube.
• Send the link of this video along with the source code of your instrument to Doga and Romain by Jan. 30, 2019.
• Good luck and have fun!