Lab 2: Sound Synthesis + Sonic Pong

Coding Sound: Additive and FM Synthesis

In the Matlab/Python portion of this lab, you’re going to continue to build your library of sound synthesis functions you can use to generate audio samples for your games. Modify / build the specified functions, and create a script for testing and answering questions in comments (Matlab comments start with %).

1. Additive Synthesis: Synthesizing sounds by combining sinusoids

Here is a fundamental, important truth about audio signals (or any signal, for that matter): Any waveform can be broken down into sinusoids (specifically, complex sinusoids - but we can talk about complex sinusoids at another point). Sinusoids are the most rudimentary building block of a signal. There are three parameters to a sinusoid: Amplitude, Frequency, and Phase. We’re going to use our sineTone function from lab 1 to build a bunch of different, crazy sounds (ok, I’ll admit that most of these sounds are pretty boring, but it’ll still be cool to see how they are made).

1.a Adding Phase to sineTone()

Before we can start adding sinusoids together, we need to add an initial phase parameter to the sineTone() function from lab 1. Phase will become important for how different sinusoids combine in the next section.

Modify your sineTone function from lab 1 to the following format:
output = sineTone(fs, frequency, phase, duration, amplitude)
Note in addition to adding the phase argument, we’re now also including an amplitude argument. It’d be nice to let amplitude be an optional argument with a default value that gets used if it’s not specified, but this gets into a more complicated Matlab implementation. We want to keep things simple, so we’ll just specify it when we call the function, but a good ‘default’ value is 0.99.

The generalized sinusoid equation we will now be using is:
$y(t) = A\sin(2\pi f t + \phi) = A\sin(\omega t + \phi)$
where $A$ is amplitude (which we assumed to be 1 in lab 1, unless you set it to 0.99 as parenthetically discussed), $f$ is frequency in Hz, and $t$ is time in seconds (which we’re still calculating as $=\frac{n}{fs}$). $\phi$ is initial phase, in radians, and as such wraps at $2\pi$ and is customarily written modulus $2\pi$ (i.e. $\in [0 - 2\pi)$). $\omega$ is frequency in radians per second instead of Hz (cycles per second) ($=2\pi\times f$).

For more about phase visit this wikipedia page for phase. The wikipedia page for sinusoids is also worth checking out for more discussion of any of the above parameters or sinusoids in general.

1.b Combining Sinusoids

Using your new and improved sineTone() function, add together two sinusoids with the same frequency, duration, amplitude, and sampling rate but with different phases. To add vectors of the same length in Matlab you can just use +. Use sampling rate of 44100, frequency of 220, duration of 2, and amplitude of 0.5 for both sinusoids. Keep the phase of the first sinusoid equal to zero, and vary the phase of the second between 0, $\pi/2$, $\pi$, and $3\pi/2$. Listen to the four resulting summed outputs. What do you notice? Plot the first 1000 samples of each summed output. Use the Matlab function subplot or the Python function matplotlib.pyplot.subplots to plot them all in a single plot window. Write a brief description of what you see and hear in a comment in your Matlab script. Hint: pay attention to the scale on the Y axis - how big are the actual numbers in the signal? You may find it useful to look up the term round-off error. If the results of summing sinusoids with different initial phase is not what you expected, look up the terms constructive and destructive interference, and if you have any additional questions about what they mean ask after class or in office hours.

1.c Other waveforms: Saw, Square, Tri

Now we’re going to make some waveforms besides a simple sinusoid, just by adding sinusoids together.
Create three functions to generate these waveforms called sawtoothTone, squareTone, and triangleTone:
output = sawtoothTone(fs, frequency, phase, duration, harmonics, amplitude)
output = squareTone(fs, frequency, phase, duration, harmonics, amplitude)
output = triangleTone(fs, frequency, phase, duration, harmonics, amplitude)
For all of these functions, the arguments will have the same meaning:
fs - sample rate, as before
frequency - the fundamental frequency ($f_0$) of each tone: the frequency of lowest sinusoid we’ll be adding together (harmonic = 1).
harmonics - the number of sinusoids we’ll add together. Each function will return a signal that is the sum of multiple sineTone() outputs with specific frequencies that are integer multiples of the fundamental frequency ( $f_0\times k $, where $k$ represents the harmonic number from 1 to harmonics).
The equation for a sawtooth wave is: $y(t)=\frac{1}{2}-\frac{1}{\pi}\sum^K_1\frac{\sin(2\pi kft+\phi)}{k}$
(summation is over $k$, and $K$ is the number of sinusoids, or harmonics)

The equation for a squareTone is: $y(t)=\frac{4}{\pi}\sum^K_1\frac{\sin(2\pi(2k-1)ft+\phi)}{(2k-1)}$

The equation for a triangle wave is: $y(t)=\frac{8}{\pi^2}\sum^K_0(-1)^k\frac{\sin(2\pi(2k+1)ft)}{(2k+1)^2}$
Note the summation index $k$ starts at zero for the triangle wave and one for the sawtooth and square waves.

Once you have your functions, experiment with changing the value of harmonics. What do the waveforms start to look like as the value of harmonics increases? Also experiment with different values of $\phi$ for the sawtooth and square waves. How does it affect the waveform? Why is there no $\phi$ present in the triangle wave equation? Answer these questions in your script comments.

Plot the first 1000 samples of the output from each function using the same parameters for each - you can choose what they are, just be sure to label what parameters you use and the function name on each plot (look up the Matlab function title() and the Python function suptitle()). You can use subplot again to put all three plots in the same plot window.

2. FM Synthesis

Finally, we’ll create a function that uses FM, or Frequency Modulation (some argue the term Phase Modulation is more technically accurate) synthesis to create a wider range of more interesting sounds. The idea is we use the output of one sinusoid to modulate the frequency (or phase) of another sinusoid. If you’re interested in learning more about FM synthesis, you’re in the right place…FM synthesis was invented/discovered by John Chowning here at CCRMA. You can read more about it here.

Create one last function called fmTone:
output = fmTone(fs, fc, fm, index, duration, phase, amplitude)
where:
fs - sample rate
fc - the carrier frequency is the base frequency of the sinusoid that gets modulated
fm - the modulator frequency is the frequency at which the carrier gets modulated. Also sometimes referred to as the rate of modulation.
index - the modulation index determines the range of modulation of the carrier by the modulator. This is $A_m$ in the equation below (Amplitude of the modulator sinusoid).
duration - length of generated tone in seconds
phase - this will be the difference in initial phase between the two sinusoids, so if you set the phase argument of the carrier to zero, you can use this value as the phase argument of the modulator
amplitude - maximum amplitude of the output signal ($A$)

The equation for FM synthesis is: $y(t) = A\sin(2\pi (f_c t+(A_m\sin(2\pi f_m t))))$

Play around with your new FM function - changing fc, fm, and index will create some drastically different sounds. fc is the closest thing to a fundamental frequency, so values close to what we’ve been using (200-500) are a good place to start. fm can range anywhere from 5-5000 with interesting results, and index can also be anything from 1-100 or more. Looking at the equation for FM above, you can understand that a given $A_m$ will correspond to modulation ranges that are a different percentage of the carrier frequency based on the actual $f_c$ value. For example, if $A_m$ = 20, this will have a different perceptual effect if $f_c$ is 100 Hz (so carrier frequency varies between 80-120 Hz) versus if $f_c$ is 500 Hz (and carrier varies between 480-520 Hz).

Submit a 0.5 second sound made with your fmTone() function that you particulary like or find interesting (as a .wav file). Record the parameter values for your sound in a comment in your script.

Whew! That’s a lot of new functions, but now you should have a fairly robust and useful set of synthesis tools in your library. Keep in mind, you can stack any of these signals together by simply adding their output samples together, or make a sequential signal out of multiple signals by combining them into a single vector like this: [output1, output2] (as many at once as you want). Don’t forget you still have your rampOn() and rampOff() functions from lab 1, which you can use on any of your new functions also. Also remember we saw how varying phase can create interesting constructive and deconstructive interference patterns in a sound!

Matlab Deliverables:

For the Matlab portion, submit your revised sineTone() function from 1.a, your plot (with 4 phase interference subplots) from section 1.b, your sawtoothTone(), squareTone(), and triangleTone() functions and plot (with 3 subplots of 1000 samples of your functions) from section 1.c, your fmTone() function and 0.5 second .wav file from section 1.d, and your script containing any test code and your question answers for each section in comments.

Game Development: Sonified Pong

Take Pong as an inspiration for your first video game. In a group of 3-4 we’d like you to design, program, and test a new version of pong with a focus on sound!

You can do anything you want with the Pong concept! You could add more players, add or modify game mechanics, add new artwork, make sound spatialized, employ frequency spectrum as a game mechanic, or create something you could play with your eyes closed! Go crazy! Just remember to focus on sound design and think about the how the sounds you are using can give cues to the player.

For your first foray into building a video game, we will provide a Unity project with all the basic building blocks of a Pong game. Please see the Canvas assignment for more details. .

Use your newly expanded library of Python/Matlab functions to create sounds for your game (for at least the majority of sounds - if you want to include a few other audio clips that’s ok, if they fit with a theme for example). As we’ve been doing, you can write your tones to a .wav file using audiowrite(signal, fs, filename);. If necessary, you may edit your sounds in a DAW such as Audacity, Logic, etc., but only for basic editing. For now, do not use instruments, other sound generating plug-ins, sound effects, or other processing plug-ins for your game sounds.

Use the same Design Process as Lab 1. Conduct your playtest with people who are not in the class.

Game Design Deliverables:

Submit links to your pitch/demo and playtest videos using the same guidelines as Lab 1, detailed on the Lab Overview page.

Write up a short design document using the template on Canvas and please include how you used sound in your game and how you think it contributes to the game experience. See if you can tie in topics we’ve discussed in lecture (loudness, masking, expectation, or anything about the auditory pathway that you’ve learned so far). In your design document you should also address what you learned through the playtest. One design document per group is sufficient.

Submit links (YouTube or Vimeo) to your pitch/demo and play testing videos, and your design document. Please DO NOT submit your entire Unity project on Canvas.

See Canvas for due dates.

Lecture

Fridays, 9:45 - 11:45 AM PM
CCRMA Classroom (Knoll 217)

Lab

Tuesdays, 6:00 - 7:50 PM
CCRMA Classroom (Knoll 217)

Office Hours

See Canvas

Questions

Post on Discord or Email/p>

Instructors

Poppy Crum
Instructor
poppy(at)stanford(dot)edu

Lloyd May
Teaching Assistant
lloydmay(at)stanford(dot)edu