Lab 2

Matlab: Additive and FM Synthesis

In the Matlab portion of this lab, you’re going to continue to build your library of Matlab functions you can use to generate audio samples for your games. Modify / build the specified functions, and create a script for testing and answering questions in comments (Matlab comments start with %).

1. Additive Synthesis: Synthesizing sounds by combining sinusoids

Here is a fundamental, important truth about audio signals (or any signal, for that matter): Any waveform can be broken down into sinusoids (specifically, complex sinusoids - but we can talk about complex sinusoids at another point). Sinusoids are the most rudimentary building block of a signal. There are three parameters to a sinusoid: Amplitude, Frequency, and Phase. We’re going to use our sineTone function from lab 1 to build a bunch of different, crazy sounds (ok, I’ll admit that most of these sounds are pretty boring, but it’ll still be cool to see how they are made).

1.a Adding Phase to sineTone()

Before we can start adding sinusoids together, we need to add an initial phase parameter to the sineTone() function from lab 1. Phase will become important for how different sinusoids combine in the next section.

Modify your sineTone function from lab 1 to the following format:
output = sineTone(fs, frequency, phase, duration, amplitude)
Note in addition to adding the phase argument, we’re now also including an amplitude argument. It’d be nice to let amplitude be an optional argument with a default value that gets used if it’s not specified, but this gets into a more complicated Matlab implementation. We want to keep things simple, so we’ll just specify it when we call the function, but a good ‘default’ value is 0.99.

The generalized sinusoid equation we will now be using is:
$y(t) = A\sin(2\pi f t + \phi) = A\sin(\omega t + \phi)$
where $A$ is amplitude (which we assumed to be 1 in lab 1, unless you set it to 0.99 as parenthetically discussed), $f$ is frequency in Hz, and $t$ is time in seconds (which we’re still calculating as $=\frac{n}{fs}$). $\phi$ is initial phase, in radians, and as such wraps at $2\pi$ and is customarily written modulus $2\pi$ (i.e. $\in [0 - 2\pi)$). $\omega$ is frequency in radians per second instead of Hz (cycles per second) ($=2\pi\times f$).

For more about phase visit this wikipedia page for phase. The wikipedia page for sinusoids is also worth checking out for more discussion of any of the above parameters or sinusoids in general.

1.b Combining Sinusoids

Using your new and improved sineTone() function, add together two sinusoids with the same frequency, duration, amplitude, and sampling rate but with different phases. To add vectors of the same length in Matlab you can just use +. Use sampling rate of 44100, frequency of 220, duration of 2, and amplitude of 0.5 for both sinusoids. Keep the phase of the first sinusoid equal to zero, and vary the phase of the second between 0, $\pi/2$, $\pi$, and $3\pi/2$. Listen to the four resulting summed outputs. What do you notice? Plot the first 1000 samples of each summed output. Use the Matlab function subplot or the Python function matplotlib.pyplot.subplots to plot them all in a single plot window. Write a brief description of what you see and hear in a comment in your Matlab script. Hint: pay attention to the scale on the Y axis - how big are the actual numbers in the signal? You may find it useful to look up the term round-off error. If the results of summing sinusoids with different initial phase is not what you expected, look up the terms constructive and destructive interference, and if you have any additional questions about what they mean ask after class or in office hours.

1.c Other waveforms: Saw, Square, Tri

Now we’re going to make some waveforms besides a simple sinusoid, just by adding sinusoids together.
Create three functions to generate these waveforms called sawtoothTone, squareTone, and triangleTone:
output = sawtoothTone(fs, frequency, phase, duration, harmonics, amplitude)
output = squareTone(fs, frequency, phase, duration, harmonics, amplitude)
output = triangleTone(fs, frequency, phase, duration, harmonics, amplitude)
For all of these functions, the arguments will have the same meaning:
fs - sample rate, as before
frequency - the fundamental frequency ($f_0$) of each tone: the frequency of lowest sinusoid we’ll be adding together (harmonic = 1).
harmonics - the number of sinusoids we’ll add together. Each function will return a signal that is the sum of multiple sineTone() outputs with specific frequencies that are integer multiples of the fundamental frequency ( $f_0\times k $, where $k$ represents the harmonic number from 1 to harmonics).
The equation for a sawtooth wave is: $y(t)=\frac{1}{2}-\frac{1}{\pi}\sum^K_1\frac{\sin(2\pi kft+\phi)}{k}$
(summation is over $k$, and $K$ is the number of sinusoids, or harmonics)

The equation for a squareTone is: $y(t)=\frac{4}{\pi}\sum^K_1\frac{\sin(2\pi(2k-1)ft+\phi)}{(2k-1)}$

The equation for a triangle wave is: $y(t)=\frac{8}{\pi^2}\sum^K_0(-1)^k\frac{\sin(2\pi(2k+1)ft)}{(2k+1)^2}$
Note the summation index $k$ starts at zero for the triangle wave and one for the sawtooth and square waves.

Once you have your functions, experiment with changing the value of harmonics. What do the waveforms start to look like as the value of harmonics increases? Also experiment with different values of $\phi$ for the sawtooth and square waves. How does it affect the waveform? Why is there no $\phi$ present in the triangle wave equation? Answer these questions in your script comments.

Plot the first 1000 samples of the output from each function using the same parameters for each - you can choose what they are, just be sure to label what parameters you use and the function name on each plot (look up the Matlab function title() and the Python function suptitle()). You can use subplot again to put all three plots in the same plot window.

2. FM Synthesis

Finally, we’ll create a function that uses FM, or Frequency Modulation (some argue the term Phase Modulation is more technically accurate) synthesis to create a wider range of more interesting sounds. The idea is we use the output of one sinusoid to modulate the frequency (or phase) of another sinusoid. If you’re interested in learning more about FM synthesis, you’re in the right place…FM synthesis was invented/discovered by John Chowning here at CCRMA. You can read more about it here.

Create one last function called fmTone:
output = fmTone(fs, fc, fm, index, duration, phase, amplitude)
where:
fs - sample rate
fc - the carrier frequency is the base frequency of the sinusoid that gets modulated
fm - the modulator frequency is the frequency at which the carrier gets modulated. Also sometimes referred to as the rate of modulation.
index - the modulation index determines the range of modulation of the carrier by the modulator. This is $A_m$ in the equation below (Amplitude of the modulator sinusoid).
duration - length of generated tone in seconds
phase - this will be the difference in initial phase between the two sinusoids, so if you set the phase argument of the carrier to zero, you can use this value as the phase argument of the modulator
amplitude - maximum amplitude of the output signal ($A$)

The equation for FM synthesis is: $y(t) = A\sin(2\pi (f_c t+(A_m\sin(2\pi f_m t))))$

Play around with your new FM function - changing fc, fm, and index will create some drastically different sounds. fc is the closest thing to a fundamental frequency, so values close to what we’ve been using (200-500) are a good place to start. fm can range anywhere from 5-5000 with interesting results, and index can also be anything from 1-100 or more. Looking at the equation for FM above, you can understand that a given $A_m$ will correspond to modulation ranges that are a different percentage of the carrier frequency based on the actual $f_c$ value. For example, if $A_m$ = 20, this will have a different perceptual effect if $f_c$ is 100 Hz (so carrier frequency varies between 80-120 Hz) versus if $f_c$ is 500 Hz (and carrier varies between 480-520 Hz).

Submit a 0.5 second sound made with your fmTone() function that you particulary like or find interesting (as a .wav file). Record the parameter values for your sound in a comment in your script.

Whew! That’s a lot of new functions, but now you should have a fairly robust and useful set of synthesis tools in your library. Keep in mind, you can stack any of these signals together by simply adding their output samples together, or make a sequential signal out of multiple signals by combining them into a single vector like this: [output1, output2] (as many at once as you want). Don’t forget you still have your rampOn() and rampOff() functions from lab 1, which you can use on any of your new functions also. Also remember we saw how varying phase can create interesting constructive and deconstructive interference patterns in a sound!

Matlab Deliverables:

For the Matlab portion, submit your revised sineTone() function from 1.a, your plot (with 4 phase interference subplots) from section 1.b, your sawtoothTone(), squareTone(), and triangleTone() functions and plot (with 3 subplots of 1000 samples of your functions) from section 1.c, your fmTone() function and 0.5 second .wav file from section 1.d, and your script containing any test code and your question answers for each section in comments.

Game Development: Modified / Augmented Breakout

The game development portion of this lab consists of two parts. The first part you will complete individually, while the second part will be completed in groups.

Solo: Getting Started with Unity

Unity is a game creation engine: a giant collection of tools that handles graphics, audio, controller input, and routines for various scripts and such. Think of it as GameSalad on steroids. If you haven’t done so already, you can find the link to install Unity on the course Resources page. You don’t need to select additional target platforms when you install unless you know you want to develop for them (you can always add them later also). You will have to create an account with Unity before you can start working with it. Select the Personal and Educational options when presented with them.

After you download and install Unity, go through this Breakout tutorial independently. Breakout is a classic game (the original was played on the Atari 2600 with a paddle controller) that you may know by the name of Brickbreaker. It’s kind of like Pong, except instead of playing against someone else, you are playing against an endless army of evil bricks.

After you finish the tutorial, change one small thing about the game. It could be the color scheme, or the amount of bricks, or the scoring mechanism, or add your own Matlab-generated sounds! In your submission for the assignment, include a short document that describes what you did to personalize your Breakout along with a few screenshots or link to a short video that shows it off.

Group Breakout, Augmented

Now that you have a basic understanding of the fundamentals of using Unity, build a basic game that uses a specific controller. This game could build off of one of your Breakout demos. Think about what your controller offers you in terms of input - what are the different ways a player can interface with the controller? Are there any real-world gestures that the controller accepts as input that we can translate to an in-game mechanism, giving the player a greater sense of physical presence in the game?

Think about creative ways of controlling game objects. Maybe you can use the microphone somehow to get auditory data, or use motion capture and a webcam. Some libraries even let you use eye tracking! If you have a sensor like a Kinect or a LeapMotion, you're certainly welcome to use that, but it might limit how many of your classmates can play the game on Tuesday.

Here are a some links for the Leap Motion (How To, Cool Examples) and Kinect (How To and Another One, Cool Examples).

Game Design Deliverables:

Once again, submit links to your (hopefully standard by now) pitch and playtest videos (details on the Lab Overview page if you’ve forgotten).

This lab will be due in two parts: the solo Breakout Demos will be due Friday, April 17th by class time and the group Augmented Games will be due Tuesday, April 21st by classtime. The games need to be completed by classtime so we can demo them in class, but the documentation (videos) can be submitted after class if necessary.

Note: As with the other labs, the Matlab due-date is flexible; you should try to stay "on-track" with the Matlab (meaning that you should try not to let them pile up), but don't worry about rushing to complete them by the lab due-date if you need more help/time.

Deliverables are due by 6:00 PM, Tuesday, April 21st.

Lecture

Fridays, 10:30 AM - 12:20 PM
CCRMA Classroom (Knoll 217)

Lab

Tuesdays, 6:00 - 7:50 PM
CCRMA Classroom (Knoll 217)

Office Hours

Monday 3:30-5:30 PM
Thursday 4:30-5:30
CCRMA Ballroom

Questions

Post on Piazza

Instructors

Poppy Crum
Instructor
poppy(at)stanford(dot)edu

Cara Turnbull
Teaching Assistant
cmt802(at)stanford(dot)edu