Lab 3

Matlab/Python: Spectrum Analysis

In the Matlab/Python portion of this lab, we’re going to perform some basic spectral analysis using the Discrete Fourier Transform. You may recall from the last lab that we said we could create any sound by adding together sinusoids, and sinusoids are the most basic building blocks of sound. Here we are going to create a function that takes any audio signal and tells us the sinusoids that create that sound.

The Fast Fourier Transform (FFT)

The Fast Fourier Transform (FFT) is an algorithm for computing the frequency content of a time series. For real signals (like audio signals), the magnitude of the first and second halves of the values returned by the FFT will be symmetrical, or even – you can think of the second half corresponding to negative frequencies. In other words, the absolute value of the output of an FFT is a list of the amplitudes of sinusoidal components evenly spaced from 0 Hz to half the sampling rate (fs/2 Hz), and then from -fs/2 to 0 Hz, respectively. The negative frequency component amplitudes are always going to be the same as the positive frequency components, as long as we are not feeding complex numbers into the FFT. So we really only care about the first half of the values returned by the FFT, since the second half is redundant for real signals.

You do not need to be familiar with the specifics of signal processing to do this assignment, but it is important to know that the FFT returns both positive and negative frequencies to accurately plot your results. The function you write in either Matlab or Python should only return the positive components and their corresponding frequencies.

Matlab

Create the following function:
[Y, F] = getSpectrum(input, fs)
Input variables:
input = any input signal
fs = sampling rate
Outputs:
Y = amplitudes of sinusoidal components
F = corresponding frequencies for amplitudes in Y

Notice this function has two outputs: Y and F. Matlab functions can output an array of variables, which is why the return values must be listed in square brackets. Y is a list of amplitudes of sinusoids spaced evenly from 0 Hz to fs/2 Hz non-inclusive, i.e. [0:fs/2). F contains the frequencies that correspond to each amplitude in Y.

Take a look at the documentation for the Matlab function fft(y,N). This is the fast Fourier transform, which takes a signal from the time domain into the frequency domain, outputting complex values for individual frequency components (sinusoids) of a given input signal. We are interested in the magnitude of these complex numbers, corresponding to the amplitudes of the sinusoids, which you can get with abs(). N is the number of sinusoids that fft() will return, spaced from (0:fs] - this corresponds to the size of the FFT window. The higher the value of N, the higher the frequency resolution of the spectrum. If N is larger than the number of samples in the input signal, the signal will be zero-padded, meaning zeros will be added to the end of the signal. A good number to use for N is the next power of two larger than the length of the input signal (check out nextpow2()).

Python

Create the following function:
def getSpectrum(input, fs)
Input variables:
input = any input signal
fs = sampling rate
Outputs:
Y = amplitudes of sinusoidal components
F = corresponding frequencies for amplitudes in Y

In Python, you can return multiple values from a function simply by writing return Y, F. In this function, Y and F will be the same as in the Matlab version.

Unlike Matlab, raw Python does not have a built-in function to compute the FFT. There are also multiple libraries that compute the FFT. For this lab, we will be using NumPy's np.fft.fft() function. The LibROSA library includes a re-implementation of this algorithm, along with a variety of additional utilities for the analysis of music and audio, and is extremely useful for more in-depth applications. However, for the purposes of this lab, NumPy's functions are sufficient.

As with the Matlab portion, the np.fft.fft() function will return a list of complex numbers, and we are interested in the magnitudes of those complex numbers. As in Matlab, the abs() function will return the magnitude. N is the number of sinusoids that fft() will return, spaced from (0:fs] - this corresponds to the size of the FFT window. The higher the value of N, the higher the frequency resolution of the spectrum. If N is larger than the number of samples in the input signal, the signal will be zero-padded, meaning zeros will be added to the end of the signal. A good number to use for N is the next power of two larger than the length of the input signal. There is no root nextpow2 function in Python, so computing this value may involve some logarithms.

Matlab/Python Deliverables:

Submit your getSpectrum() function along with two very short (less than 1 second) .wav samples of your choice (don’t use anything you’ve generated, they should be a music or audio sample), and a plot of the output of your getSpectrum() function with each of the .wav samples as the input signal. Make sure to label your plot! Here’s an example of how to do this:

figure;
subplot(2,1,1);
plot(F,Y1);
title(‘example1.wav’);
xlabel(‘Frequency (Hz)’);
ylabel(‘Amplitude’);

subplot(2,1,2);
plot(F,Y2);
title(‘example2.wav’);
xlabel(‘Frequency (Hz)’);
ylabel(‘Amplitude’);
suptitle(‘Spectrums of Short Audio Samples’);

Game Development: Modified / Augmented Breakout

The game development portion of this lab consists of two parts. The first part you will complete individually, while the second part will be completed in groups.

Solo: Getting Started with Unity

Unity is a game creation engine: a giant collection of tools that handles graphics, audio, controller input, and routines for various scripts and such.

After you download and install Unity, go through this Breakout tutorial independently. Breakout is a classic game (the original was played on the Atari 2600 with a paddle controller) that you may know by the name of Brickbreaker. It’s kind of like Pong, except instead of playing against someone else, you are playing against an endless army of evil bricks.

After you finish the tutorial, change one small thing about the game. It could be the color scheme, or the amount of bricks, or the scoring mechanism, or add your own Matlab-generated sounds! In your submission for the assignment, include a short document that describes what you did to personalize your Breakout along with a few screenshots or link to a short video that shows it off.

Group Breakout, Augmented

Now that you have a basic understanding of the fundamentals of using Unity, build a basic game that uses a specific controller. This game could build off of one of your Breakout demos. Think about what your controller offers you in terms of input - what are the different ways a player can interface with the controller? Are there any real-world gestures that the controller accepts as input that we can translate to an in-game mechanism, giving the player a greater sense of physical presence in the game?

We have Leap motion sensors and a couple Kinects you can use for controllers, or if you have something else in mind feel free to use it! I also have a Tobii eye tracker that only works with Windows machines if someone wants to use it (I can help you set this up). Standard game controllers (like an XBox controller) don’t count as ‘controllers’ for this assignment.

Here are a some links for the Leap Motion (How To, Cool Examples) and Kinect (How To and Another One, Cool Examples).

Game Design Deliverables:

Please submit links to your pitch and playtest videos (details on the Lab Overview page for more information). Please see Canvas page for due dates.

Lecture

Fridays, 9:45 - 11:45 AM PM
CCRMA Classroom (Knoll 217)

Lab

Tuesdays, 6:00 - 7:50 PM
CCRMA Classroom (Knoll 217)

Office Hours

See Canvas
CCRMA Classroom/Discord

Questions

Post on Discord or Email/p>

Instructors

Poppy Crum
Instructor
poppy(at)stanford(dot)edu

Lloyd May
Teaching Assistant
lloydmay(at)stanford(dot)edu