Lab 1

All portions of this lab are due on Tuesday, April 14th. Groups will be assigned in class on Friday, April 10.

Matlab/Python

If you are completely new to Matlab, there are a couple tutorials on the resources page that will will help you get acquainted with this powerful tool. Even if you have used Matlab, if you haven’t used it for audio-related tasks the tutorials can be useful.

If you are using Python, please be aware that there are a variety of libraries to play sound. For instance, the sounddevice library is simple and easy to use, and is well-documented, although it will not write audio files. There are multiple libraries that contain the functionality to write .wav files, including scipy.io.

If you are using Python and prefer creating Jupyter Notebooks to standalone Python scripts, please put all your functions in the same notebook.

Part 1: I Saw the Sine

Create the Matlab or Python function:

output = sineTone(fs, frequency, duration)

This function will generate an audio-rate sampled sine wave (sinusoid). Save the function in your working folder as sineTone.m if working in Matlab or sineTone.py if working in Python. The parameters passed are:

  • fs - the sampling rate, in Hertz
  • frequency - the frequency of the sine wave, also in Hertz
  • duration - the length of the sine wave, in seconds, when played at the sampling rate

The output will be a one-dimensional array or list of float values between -1.0 and 1.0 (It’s actually a good idea to keep audio samples slightly less than +/- 1.0, so +/- 0.99 might be better. You could alternatively add an argument to the function for amplitude).

Other things to know:

  • A sinusoid can be expressed by the equation: $y = \sin (2 \pi f t)$, where $f$ is frequency and $t$ is the current time.
  • Since we are using digital signals, we can define time $t$ as $\frac{n}{f_s}$, where $f_s$ is our sampling rate and $n$ is the sample number.
  • How many samples long should your output vector be given the duration and sampling rate fs?

Part 2: Ramping Up (and Down)

Create two functions:

  1. output = rampUp(fs, duration, input)
  2. output = rampDown(fs, duration, input)

These functions should be able to take the output of your sineTone function (or the function itself) as the input argument. rampUp should fade in the beginning duration seconds of the input, and rampDown should fade out the last duration seconds of the input.

Parameters:

  1. input - the input to which to apply the ramp. Experienced Matlab users: can you generalize your function to handle any number of channels (i.e. a number of channels by number of samples matrix input)?
  2. fs - sampling rate, should be the same sampling rate you used to generate the input.
  3. duration - the length in seconds of the ramp.

Notes:

  • These functions are creating an envelope: a series of numbers from 0.0 to 1.0 that will scale our signal by specific amounts at specific times. Note here since we’re dealing with scaling, we do want to go all the way up to and including 1.0 as a value.
  • How long should the ramp be given the duration and sampling rate fs?
  • Lookup the Matlab function linspace(a,b,n) (for those of you new to Matlab, type help linspace in the console) - it creates a n-length array of numbers between values a and b. This function might be particularly useful in creating your rampUp and rampDown functions…
  • The Python equivalent of linspace(a,b,n) is in the NumPy library, and works just the same as the Matlab version.
  • Indexing in Matlab uses parentheses, not square brackets, and arrays are 1-indexed not 0-indexed. So indexing the first n samples in a Matlab array looks like: array(1:n). You can use the end keyword to denote the end of an array, so indexing the last n samples in a Matlab array looks like: array(end-n+1:end).
  • Indexing in Python uses square brackets, and arrays are 0-indexed, so the first n samples in a Python list looks like: list[0:n]. Omitting the number after the colon defaults to the last element, so indexing the last n samples in a Python list looks like: list[len(list)-n:].
  • Matlab was originally built for matrix math, so the * operator defaults to the matrix multiplication operator. To do element-wise multiplication, use .*.
  • Python, on the other hand, defaults to element-wise operations for all forms of arithmetic.

Save your functions as (you guessed it) rampUp.m and rampDown.m or, if you are using Python, rampUp.py and rampDown.py.

Part 3: Complex Sounds!

Make at least one function that combines multiple sine waves to make a more complex sound! You can add a bunch of sine tones together, multiply them, use one to change the frequency of another, or anything else you can think of. Keep in mind that adding sine waves of the same frequency doesn't change the sound except by making it louder, and that the phase of the sine waves will matter in most of these functions. For inspiration, look into how to build sawtooth, square, and triangle waves and how FM synthesis works, or just play with different ways of combining multiple sine waves. Have fun making weird sounds!

Save your code as complexTone.m for Matlab or complexTone.py for Python.

Part 4: What’s it Sound Like?

Use your newly created functions to create some smoothly fading in and out sine tones! In a separate script, try making a 5 second sine tone at a frequency of 200 Hz with 1 second fade in and out times. Use 44100 Hz as the sampling rate (48000 Hz is the other common audio sampling rate). You can nest your functions, so you should be able to create your sine tone in a single line with something like:
s = rampDown(44100, 1, rampUp(44100, 1, sineTone(44100, 200, 5)));
(Note the ; at the end suppresses text output in Matlab, so unless you want hundreds of thousands of sample values printing in the console, you’ll want to use it).

To listen to your sine tone, use the sound() Matlab function (look it up!) or the sounddevice.play() Python function. You’ll want to pass in the sampling rate you used to generate your sine tone along with your generated sample vector output.
Try different frequency and duration combos to your hearts content! If you’re feeling adventurous, try experimenting with logspace() instead of linspace() in your ramp functions and see what happens to the fades. Just make sure you change back to linspace() before submitting.

Part 5: What’s it Look Like?

Plot the resulting output from your functions using Matlab’s plot() function or Python's matplotlib.plot() function. 5 seconds of 44100 Hz sampling rate is way too many samples to see clearly what’s happening, so let’s use 0.1 seconds as the duration for your sineTone and complexTone, 0.01 seconds rampUp duration and 0.04 seconds rampDown duration. Stick with 200 Hz as the sineTone frequency, but feel free to experiment with the frequencies in your complexTone. Once you’ve got the signal vector, plot it with plot() - does it look how you’d expect? Remember that in Python, you have to include a show() call after your plot() call to see the plot! Save your plots to .pdf files and submit them with your functions.

Matlab/Python Deliverables:

  1. Your sineTone.m or sineTone.py file from Part 1
  2. Your rampUp.m or rampUp.py file from Part 2
  3. Your rampDown.m or rampDown.py file from Part 2
  4. Your complexTone.m or complexTone.py file from Part 3
  5. [OPTIONAL] If you wrote your functions in a Jupyter Notebook, please submit that notebook instead of the separate function files
  6. Your plots as .pdf files from Part 5

Note you do not need to submit your script from Part 4


Game Development: Sonified Pong

Take Pong as an inspiration for your first video game. In a group of 3-4 we’d like you to design, program, and test a new version of pong with a focus on sound!

You can do anything you want with the Pong concept! You could add more players, add or modify game mechanics, add new artwork, make sound spatialized, employ frequency spectrum as a game mechanic, or create something you could play with your eyes closed! Go crazy! Just remember to focus on sound design and think about the how the sounds you are using can give cues to the player.

For your first foray into building a video game, we recommend using GameSalad, an easy-to-use drag and drop game creation engine. Its simplicity will allow you to focus less on the technical aspects of game creation and more on the design components used to make a game and how to create and manipulate those components to create compelling gameplay. To get set up with GameSalad, please see the resources page. We’ll go through an example project in class, but you are welcome to check out the beginner tutorials on the Official GameSalad Tutorial Site to get a feel for how it works, or here’s a short tutorial on starting out with GameSalad from this class a few years ago. Also check out the GameSalad Cookbook.

We have provided a base Pong program in GameSalad, but if your group would like to use a different framework or programming language feel free. Some other frameworks are Unity (here’s a 2D game development walkthrough) or Processing. We will focus on Unity starting with the next lab, so for now you should only use a different framework if you have prior experience with it.

Use your newly expanded library of Matlab and/or Python functions to create sounds for your game (for at least the majority of sounds - if you want to include a few other audio clips that’s ok, if they fit with a theme for example). You can write your tones to a .wav file using audiowrite(signal, fs, filename); in Matlab or scipy.io.wavfile.write(filename, fs, signal) in Python. If necessary, you may edit your sounds in a DAW such as Audacity, Logic, etc., but only for basic editing. For now, do not use instruments, other sound generating plug-ins, sound effects, or other processing plug-ins for your game sounds.

You must go through each step of the design process and finally hold a playtest. Pay attention to the mechanics you are employing in your game. What can a player do at each point in the game? Are each action and the overall gameplay intuitive? Most importantly, what is the overall goal the player is trying to achieve?

The Design Process

  • Inspiration
  • Design Loop
    • State Problem
    • Brainstorm
    • Prototype
    • Assess
  • Implement
  • Playtest
  • Revisit Design Loop as Necessary

If possible, conduct your playtest with people who are not in the class. This may be difficult given the social distancing guidelines, so if it is not feasible for you, that is all right.

Game Design Deliverables:

Submit links to your pitch and playtest videos using the guidelines detailed on the Lab Overview page.

Write up a short design document (100-250 words) discussing how you used sound in your game and how you think it contributes to the game experience. See if you can tie in topics we’ve discussed in lecture (loudness, masking, expectation, or anything about the auditory pathway that you’ve learned so far). In your design document you should also address what you learned through the playtest. One design document per group is sufficient.

Submit a zipped folder that contains the game program (e.g. the GameSalad project) and your design document.

Lecture

Fridays, 10:30 AM - 12:20 PM
Zoom

Lab

Tuesdays, 6:00 - 7:50 PM
Zoom

Office Hours

By appointment
Zoom

Questions

Post on Piazza

Instructors

Poppy Crum
Instructor
poppy(at)stanford(dot)edu

Noah Fram
Teaching Assistant
nfram(at)stanford(dot)edu