Lab 4

Matlab/Python: Basic Filtering

A filter is a system through which a signal can be passed to change the amplitude and phase of any sinusoidal component of that signal. Four main basic filter types exist:
1) a low-pass filter, which lets low frequencies pass through but cuts out high frequencies (also called a high-cut filter).
2) a high-pass (or low-cut) filter, which lets high frequencies pass through but cuts out low frequencies.
3) a band-pass filter, which allows frequencies through in some band in the middle of the spectrum, but filters out high and low frequencies.
4) a notch (or band-stop) filter, which allows all frequencies through except frequencies within some band in the middle of the spectrum.
The frequency at which a filter starts reducing the response (or around which the pass or stop band is centered for band-pass and notch filters) is called the cutoff frequency.

We’re going to design some basic filters just to give you some practice filtering audio signals.

First, we need to create a signal we can filter. A good signal to use for this is random noise. This will also give us a chance to add a new function to our toolbox for generating noise, which can be coupled with sounds you’ve already generated to deepen your sonic palette - many sounds can be modelled as a combination of tonal components and noise (often filtered noise) components.

Design a function that generates white noise:

[output] = generateNoise(fs, duration)
fs: sampling rate
duration: length of noise signal in seconds

White noise can be generated by simply creating a stream of random numbers between -1.0 and 1.0 (although we know by now it’s best to avoid -1.0 and 1.0; if you want to include an amplitude argument in your generateNoise(), good on you!). A handy Matlab function for generating random numbers is the randn() function.

Plot the spectrum of your white noise signal using your getSpectrum() function from last lab. Try different lengths / durations. What do you see in the spectrum for your noise signal? Why does white noise make a good signal for testing filters? Answer in script comments.

Next look up the Matlab functions butter and filter. You’ll see that the outputs of one can be used as arguments for the other. These vectors (B and A by convention) are the coefficients of the filter transfer function numerator and denominator (if you want to learn more, here’s a good place to start, or take Music 320A/B and / or EE 264). Use these two functions to design and implement the following filters for the given signals:

• White noise low-pass filtered with a cutoff frequency of 800 Hz.
• White noise high-pass filtered with a cutoff frequency of 1500 Hz.
• A short clip of a song of your choice band-pass filtered with parameters of your choice.
• A short clip of a song of your choice notch filtered with parameters of your choice.

Matlab/Python Deliverables:

Submit your generateNoise() function and test script code. Also submit spectrum plots and .wav files of the four filtered signals listed above. Remember audiowrite() can be used to generate .wav files from signal vectors.

Game Development: Biofeedback and Alternate Controls in Mixed Reality

Develop a game in AR or VR using some alternate form of control, especially types of biofeedback. Since we do not have access to the sensors at CCRMA this quarter, either make use of sensors you have such as fitness sensors or draw on the alternate controllers like those you used in Lab 2. You may either build on the AR games you built for Lab 3 or develop an entirely new game for this assignment; just be sure to close the loop in the AR/VR environment.

If you are using hardware, you will need to make sure your Unity game can receive data from it. Each manufacturer has a different architecture for sending and receiving data to and from external hardware which can vary in their integration with Unity's development pipeline.

For software, asset packages such as the RAGE Project's EmotionDetectionAsset and ARKit's eye tracking feature (implemented in AR Foundation) are excellent options that make good use of face tracking. You can also make creative use of other data sources within your smartphones. However, for these approaches, keep in mind that your control mechanism should include some physical manifestation of a cognitive process. Think about what data you can collect to determine someone's focus, emotional state, or stress level, or how to design a control system to manipulate those affective states in your players.

Also, now that we’ve learned about spectrums in Matlab, you can also leverage this knowledge in Unity. You can call GetSpectrumData() on any AudioSource object to get frequency / spectrum information from that source’s audio stream for the current game frame. Rather than return a vector of values, however, GetSpectrumData() populates an array that you’ll need to pre-allocate before calling, which has to have a length that is a power of 2 between 64 to 8192 inclusive. Syntax for allocating a float array of length 256 in C# looks like this:
float [] myArray = new float [256];
It’s best to allocate the array once - declare it with your script variables and allocate it in Awake() or Start(). You can then populate your array using GetSpectrumData() like this:
mySource.GetSpectrumData(myArray, 0, FFTWindow.Hanning);
0 is the channel - could also be 1 for stereo sources but 0 is safest. The third argument is the window applied to our time samples, which we haven’t talked about. You can use any of the other window types you like, but will probably get the best results with the Hanning or Blackman windows. We’ll go over GetSpectrumData() in class also in case anything’s unclear.

Game Design Deliverables:

Submit links to your pitch and playtest videos (details on the Lab Overview page if you’ve forgotten). You do not need to submit the actual Unity project. Be aware that screen capture may not be the best option for this Lab.

Both individual Matlab and Group Game Development portions of Lab 4 are due Friday, May 8th by 10:30AM. We will play your games in class that day. Please plan on coming to class a little early to make sure you’re set up for Gameplay.