Homework: Data Sonification

  • Out: Oct 4, 2022 Tuesday
  • Due: Oct 18, 2022 Tuesday

Objectives

  • Establishing a workflow for editing and testing algorithms in webchuck
  • Exploring Chuck unit generators (UGens) and applying simple oscillators (with frequency, amplitude and timbre dimensions)
  • Understanding data mapping functions and iteration in Chuck
  • Exploring data sonification

Key Results

  1. Local and server-side code maintained in sync (same files on your own computer and in your CCRMA 220a account).
  2. Synthesizer using one or more oscillator (or other sound producing) UGens.
  3. Time series of your choosing performed by the synthesizer.
  4. Data mappings between the time series data and synthesis parameters explored.
  5. Final work uploaded to the homework factory.

Browsers and Caches

Use Firefox, Chrome or Brave (these have been tested with webchuck). Beware of browser caches where what you see is not up to date with what you've changed. For example, suppose you've made an edit and displayed or uploaded it, but there's no effect from the change that was made to the file. The browser may still be hanging on to the earlier version when displaying the page. To force it to forget the older version, use the Ctrl + Shift + Delete command and confirm your intent to "clear" recent history. Then do a Ctrl + r to refresh the page. (On Mac, substitute Cmd for Ctrl in these commands.)

Lab -- work locally

We'll use the sonification starter project from the Music 220A code repository downloading a working copy to your own machine. After that you'll replicate it to a server-side copy in your CCRMA 220a directory.

Open a terminal and create a local 220a directory on your own computer just the way you created one on the server when completing hw0.

mkdir 220a
cd 220a

Copy the data-sonfication example directory from the course's github repository into your local 220a directory and call it hw1. That's the copy you'll work with to develop your homework solution. After copying from the repo, list its contents (there should be two files).

svn export https://github.com/ccrma/music220a/trunk/webchuck/data-sonification hw1
ls hw1

Replicate it from your local 220a directory to your 220a directory on the CCRMA server using a recursive secure copy. This is the local hw1 being copied over the network to your CCRMA directory.

scp -r hw1 USERNAME@ccrma-gate.stanford.edu:Library/Web/220a/

The above command requires substituting your CCRMA USERNAME (if your username is the same locally then you can elide USERNAME@

Next, test that you can play the CCRMA server version by going to the homework factory and testing the link for hw1. You may have to refresh the homework factory page to see it. The version of hw1 that's on the homework factory is just a placeholder, that's what you'll be updating later in the assignment.

It's also where all time series data files need to be stored. For security reasons, our web pages aren't permitted to access auxiliary files from your local computer. Setting up a local development server on your machine is fine if you'd like to work that way but beyond the scope of what's being shown here.

After replicating the local copy to the server and testing it, verify that the local version plays ok. Open with a browser command like this (or just double-click on index.html in a file browser)

firefox index.html

Ok, to recap -- you've got a working directory locally and a placeholder hw1 on the CCRMA server. They both play the same time series and code. Onward to making some personal customizations.

Live code experiments

The webchuck page index.html has a live chuck code editor like the one in the page for hw0. Experiment with the suggested modifications.

Live code modifications can't be saved directly from the page. To save, manually copy and paste to the local version of index.html file in an editor, save, and then refresh the page in the browser.

Really Quick Tutorial - Chuck script

Let's break it down. What does this do?

  SinOsc osc => dac;

SinOsc is a class of oscillator and we're instantiating one of those and naming the instance, osc

dac is a special UGen which is our sound output, aka digital-to-analog converter

osc is wired to dac so we can hear the tone

But for how long? nVals is the number of data points in the time series.

  for (0 => int i; i < nVals; i++) {
    ...
    100::ms => now;
  }

The script completes after the last iteration. Total duration = nVals * 100ms

At each iteration the frequency of the oscillator gets updated.

    osc.freq(vals[i]*1000+200); // data values in range 0 - 1

See Chuck UGens documentation for more.

The rest of index.html - overview

That's a web page? Looks more like a Frankenstein concoction. But not to worry, there's logic in this madness.

  1. <head> block
    1. make it responsive for mobile devices
    2. includes the ACE editor and webchuck stuff
    3. specifies the dataset (time series) file and a name for the data
  2. title, graph display area, project description
  3. fill Chuck editor blocks with Chuck code
    1. suggestions
    2. live code script
  4. button elements
  5. JavaScript code
    1. get display elements from the page and set any options
    2. add any hidden Chuck code to be prepended to the script above
    3. define button press events
    4. define dataFilter (see below)
    5. read data file, filter the data and send to Chuck
    6. define graph drawing functions
    7. draw the graph

Handling Time Series Data

Now is the time to pick interesting datasets to sonify. You can choose whatever you want, but an easy place to start is Google Trends. Search a term and download a CSV file from the chart. Be sure to explore various sources for interesting time series data.

Let's assume that the new .csv file is downloaded to your local computer to the hw1 working directory. In order to use it in your sonification, a copy needs to be served from the server directory. Use the scp command again, same as above, to replicate what you have locally to the server

scp -r hw1 USERNAME@ccrma-gate.stanford.edu:Library/Web/220a/

The above command requires substituting your CCRMA USERNAME (if your username is the same locally then you can elide USERNAME@

To tell your script about the file, edit the local version of index.html so that dataURL corresponds to your CCRMA USERNAME, the file name DATAFILE and the dataName Y-LABEL (which will label the Y-axis of the graph).

  <script>const dataURL = 'https://ccrma.stanford.edu/~USERNAME/220a/hw1/DATAFILE.csv';</script>
<script>const dataName = 'Y-LABEL';</script>

Formatting and Sanitization - JavaScript

We need to process the downloaded CSV file to use with the synthesizer (and draw a simple graph), so we need some code to format, sanitize, and filter the dataset. D3 is very useful for this purpose, and it is included in index.html.

The task here is to edit (or write from scratch) dataFilter() function. The following is the starting example which leverages code found here.

// define dataFilter
const dataFilter = (row) => {
// The target data set has 2 columns. Skip the row otherwise.
if (row.length !== 2) return null;

// Arranges the given row into a preferred form.
// The incoming source data could be in different formats depending on the project.
const filteredRow = {
// The first column (row[0]) produces a decimal year in the example.
// -- otherwise, if date is in strings, like "2021-09-26",
// swap in this next form to convert the string to decimal year
// year: (1969.5 + (Date.parse(row[0])) / (31556925993.6) ),
year: parseFloat(row[0]),
// The second column (row[1]) produces the actual value.
value: parseFloat(row[1])
};

// Sanitizing; if the row contains a value that is negative or not a number, remove it
// from the dataset.
for (const column in filteredRow) {
const value = filteredRow[column];
if (value < 0 || isNaN(value)) return null;
}

return filteredRow;
};

Mapping parameters

D3 is helpful in transforming different domains/ranges to one another. In the index.html js code, the time series values get rescaled for use in the Chuck script and transformed for graphing. This documentation might be useful for further exploration.

The Chuck script handles mapping of data values to musical dimensions (for example, frequency). For convenience the rescaled data range is 0 - 1, something that makes it easier to apply the time series values across a variety of musical dimensions (not just frequency, but also amplitude, timbre, etc.).

Create a Composition

Once you have the synthesizer and have mapped the dataset, now it is time for your creativity. Aiming for 1 minute and 30 seconds, try to draw out the character or the unique quality of a dataset. Or simply make some music with the material.

The simple example provided is monophonic in that it only plays one "voice" at a time. The oscillator gain is by default set to the maximum amplitude -- 1.0. If adding more sound generators / oscillators that play at the same time, be sure to reduce gains, so that the total doesn't exceed the 1.0 maximum signal, otherwise clipping distortion will occur.

This a non-exhaustive list of variables for experimentation in monophonictexture

  • The number of generators
  • The frequency, amplitude and timbre of generators
  • The harmonic relation between generators
  • Generator event timing depends on data value

Exploring polyphonic textures

  • Generator start times differ (staggered patterns)
  • Each generator's tempo or event timing gets set differently (so generators' events aren't at the same time))
  • Generators play different patterns, best if the patterns are sporked as separate shreds See Concurrency & Shreds in the Chuck documentation and the live examples spork.html and sporkTwice.html included in the starter project directory you've already downloaded.

Describe your composition with some text added to index.html:

  • Title of your composition
  • Total duration
  • Thoughts expanding on your choice of dataset(s)
  • Give some detail on your design of synths and mapping functions

Submitting HW1

That's already been quite a bit of work. This one last step is to replicate the local directory to the homework factory. The same terminal command as before does the job -- this time overwriting the placeholder files and so that now the homework factory link will publish the edited project. From your local 220a directory use scp to make a recursive copy of the local hw1 to the server

scp -r hw1 USERNAME@ccrma-gate.stanford.edu:Library/Web/220a/

The above command requires substituting your CCRMA USERNAME (if your username is the same locally then you can elide USERNAME@ Check that the submission link appears correctly in the homework factory. Feel free to update the contents whenever you like (the timestamp always shows the latest version).