Sonification

Coding Project

Objectives

Key Results

  1. Code experimentation with codesandbox (forking, customizing, sharing)
  2. Synthesizer using oscillators (or other sound producing) UGens.
  3. Time series of your choosing performed by the synthesizer.
  4. Data mappings between the time series data and synthesis parameters explored.
  5. Final work shared with codesandbox project link.

Browsers and Caches

Use Firefox, Chrome or Brave (these have been tested with webchuck). Beware of browser caches where what you see is not up to date with what you've changed. For example, suppose you've made an edit and displayed or uploaded it, but there's no effect from the change that was made to the file. The browser may still be hanging on to the earlier version when displaying the page. To force it to forget the older version, use the Ctrl + Shift + Delete command and confirm your intent to "clear" recent history. Then do a Ctrl + r to refresh the page. (On Mac, substitute Cmd for Ctrl in these commands.)

Live code experiments

The pages shown here (NOAA CO2 level sonification) and here (Chris Harley's tidal data) have a live chuck code editor. The former is a one-click, one-file project that should run anywhere and the latter is a codesandbox project for developing code. Experiment with the suggested modifications.

Live code modifications can't be saved directly from the page. To save, manually copy and paste the mods to the version in the code editor, save, and then refresh the page in the browser.

Really Quick Tutorial - Chuck script

Let's break it down. What does this do?

  SinOsc osc => dac;

SinOsc is a class of oscillator and we're instantiating one of those and naming the instance, osc

dac is a special UGen which is our sound output, aka digital-to-analog converter

osc is wired to dac so we can hear the tone

But for how long? nVals is the number of data points in the time series.

  for (0 => int i; i < nVals; i++) {
    ...
    100::ms => now;
  }

The script completes after the last iteration. Total duration = nVals * 100ms

At each iteration the frequency of the oscillator gets updated.

    osc.freq(vals[i]*1000+200); // data values in range 0 - 1

See Chuck UGens documentation for more.

The rest of index.html - overview

That's a web page? Looks more like a Frankenstein concoction. But not to worry, there's logic in this madness.

  1. <head> block
    1. make it responsive for mobile devices
    2. includes the ACE editor and webchuck stuff
    3. specifies the dataset (time series) file and a name for the data
  2. title, graph display area, project description
  3. fill Chuck editor blocks with Chuck code
    1. suggestions
    2. live code script
  4. button elements
  5. JavaScript code
    1. get display elements from the page and set any options
    2. add any hidden Chuck code to be prepended to the script above
    3. define button press events
    4. define dataFilter (see below)
    5. read data file, filter the data and send to Chuck
    6. define graph drawing functions
    7. draw the graph

Handling Time Series Data

Now is the time to pick interesting datasets to sonify. You can choose whatever you want, but an easy place to start is Google Trends. Search a term and download a CSV file from the chart. Be sure to explore various sources for interesting time series data.

Let's assume that the new .csv file is downloaded to your local computer to your working directory. In order to use it in your sonification, a copy needs to be served from some server directory, in our case, from codesandbox.

Use the codesandbox upload feature or drag and drop the file into your codesandbox project.

To tell your script about the file, edit index.html so that dataURL corresponds to the appropriate FILENAME.csv and specify an appropriate DATANAME (which will label the Y-axis of the graph).

  <script>const dataURL = 'FILENAME.csv';</script>
  <script>const dataName = 'DATANAME';</script>

Formatting and Sanitization - JavaScript

We need to process the downloaded CSV file to use with the synthesizer (and draw a simple graph), so we need some code to format, sanitize, and filter the dataset. D3 is very useful for this purpose, and it is included in index.html.

The task here is to edit (or write from scratch) dataFilter() function. The following is the starting example which leverages code found here.

// define dataFilter
const dataFilter = (row) => {
  // The target data set has 2 columns. Skip the row otherwise.
  if (row.length !== 2) return null;

  // Arranges the given row into a preferred form.
  // The incoming source data could be in different formats depending on the project.
  const filteredRow = {
    // The first column (row[0]) produces a decimal year in the example.
    // -- otherwise, if date is in strings, like "2021-09-26", 
    // swap in this next form to convert the string to decimal year
    // year: (1969.5 + (Date.parse(row[0])) / (31556925993.6) ),
    year: parseFloat(row[0]),
    // The second column (row[1]) produces the actual value.
    value: parseFloat(row[1])
  };

  // Sanitizing; if the row contains a value that is negative or not a number, remove it
  // from the dataset.
  for (const column in filteredRow) {
    const value = filteredRow[column];
    if (value < 0 || isNaN(value)) return null;
  }

  return filteredRow;
};

Mapping parameters

D3 is helpful in transforming different domains/ranges to one another. In the index.html js code, the time series values get rescaled for use in the Chuck script and transformed for graphing. This documentation might be useful for further exploration.

The Chuck script handles mapping of data values to musical dimensions (for example, frequency). For convenience the rescaled data range is 0 - 1, something that makes it easier to apply the time series values across a variety of musical dimensions (not just frequency, but also amplitude, timbre, etc.).

Create a Composition

Once you have the synthesizer and have mapped the dataset, now it is time for your creativity. Try to draw out the character or the unique quality of a dataset. Or simply make some music with the material.

The simple examples provided are monophonic in that they only play one "voice" at a time. The oscillator gain is by default set to the maximum amplitude -- 1.0. If adding more sound generators / oscillators that play at the same time, be sure to reduce gains, so that the total doesn't exceed the 1.0 maximum signal, otherwise clipping distortion will occur.

This a non-exhaustive list of variables for experimentation in monophonictexture

Exploring polyphonic textures

Happy sonifying!