Homework 2: Radioplay

due on 10/23 (Thr) 10:00am in the Homework Factory

Overview

The assignment is to create a short radioplay with recorded text, quad audio and various sound effects. Shoot for a duration ca. 120 secs. The turned in file will be a binaural recording intended for headphones. Techniques include localization with interaural intensity difference (IID), interaural time difference (ITD), Schroeder-style reverb (e.g., NRev), and processing for time and pitch transposition (e.g., SndBuf rate and PitShift, respectively).

Background

True binaural recording uses stereo mics pointed outwards from inside your ears to capture as closely as possible the exact sound pressure waves entering your ear canals. The binaural technique captures filter (transfer function) differences caused by body parts shadowing and reflecting sounds arriving from various directions: the ear flaps (pinnae), head, shoulders, etc. Played back over headphones or earbuds, binaural preserves the IID and ITD cues which are basic to sound localization. We'll modulate sound locations in the quad speaker setup using panning control written in ChucK. We'll then process those quad signals into a stereo binaural file for posting to the homework factory.

Early work in binaural recording was accompanied by predictions that its superior imaging would create a world where everyone would eventually listen through headphones. A few decades later headphones and earbuds are ubiquitous but a huge amount of material in binaural has yet to happen. The earbud phenomenon is begging for binaural content. Playing binaurally-encoded sounds over stereo loudspeakers doesn't result in either good binaural or good stereo and that's one thing holding back wider use. For a position paper on where this may be going see Jens Blauert's AES Heyser Lecture. He makes a provocative case for binaural as a part of an increasingly realistic synthetic world combined with many other contributing modalities.

One artist whose work leverages the medium is Janet Cardiff. She composes site-specific 3D audio narratives with spine-tingling interplay of real and phantom presences, binaurally-produced. Her telephonecall from SFMOMA 2001 is a benchmark piece which opens up the possibilities of what you might expect to compose with mobile devices. The composition led participants by the nose through the gallery, each directed by a camcorder which they were holding while watching with a pre-recorded self-guided tour. You'd turn a corner and someone in your earphones would be there singing in the space (acoustically convincing, so that you could point to them) only they weren't there then, but at some other point in time, past, alternative present, or future.

Our approach starts with composing spatialized sound for the quad speaker arrangement. Pick a short text which might be a monologue, group dialog, or whatever you want, but it should constitute some sort of a script (feel free to write it from scratch). You'll first record your own voice (with your new mic) and then possibly other voices in combination depending on the text to be read.

If it's a dialog, invite others to read or if you're theatrically inclined, use your voice for the different characters (the PitShift ugen can help distort your identity). First, second or third person narratives are all fine and we'll get a variety from the class.

Hint 1: timing in the dialog track of a radioplay is different than straight reading and you'll need to leave gaps which will provide space for sound effects where appropriate. Pace your reading accordingly.

Hint 2: this assignment involves many, many intermediate files, so invent a descriptive naming scheme and stay consistent.

Creating Source Material

Use a decent soundcard. High-quality audio is essential for this assignment and most laptop or mobile mic inputs are too noisy. Please use an outboard soundcard like those available at the CCRMA workstations.

  1. Start Audacity.
  2. Set up the preference for mono recording from your mic:
    Preferences > Devices > Recording > Channels > 1 (Mono)
  3. Record a dialog track reading the text you've chosen. If more than one character is needed, then read these other voices subsequently into separate tracks, using Audacity's overdub mode:
    Preferences > Recording > Overdub(Check)
  4. Monitor previously recorded tracks via headphones so they don't bleed into the new track.
  5. You generally want to consider normalizing the files if there's an undesired discrepancy in levels. Select a track and apply normalization:
    Effect > Normalize...
  6. With just your own voice, overdub one more track imitating any sound effects that go with the text.
  7. Then, export the tracks into separate mono files. Select and export one by one:
    File > Export Selection...
  8. Save your Audacity project and quit. Save often!

Localization, Sound Effects, Time-warping, and Transposition

Listen again to your original, unpanned, dry, vocal imitations sound effects track. Gather sources and create new tracks containing the real sound effects, recording or downloading as needed.

Modify this material using two options:

  1. Audacity's effects
    : see Audacity's Effect menu (online reference)
  2. Passing through ChucK
    : see ChucK code examples in the course code repository, subdirectory effects
    • There are various templates for processing: pit(pitchShift), chorus, cub(cubic distortion), pan(IID, ITD, distance), reso(resonator), stretch(pitch shift with speed change).
    • These are variously labeled as: ADAC(realtime I/O), FileDAC(file to realtime output), File(read/write file).
    • Note: A good way to pan in ChucK is to shred the DBAP4.ck file and pass through a portion of a wav file that you would like to pan between 2-4 channels with this file: DBAP4_ex.ck. You can record the output of ChucK into a new Audacity file, like we did in Homework 1, and carefully fold your panned output into your main Audacity radioplay project.

Pan and reverberate each sound effect track as appropriate for your radioplay.

Final Assembly

First, think through the work flow in reverse:

  • Ultimately, there's going to be a binaural stereo final product (for the homework factory submission). To create it, the last step is to play the content you've developed in four channels into the binaural.input inputs while recording the stereo binaural output. (Refer this instruction for converting 4-channel chuck file to binaural recording.)
  • A four-channel chuck player is needed that plays your whole project. This can be in the style of either:
    1. A simple four-channel soundfile player
      : for this configuration, you need four finished mono soundfiles, one per channel. (4x1, four mono soundfiles)
    2. A ChucK "score" that plays the whole radioplay.
      : the "score" will play all four channels itself, performing the entire finished product. (one four-channel "score")

Deliverables

These files are required to be visible on your 220a directory by due date.

  • hw2.html
    with a link to the final binaural wav file. Also make links to other wav files that were used to create the original 4-channel version.
  • ChucK codes and sound files (choose one depending on your style)
    1. Four mono sound files and a ChucK code used to play them in 4 channels.
    2. One four-channel chuck "score" with sound files.