Internet Ensemble Tech Force, Music 153b
Spring, 2020
Center for Computer Research in Music and Acoustics (CCRMA)
Stanford University

Sign up here:
IETF CCRMA email list

Navigate this deck with keyboard or touch

type "m" for the menu of pages and
arrow or page keys to advance or rewind

two-finger tap for the menu of pages and
the usual left/right swipe to advance or rewind

embedded media clips (audio and video) use an extra click (or swipe) to begin playing
and another to finish

lecture recordings

April 6 intro and installation
Session 1 (edited)

April 10 group connections
Session 2 (edited)

April 13 dialing into the sweet spot
Session 3 (edited)

April 17 server side
Session 4 (edited)

     (April 20 hands-on workshop, not recorded)

April 24 simple two-way connections
Session 6 (edited)

     (April 27 registered student meeting, not recorded)

May 1 concerts, testing, bigger servers!
Session 8 (edited)

May 4 documentation workgroup
Session 9 (edited)

lecture recordings (continued)

May 8 guest presentations
Sarah Weaver (edited)
Constantin Basica (edited)

     (May 11 group round-robin discussion, not recorded)

May 15 hub server bug fix
Session 12 (edited)

May 18 guest presentations
John Lee (edited)
Paolo Lattanzi (edited)

May 22 signal processing with Julius Smith and Prateek Verma
Session 14 (edited)

May 29 JMess, soundcards, Raspberry Pi
Session 15 (edited)
SoundWIRE (edited)

June 1 live playing
Session 16 (edited)

June 5 last class, retrospective wrap up tour in retrograde
Session 17 (edited)

This course inaugurates the Internet Ensemble Tech Force which is needed urgently worldwide and locally to support music ensembles going online. Calling it urgent is not an exaggeration and we can provide a valuable service. Course participants will quickly come up to speed on low-latency audio collaboration technology and will then pair with ensembles interested in using it.

153b participants will work from home and be able to use CCRMA facilities remotely. The course can be audited or coordinated with another course. Let's help make group playing possible during this public health challenge and period of 'musical distancing.' Ensemble rehearsals, coaching and concert broadcasting are planned in conjunction with several local ensembles.

The one-quarter course meets online twice a week: Mondays and Fridays at 2:30pm for 50 minutes. The Zoom URL will be emailed to participants ahead of each meeting.

Sign-up: students should enroll in Music 153b, auditors are welcome and can join by contacting Chris Chafe

Listserv: emails will go out to ietf@ccrma.stanford.edu

  • small groups playing together between home locations (say < 12 members)
  • concert broadcast via Youtube Live
  • linux, mac, windows laptops and desktops (also Raspberry Pi)
  • own choice of audio interfaces with mics, etc.

  • larger groups (but we haven't tried)
  • wifi (requires ethernet wire from home router)
  • built-in sound on laptops

  • syncronized video (but try LOLA for that)
  • conducted ensembles
  • mobile devices
  • hosted wine and cheese receptions

Previous course:

Online Jamming, TU Berlin, Winter 2019-2020

Upcoming course in Stanford sequence:

Music 153a Online Jamming and Concert Technology, Autumn 2020-2021

Monday, April 6
  • introduction
  • installation
  • test machine:
  • settings:
    srate = 48kHz, buffer size = 1024, channels = 1
  • command:
    ./jacktrip -C -n1

free online course for background (mostly videos)

(click below) to open https://www.kadenze.com/courses/online-jamming-and-concert-technology-x/info

see CCRMA's web page, under Research : Software : jacktrip for installation guides

(click below) to open https://ccrma.stanford.edu

Session 2

April 10 -- group connections

admin stuff

for ugrad & grad students: zero-unit enrollment -- is possible
read this
let me know via an email you'd like to enroll for zero units because I need to file a request

ping -- the ping utility is turned off (by policy) for all CCRMA servers

I'll let you know if that changes


USB mic works great

qjackctl can be configured to independently choose a USB mic as the input device and a separate output device

Blue Yeti USB mic can be both (it has a headphone jack)

group connections

Before going on the network, do an audio test first:
  • start qjackctl
  • in the connection window, connect mic to speakers or headphones
  • verify that there is signal
  • tap directly on the mic to be sure it's the right mic (not a built-in laptop mic)
  • click on disconnect
  • verify that the signal goes away (otherwise it's not qjackctl that's created the connection and most likely it's a direct sound monitoring path in your sound card)

Which jacktrip mode to run:
  • simple peer-to-peer server
    ./jacktrip -s
  • simple peer-to-peer client
    ./jacktrip -c <IP Address or Name of server>
  • hub (group) server
    ./jacktrip -S
  • client of hub (group) server
    ./jacktrip -C <IP Address of server>
    (-C mode doesn't recognize name, yet)

Connect to jacktrip loopback server:
  • command:
    ./jacktrip -C -n1
  • play or sing a sustained tone
  • listen for dropouts
  • watch for "waited too long" messages
  • too many dropouts?
  • increase the input queue by adding -q<N>, for example, a queue of 8 packets (default = 4)
  • command with queue of 8 packets:
    ./jacktrip -C -n1 -q8

Things that prevent connection
  • mismatch between local and peer sample rate
    --restart qjackctl with same sample rate as peer
  • mismatch between local and peer buffer size
    --restart qjackctl with same buffer size as peer, also called frames / period
  • "waiting for peer" forever
    usually this is because computer firewall or router firewall is blocking incoming data

Matt's explanation of the all those "jacks".

jackd is the actual Jack (jack audio connection kit process -- a daemon)
The "d" stands for daemon
It runs in the background letting other programs on the computer send and receive audio to each other and to whatever audio hardware jack is connected to.

jackd can be started from the command line but is more often launched from JackPilot or qjackctl

JackPilot is a graphical front-end to Jack.
It’s a program that opens a window you can click on, has a "Preferences" window, etc.
JackPilot’s preferences window is where you can:
- select which audio interface you want Jack to use
- select the Sample Rate and Buffer Size that Jack will use
- configure JackRouter
You must set all such preferences before starting Jack.
You can start Jack by clicking "Start" in the main JackPilot window. (On OSX this is the recommended way to start Jack)
On OSX with a normal Jack installation, the JackPilot program lives in
/Applications/Jack, that is, the "Jack" subfolder of the main Applications folder.

Matt's explanation (cont'd)
qjackctl is another graphical front-end to Jack.
"qjackctl" is pronounced "cue jack control" (though Mark Applebaum suggests making it rhyme with "pterodactyl") Its main window is (confusingly) titled "JACK Audio Connection Kit"

On OSX with a normal Jack installation, the qjackctl program lives in
/Applications/Jack, that is, the "Jack" subfolder of the main Applications folder.

The yellow word at the center top of the window will be "Active" if (qjackctl understands that) jack is running, or "Inactive" or "Stopped" otherwise
Please ignore all the green writing (which has to do with MIDI, not audio)

The "preferences" for qjackctl are in the "Setup" window, which you open and close by clicking the "Setup…" button on the main window

The most useful part of qjackctl is the "Connections" window, which you open and close by clicking the "Connect" button on the main window
The connections window shows all the jack-enabled programs currently running through Jack. It lets you connect any output of any program to any input of any program.

On OSX it appears that you cannot start Jack from qjackctl. JackRouter is part of Jack on OSX that lets non-jack-enabled programs (for example iTunes, QuickTime Player, Chrome) connect their audio to Jack.

Session 3

April 13 -- dialing into the sweet spot

admin stuff

the 1-unit course limit can't be changed, use 220d to add 2-4 more units as desired and let me know

I'm editing and streaming the class video recordings from CCRMA.

I've had requests for a commercial streaming service so I simply added links to the raw video on Zoom. These include the chat transcript which isn't anonymous. If that's problematic please let me know after this session.

minimizing delay

maximizing audio quality

maximizing network quality (as much as possible, given best effort network protocols)

latency is the time delay between cause and effect -- used interchangeably

an internet host is a computer on the network -- used interchangeably

a server listens for a client to connect to it -- different jacktrip modes

each jacktrip instance runs a transmitter and receiver -- bidirectional

my local host and a remote peer host -- either can run in either jacktrip mode

our IETF -- owes everything to

Audio quality:
  • buffer size (affects latency)
  • gain
  • feedback
  • number of channels
  • self-monitoring
  • muting other audio applications (Zoom)
  • sharing audio signal between applications (audacity)
  • running Zoom on a second host (mobile or laptop)

Network quality:
  • physical distance and number of intermediate router hops in path (affects latency)
  • playback queue (affects latency)
  • jitter vs. packet loss
  • redundancy for forward error correction
  • wavetable vs. zeroes (very "backwards" error concealment)
  • better error concealment (under construction)
  • network peering locations can be a surprise
  • last mile (fiber, cable, vdsl, microwave, 5G)
  • wifi inadequate but useful for basic testing
  • choke points, competing applications on the same local host and local network, internet weather
  • venues, when we get to use them again

cello demo

acoustic cello with air mic

electric cello

talkback channel, in this case it's Zoom

breakout rooms here we go again, 5 this time


public IP address for ensemble members to connect to

start jackd from command line and jacktrip server in loopback (or other audio patch) mode

use tmux to be able to detach job and leave it running 24/7

basic manual operation
(no watchdog process or email alerts from server)

CCRMA machines are available

CCRMA account required to login to CCRMA machines

Ensemble info :
  • which instruments in the ensemble
  • audio equipment needed
  • computers needed
  • which operating systems will be used
  • (email to cc@ccrma.stanford.edu)

Session 4

April 17 -- server side operation

jacktrip molecules and atoms

we've been practicing hub mode (-S / -C)

which is made out of multiple atoms:
simple two-way, connections (-s / -c)

hub mode
(-S / -C)

simple two-way (aka, full-duplex) stereo connection
(-s / -c)

Cristina's comparisons, a) acoustic room b) hub mode network
c) labyrinths d) shortest path network

when the "atoms" combine in a hub server, they can be connected in different ways

use -p to patch them together into a "molecule"

4=full mix
(everyone hears everyone but not the server itself)

2=client fan out/in but not loopback

1=client loopback


-p, --hubpatch # (0, 1, 2, 3, 4)

Hub auto audio patch, only has effect if running HUB SERVER mode,

0=server-to-clients, 1=client loopback, 2=client fan out/in but not loopback, 3=reserved for TUB, 4=full mix

(default: 0)

153b groups next week
  • jazz sextet+
  • new music Bay Area
  • Stanford new music ensemble
  • new music quartets (JACK, Kronos)
  • wind quintet
  • 153b jam band
  • volunteers to help the above groups or other groups

(click below) to open https://ccrma.stanford.edu/wiki/IETF

Ensemble sound checks
  • get the group talking together on Zoom, jitsi, skype or the like
  • have each member test their local audio loopback:
    qjackctl connect capture -> playback
  • tech lead person start group server in full mix mode:
    ./jacktrip -S -p4
  • have each member test their network audio:
    ./jacktrip -C <server_IP_address> -n1
  • when everyone can hear everyone else well, end the connections
  • tech lead person restart group server in FOFI mode:
    ./jacktrip -S -p2
  • then everyone rejoin and take it from there

QoS measurements

chris WIFI / 12Mb cable

chris ethernet wired / 12Mb cable

sarah / 1Gb fiber

sarah / 200Mb? cable

Monday, 19th (open forum using jacktrip audio
I'll pipe in some historical examples)

work with a group from now - Thursday and give us a report

next lecture session on Friday, 24th, 2:30pm

(how to get software updates,
how to run simple two-way connections and server-less operation,
utilities: ping, traceroute, iperf)

Server stress test
  • test your local audio loopback:
    qjackctl connect capture -> playback
  • test your network with loopback server:
    ./jacktrip -C -n1
  • join the horde, crash the server:
    ./jacktrip -C -n1

Session 6

April 24 -- simple two-way setup

We've been using hub mode (a fancy conference call) which depends on relaying all signals via a hub server. Simple two-way connections are direct between two peers and don't require a hub server.

jacktrip two-way, bidirectional audio connection
  • server and client
  • start jack audio on both sides
  • start the server first
  • server:
    ./jacktrip -s -n1
  • client:
    ./jacktrip -c <server_IP_address> -n1
  • or:
    ./jacktrip -c <server_host_name> -n1

server must have port open
  • incoming UDP port 4464 (is default)
  • select a different port with "-o" port offset
  • server:
    ./jacktrip -s -n1 -o10 (uses UDP port 4474)
  • client:
    ./jacktrip -c <server_IP_address> -n1 -o10

useful debugging utilities: ping, traceroute, iperf

  • ping www.yahoo.com
  • ping
  • might not respond (dead or wrong name, wrong IP address)
  • might not want to respond (policy reasons, for example no CCRMA machines will answer)

  • traceroute www.yahoo.com
  • shows path through intervening routers
  • pinging them in succession
  • prints * * * if no response

  • server and client (same style as jacktrip)
  • sends a test batch of data from client to server
  • server:
    iperf -u -s -p 4464
  • client:
    iperf -u -c <server_IP_address> -p 4464
  • I use iperf2 but iperf3 exists

upcoming software updates and how to get them

what I have learned in the last 3 weeks

terminology, names
  • jack server (audio)
  • hub server (a host for hub mode)
  • peer-to-peer server (starts a two-way)

jack operation
  • where settings are set
  • only one jackd at a time
  • use "killall jackd" command
  • doesn’t have to be issued from the same directory, and should result in
    "jackd: no process found" once there's no longer a jackd

OS installation
  • a file permissions bug in the current macOS installer
  • unidentified developer, package cannot be opened, or installation is potential malware (Go to System Preferences / Security and Privacy)
  • Windows security settings; Allow an app through firewall.
  • Windows Peer to Peer Collaboration Foundation
  • curious error messages that need explaining: MMCSS API not used

server management
  • accidental root login will lock out their next attempts, remote CCRMA users to use their account name explicitly
  • need to initiate the ssh -Y from an X server (such as XQuartz on macOS)
  • audio permissions on server for remote users
  • script timings vs. jack (sometimes need to add a small amount of sleep)

  • jacktrip versions (-v) -- should be latest, V1.2
  • qjackctl changed and there are two versions running "Connections" is now labeled "Graph" instead but you can change it back
  • not all dongles
  • details that get in the way
    • selection of mode (particularly "-C" vs. "-c")
    • audio settings on all machines must match, particularly sample rate
    • running only 1 channel (using only one side of headphones)
    • builtin mic / builtin speaker can squeal with loopback insanely
    • using a disabled command argument, the software should print a warning (-S -n)

    good surprises!
    • running associated audio applications can work
    • there are some unexpected and working audio interfaces (USB microphones, guitar amps)
    • we are in a rare position (worldwide) having a flock of physical servers to use (others are using cloud and paying)

    good and not a surprise!
    • many 153b jacktrip saints!
      especially Matt, Synthia, Dave, Bonnie, Nick and all who are helping others

    to discover through use
    • home ISP service can degrade
    • push the limits on number of channels
    • how low can you go, latency-wise

    Monday's meeting, session 7 (5 enrolled students) -- entirely on jacktrip

    Session 8

    May 1 -- Concerts, and testing! And bigger servers!

    Saturday concerts (May 2) --

    (Sarah Weaver) Magnify the Sound will present a 20 minute live musical performance from 3 different locations in Norway exploring the interplay and live processing between drums/percussion and electric guitar.
    10am (PDT)
    NowNet Arts Online Performance Series

    (Constantin Basica) CCRMA Live "In the past few weeks, we have been programming, testing, and rehearsing in an online environment between California, which is facing a 'shelter-in-place' situation, Berlin (DE) with strict 'Kontakt- und Ausgangsbeschränkungen', and Ghent (BE). Each Saturday afternoon we are presenting a concert that connects six musicians from these locations and guests from other places to each other. The sessions are broadcast live with audio and video feeds from each site.
    1pm (PDT)
    Quarantine Sessions #6

    "JackTrip in Zoom (Webinar)" -- the 10am

    "They are running JackTrip, routed into Zoom. Thomas was able to get into their university facilities and bring the drumset player Carl. Trond has a network arts studio he built out at home. He was able to coordinate with the local public internet provider to optimize his routing and configurations. So that's how they are running JackTrip.

    They have the whole JackTrip connection in one computer, they are slightly delaying the signal in order to line it up with the Zoom video latency, adding slight reverb, and then outputting into a Zoom computer.

    I have a "Webinar" room in Zoom now, so the viewer only sees the artists. The audience is listed as attendees and only communicates via the text chat. It is a different presence but it's a significant improvement to the look of the event. Still would prefer to see small visual icons of the audience on the bottom of the screen, maybe this will be a feature in the future or in a different platform that emerges.

    The dress rehearsal yesterday was impressive. This is the closest I have experienced to a real "virtual venue". It's much better sound with JackTrip and the "Webinar" room looks good. Adding the slight delay to JackTrip lines up the visuals. This felt exciting. "

    "jacktrip in Youtube and Vimeo Live" -- the 1pm

    The 'Quarantine Sessions' are realized using free and open source technologies, which can be easily adopted by anyone:

    jacktrip (audio)
    jitsi (video)
    OBS (streaming)

    network fiber laces the world

    watch this now

    Inside the Beach House

    153b concert? For end-of-term?
    • who's already planning something?
    • bring some of the current groups to the "stage"
    • form new groups

    ensemble query followup in our group
    • Mac Kenley, trombonist
    • Lesley Roberston, violist
    • Jerry McBride, wind quintet

    ensemble query followup, new groups
    • Golden Gate Men's Chorus
    • San Francisco Early Music Society
    • Paul DeMarinis, Laetitia Sonami, Sue Costabile, event at SAIC (Chicago)
    • Marie-Louise Catsalis (work with music theater students)

    network fiber laces the world

    watch this now

    Inside the Beach House

    new upgrade version of jacktrip
    • -C (connect to server by name or IP address)
    • -Sz (hub mode with zeroes)

    testing it
    • today both cmn9.stanford.edu and
      cm-jam.stanford.edu are servers running the new version
    • number of clients is limited to virtual cores still

    • win10 (but broke -C with IP address)
    • linux (fedora, ubuntu?)
    • macOS (mojave, catalina)

    Here's a head scratcher:

    Single jacktrips were originally created to do 3-way audio patches over the net
    a-b, b-c, c-a

    Hub server mode is a centralized relay which spawns on demand,
    a-s, b-s, c-s -- let's call that server"sa."

    In the past couple weeks "sa" has started to get too big to host on a single machine. We need to do fully connected
    sa-sb, sb-sc, sc-sa

    I've only faked it using independent single tie points, so far

    and that's a pain, it works but seems like it won't scale.

    Monday's meeting, session 9 -- developing a document (succinct, ready-to-roll) breakout rooms

    Next Friday's meeting, session 10 -- guest presenters: Sarah Weaver, Constantin Basica

    now some live testing with up to 40 participants split into four groups
    • I've started cm-jam, cmn5, cmn9, cmn13 with -Sz
    • and this needs some coordination
    • 16 in first group, -C cm-jam.stanford.edu (
    • 8 in second group, -C cmn5.stanford.edu (
    • 8 in third group, -C cmn9.stanford.edu (
    • 8 in fourth group, -C cmn13.stanford.edu (

    session 12 -- groups continued -- exercise in latency testing, how low can you go?

    exercise in latency testing, how low can you go?

    six servers are running different frames per period (aka Jack buffer size and packet size)
    FPP = 32, 64..., 512, 1024
    • cmn5.stanford.edu (32) (
    • cmn13.stanford.edu (64) (
    • cmn14.stanford.edu (128) (
    • cmn15.stanford.edu (256) (
    • cmn25.stanford.edu (512) (
    • cmn26.stanford.edu (1024) (
    for example, set audio to 48000 / 1024 and do
    jacktrip -C cmn26.stanford.edu (or

    session 13 -- new concepts in concert promotion & production (guests)

    John Lee

    Paolo Lattanzi

    session 14 -- incorporating DSP into jacktrip
    for audio compression, limiting, error concealment,
    data compression, session mixing

    discussion with group, Prateek Verma, Julius Smith and others

    The lines which we connect and disconnect in qjackctl are audio signals. The objects that get connected are processes like jacktrip. We can add processes which alter the audio, for example, reverberators, panners, and much more.

    microphone connected through a reverberator to loudspeaker

    loopback with loop filter

    Monday, May 25 -- Memorial Day Holiday

    session 15 -- JMess

    soundcard latency shootout

    microtiming behavior of jacktrip playback buffers

    and a very cool $50 surprise

    a week ago: loopback with loop filter

    jmess is your friend

    listening to roundtrip audio latency on a network path (soundWIRE)

    testing audio latency within the audio interface itself

    not all soundcards are created the same

    Behringer vs. Presonus

    1024 frames/period Jack = 42.7

    Behringer = 62
    Presonus = 55

    512 frames/period Jack = 21.3

    Behringer = 42
    Presonus = 34

    256 frames/period Jack = 10.7

    Behringer = 24
    Presonus = 22

    128 frames/period Jack = 5.33

    Behringer = 14
    Presonus = 13

    soundcard latencies measured through Jack at FPP from 16 t0 1024 (input connected to output)

    playback buffer dynamics

    large playback buffers are required to compensate network jitter
    when using lower FPP settings (over home internet connections)

    large input buffers create delay and the delay changes in response to jitter
    are they behaving in an optimal way? Probably not.

    combinations of -q settings at FPP = 64

    combinations of -q settings at FPP = 32

    that was the very cool $50 surprise!

    Raspberry Pi 4 + Presonus soundcard (audio interface)

    the paths forward for local ensembles

    equip musicians with their choice of devices
    set up local servers
    • DIY software installs (like we know)
    • nearby hardware-based servers (like we know)
    • Raspberry Pi + soundcard (plug+play with interface)
    • nearby cloud servers
    • ...and DIY Raspberry Pi systems (mouse, keyboard, monitor)
    all open source
    Raspberry Pi solutions and cloud-based services are in the works and will be obtainable from third parties

    next session 16 June 1st -- live playing

    "Blame it on the Network" live playing
    with ensembles from the Spring quarter group

    Before class
    • Prior to the class, ensembles perform a soundcheck
    • Each nsemble should let dave@ccrma.stanford.edu know their hub server address
    • And let cc@ccrma.stanford.edu know all members likely zoom names
    • Everyone joins the regular class meeting zoom
    • CC puts the first ensemble members plus DK1 into a room
    • DK1 streams the room's gallery view and audio to the remainder of the class via participant DK2 (audio to the rest of the class is the jacktrip audio rather than the zoom meeting audio)
    • CC spotlights DK2 so everyone sees the pinned view of the ensemble
    • rinse and repeat
    (CC will be recording from Zoom)

    session 17 June 5th -- wrap up


    some live playing (Brahms)
    • what's in store over Summer and beyond
    • super server (underway)
    • WebRTC jacktrip (underway)
    • new jacktrip version (finally, nearly, almost)
    • SlipStream (might start soon)
    • Raspberry Pi systems (DIY ready now, commercial version a couple weeks from now)
    ...course recap in 14 slides...
    (in retrograde order)

    session 17+ -- tonight (from the future)

    World According to Sound -- the show begins at 8:30 PDT. All you've got to do is click on the link on the CCRMA home page (https://ccrma.stanford.edu/live/), put your eye mask on, and enjoy the show!

    session 16 -- live playing

    session 15 -- the very cool $50 surprise

    session 14 -- incorporating signal processing

    session 13 -- new concepts in concert promotion & production

    (guests) John Lee, Paolo Lattanzi

    session 12 -- exercise in latency testing, how low can you go?

    FPP = 32, 64..., 512, 1024

    session 10 -- concerts from homes to homes

    (guests) Sarah Weaver, Constantin Basica

    session 9 -- group documentation development

    session 8 -- breakout rooms with servers

    session 6 -- simple two-way setup

    session 4 -- server side operation, "server stress test"

    lots of clients on a growing population of loopback servers, for example
    ./jacktrip -C cmn16.stanford.edu

    session 3 -- dialing into the sweet spot and "adopt-a-server"
    mics, monitoring, sample rate, FPP, hub server

    session 2 -- group connections
    do an audio test first
    things that prevent connection

    session 1 -- intro
    This course inaugurates the Internet Ensemble Tech Force which is needed urgently worldwide and locally to support music ensembles going online. Calling it urgent is not an exaggeration and we can provide a valuable service. Course participants will quickly come up to speed on low-latency audio collaboration technology and will then pair with ensembles interested in using it.

    153b participants will work from home and be able to use CCRMA facilities remotely. The course can be audited or coordinated with another course. Let's help make group playing possible during this public health challenge and period of 'musical distancing.' Ensemble rehearsals, coaching and concert broadcasting are planned in conjunction with several local ensembles.