...loading...

Network Performance Practice, Music 153a
Autumn, 2020
Center for Computer Research in Music and Acoustics (CCRMA)
Stanford University

Sign up here:
NPP email list
(to receive announcements and online meeting info)


Navigate this deck with keyboard or touch

type "m" for the menu of pages and
arrow or page keys to advance or rewind

two-finger tap for the menu of pages and
the usual left/right swipe to advance or rewind



embedded media clips (audio and video) use an extra click (or swipe) to begin playing
and another to finish



14 Sep -- intro
Session 1


16 Sep -- demo
Session 2


21 Sep -- install
Session 3


23 Sep -- connect
Session 4


28 Sep -- projects
Session 5


30 Sep -- audio planes
Session 6


5 Oct -- code base
Session 7


7 Oct -- Sarah Weaver
Session 8



12 Oct -- rehearsal uses
Session 9


14 Oct -- performance planning
Session 10


19 Oct -- historical recordings
Session 11 lecture material


21 Oct -- grad student theses
Session 12 lecture material


26 Oct -- JackTrip only
reading: Machine learning for packet loss concealment


28 Oct -- Zoom and JackTrip
reading: GPS audio sync

Network Performance Practice (NPP)

JackTrip software, developed at Stanford, provides the means for ultra-low-latency, uncompressed sound transmission for live music-making. Remote ensemble rehearsals, coaching, music lessons, jamming and concert broadcasting during the COVID-19 pandemic are making use of the technology. The open-source project has developed rapidly in the past 6 months, especially in its ability to support large ensembles of home-to-home connections. The course will cover recent features, history and theory of JackTrip and engage in a series of practical, participatory performance sessions. Students will learn the software and related network and audio principles with a focus on intuition building and ear training. Course participants will work from home and be able to use CCRMA facilities remotely. The course can be audited or coordinated with another course.

M W, 1pm - 2:20pm, Zoom meetings will be announced via
NPP email list
(first meeting is 14-Sep-2020)

All are welcome and there are no prerequisites. The course will start with how to get set up to play together online. Auditors may attend and anyone who is working with their ensemble or band is especially invited.

Tips, tricks and background info on various types of JackTrip use will be presented. For example, the Music Department is distributing JackStreamer kits to its instructors and others are using devices and services from JackTrip.org or other vendors. DIY software installs are now commonplace and have been developed in conjunction with the current working group.
For example,
JackTrip on Mac

Monday, 14-Sep
  • intro to the course
  • administrative stuff

--administrative stuff--

Zero-unit enrollment is possible. Contact Chris if interested.

network fiber laces the world

watch this now

Inside the Beach House

QoS measurements

incoming stream interpacket timimngs seen by Stanford server

(msec vs. sec) chris WIFI / 12Mb cable

(msec vs. sec) chris ethernet wired / 12Mb cable

(msec vs. sec) sarah / 1Gb FIOS fiber

(msec vs. sec) sarah / 200Mb Spectrum cable

Wednesday, 16-Sep
  • improv with Scott Oshiro and Chris Chafe
  • optimizing audio and network quality
  • hub mode vs. peer-to-peer

Audio quality:
  • buffer size (affects latency)
  • gain
  • feedback
  • number of channels
  • self-monitoring
  • muting other audio applications (Zoom)
  • sharing audio signal between applications (audacity)
  • running Zoom on a second host (mobile or laptop)

Network quality:
  • physical distance and number of intermediate router hops in path (affects latency)
  • playback queue (affects latency)
  • jitter vs. packet loss
  • redundancy for forward error correction
  • wavetable vs. zeroes (very "backwards" error concealment)
  • better error concealment (under construction)
  • network peering locations can be a surprise
  • last mile (fiber, cable, vdsl, microwave, 5G)
  • wifi inadequate but useful for basic testing
  • choke points, competing applications on the same local host and local network, internet weather
  • venues, when we get to use them again

jacktrip molecules and atoms

molecules:
we've been practicing hub mode (-S / -C)

which is made out of multiple atoms:
simple two-way, connections (-s / -c)

simple two-way (aka, full-duplex) stereo connection
(-s / -c)

hub mode
(-S / -C)

Cristina's comparisons, a) the original Jack Triple connection (aka JackTrip) if you put a network audio host at each endpoint b) hub mode network adds a relay host in the middle

when the "atoms" combine in a hub server, they can be connected in different ways

use -p to patch them together into a "molecule"

4=full mix
(everyone hears everyone but not the server itself)

2=client fan out/in but not loopback

1=client loopback

0=server-to-clients

-p, --hubpatch # (0, 1, 2, 3, 4)

Hub auto audio patch, only has effect if running HUB SERVER mode,

0=server-to-clients, 1=client loopback, 2=client fan out/in but not loopback, 3=reserved for TUB, 4=full mix

(default: 0)

(click below) to open https://ccrma.stanford.edu/wiki/IETF

Reading for Monday

From Telephone to High Speed Internet:
A Brief History of My Tele-Musical Performances
Pauline Oliveros
December 14, 2007

(the first section of the document linked on the next slide)

(click below) to open https://ccrma.stanford.edu/~cc/pub/pdf/chafeLMJ19-2009.pdf

Monday, 21-Sep
  • install fest
  • testing hub mode with CCRMA servers at different latencies
  • preparing for Wednesday's meeting (on jacktrip)

jack audio settings
  • srate = 48kHz
  • channels = 2 (default)
  • frames per period (aka FPP, aka buffer size)
  • = 1024, 512, 256, 128, 64, 32

CCRMA servers running at different FPP settings
  • full loopback on both channels
  • looping clapping track on ch1
  • jackloop1024.stanford.edu
    jackloop512.stanford.edu
    jackloop128.stanford.edu
    etc.
  • hub client start command (for example, at FPP = 64)
    ./jacktrip -C jackloop64.stanford.edu

Latency
  • Geographical latency and Internet backbone
  • Internet service latency
  • Home network latency
  • ADC and DAC latency in computers
  • Acoustical latency
Why is Latency Important?
(explanation from newly-founded JackTrip.org)

Installation guides on CCRMA site
Instructions and Downloads
(linux, win10, macOS)



Building from github source code
Jacktrip : Build Instructions
(linux, Raspbian/Debian, win10, macOS)

Instructions and Downloads
(linux, win10, macOS)



Building from github source code
Jacktrip : Build Instructions
(linux, Raspbian/Debian, win10, macOS)

Preparing for Wednesday's meeting (on jacktrip)

soundcheck these three steps before class
(making sure Zoom works side-by-side)
  • test local analog loopback (no jack, no jacktrip)
  • test local jack audio loopback (no jacktrip)
  • test jacktrip loopback with a CCRMA server

Reading for Wednesday

Networked music performance:
An introduction for musicians and educators
Michael Dessen
Medium, September 18, 2020

(next slide)

(click below) to open https://medium.com/@mdessen/networked-music-performance-an-introduction-for-musicians-and-educators-d31d33716bd2

Wednesday, 23-Sep
  • get together on JackTrip
  • test some clapping
  • jam

Monday, 28-Sep
  • Allan's Virtual Choir project
  • project(s) for the term
  • Zoom with JackTrip Bridge
  • JackTrip webRTC test

Zoom with JackTrip Bridge

connects a JackTrip room into Zoom using Music Mode
two audio interfaces connected by analog wire
  • JackTrip server (the JackTrip room)
  • JackTrip clients join JackTrip room
  • JackTrip side of Bridge joins, too
  • Zoom side of Bridge hosts Zoom meeting
  • Zoom side uses Original Sound, Music Mode

connect
  • 48000 sample rate
  • 32FPP
  • -q 24
jacktrip -C cmn9.stanford.edu -q24

JackTrip webRTC test

JackTrip runs directly in the browser
video is included
test with Chrome (for now)
  • replaces Zoom
  • uncompressed stereo audio, 128FPP
  • use loopback feature to test
  • audio latency depends on the browser's audio handling (quite long)

connect
  • may need to quit Zoom
  • may need to allow non-certified connection (advanced feature)
https://cmn9.stanford.edu:61111/room/cdd3d641d7aa

Wednesday, 30-Sep
  • Hub server testing (urgent)
  • Zoom with JackTrip Bridge (cont'd)
  • feedback on chapter "Qualities and Flow in Imagined Sound and Music"
  • High Fidelity webRTC test
    https://map.highfidelity.com/Mmb2UpAMwO7qf130/570dbce0d465c2a894fe8a59a5e1c3dc44e8cccf438d319ce447301376ddfc5d/?map=HQ

Wednesday, 30-Sep
  • disconnect your Zoom mic (or turn it off)
  • disconnect your Zoom output (or turn it off)
  • wait for the next slide to appear
  • we will begin the class together, silently

The sound you hear is the sound of your inner voice reading these words.

Where is it located, your inner voice?

Most will report that it's inside their head.

Let's call that an audio plane or layer.

Sounds in your environment are perceived outside that plane.

Mess around a bit. See if you can hear on both planes at once.

(interior and exterior)

Here's a conjecture:

When we turn on Zoom audio,
we will be meeting on a shared audio plane.

Zoom conference is in white.

And...

When we meet on JackTrip server,
we will be meeting on a different shared audio plane.

JackTrip audio session is in black.

JackTrip audio bridge to Zoom conference.

Looks reasonable? It's confusing, though...

(as we've been learning in class)

Unlike inner voice + perceived audio planes,
we're not yet good at simultaneously handling Zoom + JackTrip.

It's probably even worse, cognitively speaking.

(interior, exterior, Zoom and JackTrip)

Thought experiment: produce a production like the one on the next screen?

(click below) to open https://www.youtube.com/watch?v=I6kZQ5nW9zw

Reading for Monday (1)

Internet2 was founded (1996)

(click below) to open https://www.internet2.edu/news/detail/10903/

Reading for Monday (2)

Networking Audio and Music:
Using Internet2 and Next-Generation Internet Capabilities
Audio Engineering Society

commissioned (1998) -- two years after Internet2 was founded

(click below) to open https://ccrma.stanford.edu/~cc/153resources/AESwhitePaper98.pdf

Monday, 5-Oct
  • (very) Short history tour
  • More on quarter projects
  • Zoom recording with JackTrip Bridge -- we're going to nail this but not in stereo! Zoom issue.
  • Rehearse "Remote Control"
  • JackTrip code base and development

(click below) to open https://ccrma.stanford.edu/~cc/deck.js/153aAutumn2020/HistLinksOnePage.html

Remote Control

a structure for small ensemble improv

rehearse perfect fusion
(sustained tone)

JackTrip code base and development

(click below) to open https://github.com/jacktrip/jacktrip

Reading for Wednesday

Caceres, J.-P. and Chafe, C. . "Jacktrip: Under the hood of an engine for network audio." J. New Music Res. 39 (2010) 183–187. doi:10.1080/09298215.2010.481361

(click below) to open https://ccrma.stanford.edu/groups/soundwire/publications/papers/2009-caceres_chafe-ICMC-jacktrip.pdf

Wednesday, 7-Oct
  • guest lecture, Sarah Weaver

Reading for Monday

J-P. Caceres, C. Chafe, "JackTrip/SoundWIRE Meets Server Farm" Computer Music J. 34(3) (2010) 29–34

(click below) to open https://ccrma.stanford.edu/groups/soundwire/publications/papers/2009-caceres_chafe-SMC-jacktripmulticlient.pdf

Monday, 12-Oct
  • ensemble playing
  • Music Dept. uses of JackTrip

the group of clients gets distributed across 9 inputs and panned separately
(the distribution depends on the number of clients)

the stereo panning output from the last slide is piped through stereo "freeverb" and then delivered the same to all clients

example Stanford Music Dept. usage
  • vocal lessons with accompaniment
  • jazz combo
  • Beethoven Septet
  • Stanford New Ensemble

clients
  • mix of laptop and Raspberry Pi (JackStreamer)
  • students in campus residences (IT registration)
  • Braun classrooms and faculty offices (different IT registration)
  • students and faculty from home

servers
  • hardware at CCRMA
  • jacktrip v.1.2.1
  • experiemental panning and reverb version
  • clapping loop for testing (of clients and servers)
  • don't fully understand how many servers are needed, yet
  • booking system through email and online system

ease-of-use improvements
  • figure out how to switch from testing to production
  • (turn off clapping loop somehow?)
  • clients power on their JackStreamers and automatically join their last server

Reading for Wednesday

C. Chafe, J-P. Caceres, M. Gurevich, "Effect of temporal separation on synchronization in rhythmic performance" Perception 39(7) (2010) 982–992

(click below) to open http://chrischafe.net/wp-content/uploads/2014/08/temporalSep.pdf

Wednesday, 14-Oct
  • recent ensemble playing
  • rehearsing for NowNetArts Conference

listen and make recommendations #1

Beethoven Septet rehearsal from Monday
(link on next slide, see you in 6 minutes)

(click below) to open https://ccrma.stanford.edu/~cc/153resources/wav/beethovenSeptet12-Oct.mp3

listen and make recommendations #2

two takes of Remote Contol from Monday
(link on next slide, see you in 8 minutes)

(click below) to open https://ccrma.stanford.edu/courses/153a-fall-2020/webm/12Oct.webm

possible one-hour slots for
NowNetArts Conference
  • Thu, 5 Nov, 2pm (PST)
  • Fri, 6 Nov, 1pm (PST)
  • Fri, 6 Nov, 2pm (PST)

and / or join lab ensemble
  • Sat, 6 Nov, 9am (PST)
  • Sat, 6 Nov, 6pm (PST)

Sarah Weaver sound painting reference
video (2 min)

rewire panning and reverb
  • like -p2, only monitor others
  • add self-monitoring using local reverb

Reading for Monday

C. Rottondi, C. Chafe, C. Allochio, A. Sarti "An Overview on Networked Music Performance Technologies" IEEE Access 4 (2017) 8823–8843

(click below) to open https://ieeexplore.ieee.org/abstract/document/7769205

Monday, 19-Oct
  • historical recordings
  • rehearsing for NowNetArts Conference



19 Oct -- historical recordings
Session 11 lecture material

Wednesday, 21-Oct
  • rehearsing for NowNetArts Conference
  • grad student theses



21 Oct -- grad student theses
Session 12 lecture material

Reading for Monday

P. Verma, A. Mezza, C. Chafe, C. Rottondi "A Deep Learning Approach for Low-Latency Packet Loss Concealment of Audio Signals in Networked Music Performance Applications" arXiv.org arXiv:2007.07132

(click below) to open https://arxiv.org/abs/2007.07132

Monday, 26-Oct
  • power cut -- JackTrip only
  • rehearsing for NowNetArts Conference
  • settings:
    srate = 48kHz, FPP = 32, queue length = 12, channels = 1
  • command:
    ./jacktrip -C jacktemp32.stanford.edu -n1 -q12
  • JackStreamer, select jacktemp32.stanford.edu

Reading for Wednesday

P. Ferguson, C. Chafe and S. Gapp "Trans-Europe Express Audio: testing 1000 mile low-latency uncompressed audio between Edinburgh and Berlin using GPS-derived word clock, first with jacktrip then with Dante" 148th Convention2020 June 2 – 5, Vienna, Austria

(click below) to open https://ccrma.stanford.edu/~cc/153resources/lectureMaterials/AES148_Ferguson-Chafe-Gapp.pdf

----------begin last quarter web site----------

The following pages are from the previous course. Music 153b IETF Spring, 2020



April 6 intro and installation
Session 1 (edited)


April 10 group connections
Session 2 (edited)


April 13 dialing into the sweet spot
Session 3 (edited)


April 17 server side
Session 4 (edited)

     (April 20 hands-on workshop, not recorded)

April 24 simple two-way connections
Session 6 (edited)

     (April 27 registered student meeting, not recorded)

May 1 concerts, testing, bigger servers!
Session 8 (edited)


May 4 documentation workgroup
Session 9 (edited)

lecture recordings (continued)

May 8 guest presentations
Sarah Weaver (edited)
Constantin Basica (edited)

     (May 11 group round-robin discussion, not recorded)

May 15 hub server bug fix
Session 12 (edited)


May 18 guest presentations
John Lee (edited)
Paolo Lattanzi (edited)


May 22 signal processing with Julius Smith and Prateek Verma
Session 14 (edited)


May 29 JMess, soundcards, Raspberry Pi
Session 15 (edited)
SoundWIRE (edited)


June 1 live playing
Session 16 (edited)


June 5 last class, retrospective wrap up tour in retrograde
Session 17 (edited)

This course inaugurates the Internet Ensemble Tech Force which is needed urgently worldwide and locally to support music ensembles going online. Calling it urgent is not an exaggeration and we can provide a valuable service. Course participants will quickly come up to speed on low-latency audio collaboration technology and will then pair with ensembles interested in using it.

153b participants will work from home and be able to use CCRMA facilities remotely. The course can be audited or coordinated with another course. Let's help make group playing possible during this public health challenge and period of 'musical distancing.' Ensemble rehearsals, coaching and concert broadcasting are planned in conjunction with several local ensembles.

The one-quarter course meets online twice a week: Mondays and Fridays at 2:30pm for 50 minutes. The Zoom URL will be emailed to participants ahead of each meeting.

Sign-up: students should enroll in Music 153b, auditors are welcome and can join by contacting Chris Chafe

Listserv: emails will go out to ietf@ccrma.stanford.edu

Doable:
  • small groups playing together between home locations (say < 12 members)
  • concert broadcast via Youtube Live
  • linux, mac, windows laptops and desktops (also Raspberry Pi)
  • own choice of audio interfaces with mics, etc.

Difficult:
  • larger groups (but we haven't tried)
  • wifi (requires ethernet wire from home router)
  • built-in sound on laptops

Impossible:
  • syncronized video (but try LOLA for that)
  • conducted ensembles
  • mobile devices
  • hosted wine and cheese receptions

Previous course:

Online Jamming, TU Berlin, Winter 2019-2020

Upcoming course in Stanford sequence:

Music 153a Online Jamming and Concert Technology, Autumn 2020-2021

Monday, April 6
  • introduction
  • installation
  • test machine: 171.64.197.160
  • settings:
    srate = 48kHz, buffer size = 1024, channels = 1
  • command:
    ./jacktrip -C 171.64.197.160 -n1

free online course for background (mostly videos)

(click below) to open https://www.kadenze.com/courses/online-jamming-and-concert-technology-x/info

see CCRMA's web page, under Research : Software : jacktrip for installation guides

(click below) to open https://ccrma.stanford.edu

Session 2

April 10 -- group connections

admin stuff

for ugrad & grad students: zero-unit enrollment -- is possible
read this
let me know via an email you'd like to enroll for zero units because I need to file a request


ping -- the ping utility is turned off (by policy) for all CCRMA servers

I'll let you know if that changes

gear

USB mic works great

qjackctl can be configured to independently choose a USB mic as the input device and a separate output device

Blue Yeti USB mic can be both (it has a headphone jack)

group connections

Before going on the network, do an audio test first:
  • start qjackctl
  • in the connection window, connect mic to speakers or headphones
  • verify that there is signal
  • tap directly on the mic to be sure it's the right mic (not a built-in laptop mic)
  • click on disconnect
  • verify that the signal goes away (otherwise it's not qjackctl that's created the connection and most likely it's a direct sound monitoring path in your sound card)

Which jacktrip mode to run:
  • simple peer-to-peer server
    ./jacktrip -s
  • simple peer-to-peer client
    ./jacktrip -c <IP Address or Name of server>
  • hub (group) server
    ./jacktrip -S
  • client of hub (group) server
    ./jacktrip -C <IP Address of server>
    (-C mode doesn't recognize name, yet)

Connect to jacktrip loopback server:
  • command:
    ./jacktrip -C 171.64.197.160 -n1
  • play or sing a sustained tone
  • listen for dropouts
  • watch for "waited too long" messages
  • too many dropouts?
  • increase the input queue by adding -q<N>, for example, a queue of 8 packets (default = 4)
  • command with queue of 8 packets:
    ./jacktrip -C 171.64.197.160 -n1 -q8

Things that prevent connection
  • mismatch between local and peer sample rate
    --restart qjackctl with same sample rate as peer
  • mismatch between local and peer buffer size
    --restart qjackctl with same buffer size as peer, also called frames / period
  • "waiting for peer" forever
    usually this is because computer firewall or router firewall is blocking incoming data

Matt's explanation of the all those "jacks".

jackd is the actual Jack (jack audio connection kit process -- a daemon)
The "d" stands for daemon
It runs in the background letting other programs on the computer send and receive audio to each other and to whatever audio hardware jack is connected to.

jackd can be started from the command line but is more often launched from JackPilot or qjackctl

JackPilot is a graphical front-end to Jack.
It’s a program that opens a window you can click on, has a "Preferences" window, etc.
JackPilot’s preferences window is where you can:
- select which audio interface you want Jack to use
- select the Sample Rate and Buffer Size that Jack will use
- configure JackRouter
You must set all such preferences before starting Jack.
You can start Jack by clicking "Start" in the main JackPilot window. (On OSX this is the recommended way to start Jack)
On OSX with a normal Jack installation, the JackPilot program lives in
/Applications/Jack, that is, the "Jack" subfolder of the main Applications folder.

Matt's explanation (cont'd)
qjackctl is another graphical front-end to Jack.
"qjackctl" is pronounced "cue jack control" (though Mark Applebaum suggests making it rhyme with "pterodactyl") Its main window is (confusingly) titled "JACK Audio Connection Kit"

On OSX with a normal Jack installation, the qjackctl program lives in
/Applications/Jack, that is, the "Jack" subfolder of the main Applications folder.

The yellow word at the center top of the window will be "Active" if (qjackctl understands that) jack is running, or "Inactive" or "Stopped" otherwise
Please ignore all the green writing (which has to do with MIDI, not audio)

The "preferences" for qjackctl are in the "Setup" window, which you open and close by clicking the "Setup…" button on the main window

The most useful part of qjackctl is the "Connections" window, which you open and close by clicking the "Connect" button on the main window
The connections window shows all the jack-enabled programs currently running through Jack. It lets you connect any output of any program to any input of any program.

On OSX it appears that you cannot start Jack from qjackctl. JackRouter is part of Jack on OSX that lets non-jack-enabled programs (for example iTunes, QuickTime Player, Chrome) connect their audio to Jack.

Session 3

April 13 -- dialing into the sweet spot

admin stuff

the 1-unit course limit can't be changed, use 220d to add 2-4 more units as desired and let me know

I'm editing and streaming the class video recordings from CCRMA.

I've had requests for a commercial streaming service so I simply added links to the raw video on Zoom. These include the chat transcript which isn't anonymous. If that's problematic please let me know after this session.

minimizing delay

maximizing audio quality

maximizing network quality (as much as possible, given best effort network protocols)

latency is the time delay between cause and effect -- used interchangeably

an internet host is a computer on the network -- used interchangeably

a server listens for a client to connect to it -- different jacktrip modes

each jacktrip instance runs a transmitter and receiver -- bidirectional

my local host and a remote peer host -- either can run in either jacktrip mode

our IETF -- owes everything to
ietf.org

Audio quality:
  • buffer size (affects latency)
  • gain
  • feedback
  • number of channels
  • self-monitoring
  • muting other audio applications (Zoom)
  • sharing audio signal between applications (audacity)
  • running Zoom on a second host (mobile or laptop)

Network quality:
  • physical distance and number of intermediate router hops in path (affects latency)
  • playback queue (affects latency)
  • jitter vs. packet loss
  • redundancy for forward error correction
  • wavetable vs. zeroes (very "backwards" error concealment)
  • better error concealment (under construction)
  • network peering locations can be a surprise
  • last mile (fiber, cable, vdsl, microwave, 5G)
  • wifi inadequate but useful for basic testing
  • choke points, competing applications on the same local host and local network, internet weather
  • venues, when we get to use them again

cello demo

acoustic cello with air mic

electric cello

talkback channel, in this case it's Zoom

breakout rooms here we go again, 5 this time



171.64.197.160

171.64.197.169

171.64.197.156

171.64.197.166

171.64.197.207

adopt-a-server

public IP address for ensemble members to connect to

start jackd from command line and jacktrip server in loopback (or other audio patch) mode

use tmux to be able to detach job and leave it running 24/7

basic manual operation
(no watchdog process or email alerts from server)

CCRMA machines are available

CCRMA account required to login to CCRMA machines

Ensemble info :
  • which instruments in the ensemble
  • audio equipment needed
  • computers needed
  • which operating systems will be used
  • (email to cc@ccrma.stanford.edu)

Session 4

April 17 -- server side operation

jacktrip molecules and atoms

molecules:
we've been practicing hub mode (-S / -C)

which is made out of multiple atoms:
simple two-way, connections (-s / -c)

hub mode
(-S / -C)

simple two-way (aka, full-duplex) stereo connection
(-s / -c)

Cristina's comparisons, a) acoustic room b) hub mode network
c) labyrinths d) shortest path network

when the "atoms" combine in a hub server, they can be connected in different ways

use -p to patch them together into a "molecule"

4=full mix
(everyone hears everyone but not the server itself)

2=client fan out/in but not loopback

1=client loopback

0=server-to-clients

-p, --hubpatch # (0, 1, 2, 3, 4)

Hub auto audio patch, only has effect if running HUB SERVER mode,

0=server-to-clients, 1=client loopback, 2=client fan out/in but not loopback, 3=reserved for TUB, 4=full mix

(default: 0)

153b groups next week
  • jazz sextet+
  • new music Bay Area
  • Stanford new music ensemble
  • new music quartets (JACK, Kronos)
  • wind quintet
  • 153b jam band
  • volunteers to help the above groups or other groups

(click below) to open https://ccrma.stanford.edu/wiki/IETF

Ensemble sound checks
  • get the group talking together on Zoom, jitsi, skype or the like
  • have each member test their local audio loopback:
    qjackctl connect capture -> playback
  • tech lead person start group server in full mix mode:
    ./jacktrip -S -p4
  • have each member test their network audio:
    ./jacktrip -C <server_IP_address> -n1
  • when everyone can hear everyone else well, end the connections
  • tech lead person restart group server in FOFI mode:
    ./jacktrip -S -p2
  • then everyone rejoin and take it from there

QoS measurements

chris WIFI / 12Mb cable

chris ethernet wired / 12Mb cable

sarah / 1Gb fiber

sarah / 200Mb? cable

Monday, 19th (open forum using jacktrip audio
I'll pipe in some historical examples)

work with a group from now - Thursday and give us a report

next lecture session on Friday, 24th, 2:30pm

(how to get software updates,
how to run simple two-way connections and server-less operation,
utilities: ping, traceroute, iperf)

Server stress test
  • test your local audio loopback:
    qjackctl connect capture -> playback
  • test your network with loopback server:
    ./jacktrip -C 171.64.197.160 -n1
  • join the horde, crash the server:
    ./jacktrip -C 171.64.197.158 -n1

Session 6

April 24 -- simple two-way setup

We've been using hub mode (a fancy conference call) which depends on relaying all signals via a hub server. Simple two-way connections are direct between two peers and don't require a hub server.

jacktrip two-way, bidirectional audio connection
  • server and client
  • start jack audio on both sides
  • start the server first
  • server:
    ./jacktrip -s -n1
  • client:
    ./jacktrip -c <server_IP_address> -n1
  • or:
    ./jacktrip -c <server_host_name> -n1

server must have port open
  • incoming UDP port 4464 (is default)
  • select a different port with "-o" port offset
  • server:
    ./jacktrip -s -n1 -o10 (uses UDP port 4474)
  • client:
    ./jacktrip -c <server_IP_address> -n1 -o10

useful debugging utilities: ping, traceroute, iperf

ping
  • ping www.yahoo.com
  • ping 171.64.197.1
  • might not respond (dead or wrong name, wrong IP address)
  • might not want to respond (policy reasons, for example no CCRMA machines will answer)

traceroute
  • traceroute www.yahoo.com
  • shows path through intervening routers
  • pinging them in succession
  • prints * * * if no response

iperf
  • server and client (same style as jacktrip)
  • sends a test batch of data from client to server
  • server:
    iperf -u -s -p 4464
  • client:
    iperf -u -c <server_IP_address> -p 4464
  • I use iperf2 but iperf3 exists

upcoming software updates and how to get them

what I have learned in the last 3 weeks

terminology, names
  • jack server (audio)
  • hub server (a host for hub mode)
  • peer-to-peer server (starts a two-way)

jack operation
  • where settings are set
  • only one jackd at a time
  • use "killall jackd" command
  • doesn’t have to be issued from the same directory, and should result in
    "jackd: no process found" once there's no longer a jackd

OS installation
  • a file permissions bug in the current macOS installer
  • unidentified developer, package cannot be opened, or installation is potential malware (Go to System Preferences / Security and Privacy)
  • Windows security settings; Allow an app through firewall.
  • Windows Peer to Peer Collaboration Foundation
  • curious error messages that need explaining: MMCSS API not used

server management
  • accidental root login will lock out their next attempts, remote CCRMA users to use their account name explicitly
  • need to initiate the ssh -Y from an X server (such as XQuartz on macOS)
  • audio permissions on server for remote users
  • script timings vs. jack (sometimes need to add a small amount of sleep)

versions
  • jacktrip versions (-v) -- should be latest, V1.2
  • qjackctl changed and there are two versions running "Connections" is now labeled "Graph" instead but you can change it back
  • not all dongles
  • details that get in the way
    • selection of mode (particularly "-C" vs. "-c")
    • audio settings on all machines must match, particularly sample rate
    • running only 1 channel (using only one side of headphones)
    • builtin mic / builtin speaker can squeal with loopback insanely
    • using a disabled command argument, the software should print a warning (-S -n)

    good surprises!
    • running associated audio applications can work
    • there are some unexpected and working audio interfaces (USB microphones, guitar amps)
    • we are in a rare position (worldwide) having a flock of physical servers to use (others are using cloud and paying)

    good and not a surprise!
    • many 153b jacktrip saints!
      especially Matt, Synthia, Dave, Bonnie, Nick and all who are helping others

    to discover through use
    • home ISP service can degrade
    • push the limits on number of channels
    • how low can you go, latency-wise

    Monday's meeting, session 7 (5 enrolled students) -- entirely on jacktrip

    Session 8

    May 1 -- Concerts, and testing! And bigger servers!

    Saturday concerts (May 2) --

    (Sarah Weaver) Magnify the Sound will present a 20 minute live musical performance from 3 different locations in Norway exploring the interplay and live processing between drums/percussion and electric guitar.
    10am (PDT)
    NowNet Arts Online Performance Series


    (Constantin Basica) CCRMA Live "In the past few weeks, we have been programming, testing, and rehearsing in an online environment between California, which is facing a 'shelter-in-place' situation, Berlin (DE) with strict 'Kontakt- und Ausgangsbeschränkungen', and Ghent (BE). Each Saturday afternoon we are presenting a concert that connects six musicians from these locations and guests from other places to each other. The sessions are broadcast live with audio and video feeds from each site.
    1pm (PDT)
    Quarantine Sessions #6

    "JackTrip in Zoom (Webinar)" -- the 10am

    "They are running JackTrip, routed into Zoom. Thomas was able to get into their university facilities and bring the drumset player Carl. Trond has a network arts studio he built out at home. He was able to coordinate with the local public internet provider to optimize his routing and configurations. So that's how they are running JackTrip.

    They have the whole JackTrip connection in one computer, they are slightly delaying the signal in order to line it up with the Zoom video latency, adding slight reverb, and then outputting into a Zoom computer.

    I have a "Webinar" room in Zoom now, so the viewer only sees the artists. The audience is listed as attendees and only communicates via the text chat. It is a different presence but it's a significant improvement to the look of the event. Still would prefer to see small visual icons of the audience on the bottom of the screen, maybe this will be a feature in the future or in a different platform that emerges.

    The dress rehearsal yesterday was impressive. This is the closest I have experienced to a real "virtual venue". It's much better sound with JackTrip and the "Webinar" room looks good. Adding the slight delay to JackTrip lines up the visuals. This felt exciting. "

    "jacktrip in Youtube and Vimeo Live" -- the 1pm

    The 'Quarantine Sessions' are realized using free and open source technologies, which can be easily adopted by anyone:

    jacktrip (audio)
    jitsi (video)
    OBS (streaming)

    network fiber laces the world

    watch this now

    Inside the Beach House

    153b concert? For end-of-term?
    • who's already planning something?
    • bring some of the current groups to the "stage"
    • form new groups

    ensemble query followup in our group
    • Mac Kenley, trombonist
    • Lesley Roberston, violist
    • Jerry McBride, wind quintet

    ensemble query followup, new groups
    • Golden Gate Men's Chorus
    • San Francisco Early Music Society
    • Paul DeMarinis, Laetitia Sonami, Sue Costabile, event at SAIC (Chicago)
    • Marie-Louise Catsalis (work with music theater students)

    network fiber laces the world

    watch this now

    Inside the Beach House

    new upgrade version of jacktrip
    • -C (connect to server by name or IP address)
    • -Sz (hub mode with zeroes)

    testing it
    • today both cmn9.stanford.edu and
      cm-jam.stanford.edu are servers running the new version
    • number of clients is limited to virtual cores still

    versions
    • win10 (but broke -C with IP address)
    • linux (fedora, ubuntu?)
    • macOS (mojave, catalina)

    Here's a head scratcher:

    Single jacktrips were originally created to do 3-way audio patches over the net
    a-b, b-c, c-a

    Hub server mode is a centralized relay which spawns on demand,
    a-s, b-s, c-s -- let's call that server"sa."

    In the past couple weeks "sa" has started to get too big to host on a single machine. We need to do fully connected
    sa-sb, sb-sc, sc-sa

    I've only faked it using independent single tie points, so far
    sa-t-sb

    and that's a pain, it works but seems like it won't scale.

    Monday's meeting, session 9 -- developing a document (succinct, ready-to-roll) breakout rooms

    Next Friday's meeting, session 10 -- guest presenters: Sarah Weaver, Constantin Basica

    now some live testing with up to 40 participants split into four groups
    • I've started cm-jam, cmn5, cmn9, cmn13 with -Sz
    • and this needs some coordination
    • 16 in first group, -C cm-jam.stanford.edu (171.64.197.121)
    • 8 in second group, -C cmn5.stanford.edu (171.64.197.154)
    • 8 in third group, -C cmn9.stanford.edu (171.64.197.158)
    • 8 in fourth group, -C cmn13.stanford.edu (171.64.197.162)

    session 12 -- groups continued -- exercise in latency testing, how low can you go?

    exercise in latency testing, how low can you go?

    six servers are running different frames per period (aka Jack buffer size and packet size)
    FPP = 32, 64..., 512, 1024
    • cmn5.stanford.edu (32) (171.64.197.154)
    • cmn13.stanford.edu (64) (171.64.197.162)
    • cmn14.stanford.edu (128) (171.64.197.163)
    • cmn15.stanford.edu (256) (171.64.197.164)
    • cmn25.stanford.edu (512) (171.64.197.174)
    • cmn26.stanford.edu (1024) (171.64.197.175)
    for example, set audio to 48000 / 1024 and do
    jacktrip -C cmn26.stanford.edu (or 171.64.197.175)

    session 13 -- new concepts in concert promotion & production (guests)

    John Lee

    Paolo Lattanzi

    session 14 -- incorporating DSP into jacktrip
    for audio compression, limiting, error concealment,
    data compression, session mixing

    discussion with group, Prateek Verma, Julius Smith and others

    The lines which we connect and disconnect in qjackctl are audio signals. The objects that get connected are processes like jacktrip. We can add processes which alter the audio, for example, reverberators, panners, and much more.

    microphone connected through a reverberator to loudspeaker

    loopback with loop filter

    Monday, May 25 -- Memorial Day Holiday

    session 15 -- JMess

    soundcard latency shootout

    microtiming behavior of jacktrip playback buffers

    and a very cool $50 surprise

    a week ago: loopback with loop filter

    jmess is your friend

    listening to roundtrip audio latency on a network path (soundWIRE)

    testing audio latency within the audio interface itself

    not all soundcards are created the same

    Behringer vs. Presonus

    1024 frames/period Jack = 42.7

    Behringer = 62
    Presonus = 55

    512 frames/period Jack = 21.3

    Behringer = 42
    Presonus = 34

    256 frames/period Jack = 10.7

    Behringer = 24
    Presonus = 22

    128 frames/period Jack = 5.33

    Behringer = 14
    Presonus = 13

    soundcard latencies measured through Jack at FPP from 16 t0 1024 (input connected to output)

    playback buffer dynamics

    large playback buffers are required to compensate network jitter
    when using lower FPP settings (over home internet connections)

    large input buffers create delay and the delay changes in response to jitter
    are they behaving in an optimal way? Probably not.

    combinations of -q settings at FPP = 64

    combinations of -q settings at FPP = 32

    that was the very cool $50 surprise!

    Raspberry Pi 4 + Presonus soundcard (audio interface)

    the paths forward for local ensembles

    equip musicians with their choice of devices
    set up local servers
    • DIY software installs (like we know)
    • nearby hardware-based servers (like we know)
    • Raspberry Pi + soundcard (plug+play with interface)
    • nearby cloud servers
    • ...and DIY Raspberry Pi systems (mouse, keyboard, monitor)
    all open source
    Raspberry Pi solutions and cloud-based services are in the works and will be obtainable from third parties

    next session 16 June 1st -- live playing

    "Blame it on the Network" live playing
    with ensembles from the Spring quarter group

    Before class
    • Prior to the class, ensembles perform a soundcheck
    • Each nsemble should let dave@ccrma.stanford.edu know their hub server address
    • And let cc@ccrma.stanford.edu know all members likely zoom names
    2:30pm
    • Everyone joins the regular class meeting zoom
    • CC puts the first ensemble members plus DK1 into a room
    • DK1 streams the room's gallery view and audio to the remainder of the class via participant DK2 (audio to the rest of the class is the jacktrip audio rather than the zoom meeting audio)
    • CC spotlights DK2 so everyone sees the pinned view of the ensemble
    • rinse and repeat
    (CC will be recording from Zoom)

    session 17 June 5th -- wrap up

    today

    some live playing (Brahms)
    • what's in store over Summer and beyond
    • super server (underway)
    • WebRTC jacktrip (underway)
    • new jacktrip version (finally, nearly, almost)
    • SlipStream (might start soon)
    • Raspberry Pi systems (DIY ready now, commercial version a couple weeks from now)
    ...course recap in 14 slides...
    (in retrograde order)

    session 17+ -- tonight (from the future)


    World According to Sound -- the show begins at 8:30 PDT. All you've got to do is click on the link on the CCRMA home page (https://ccrma.stanford.edu/live/), put your eye mask on, and enjoy the show!

    session 16 -- live playing

    session 15 -- the very cool $50 surprise

    session 14 -- incorporating signal processing

    session 13 -- new concepts in concert promotion & production

    (guests) John Lee, Paolo Lattanzi

    session 12 -- exercise in latency testing, how low can you go?

    FPP = 32, 64..., 512, 1024

    session 10 -- concerts from homes to homes

    (guests) Sarah Weaver, Constantin Basica

    session 9 -- group documentation development

    session 8 -- breakout rooms with servers

    session 6 -- simple two-way setup

    session 4 -- server side operation, "server stress test"

    lots of clients on a growing population of loopback servers, for example
    ./jacktrip -C cmn16.stanford.edu

    session 3 -- dialing into the sweet spot and "adopt-a-server"
    mics, monitoring, sample rate, FPP, hub server

    session 2 -- group connections
    do an audio test first
    things that prevent connection

    session 1 -- intro
    This course inaugurates the Internet Ensemble Tech Force which is needed urgently worldwide and locally to support music ensembles going online. Calling it urgent is not an exaggeration and we can provide a valuable service. Course participants will quickly come up to speed on low-latency audio collaboration technology and will then pair with ensembles interested in using it.

    153b participants will work from home and be able to use CCRMA facilities remotely. The course can be audited or coordinated with another course. Let's help make group playing possible during this public health challenge and period of 'musical distancing.' Ensemble rehearsals, coaching and concert broadcasting are planned in conjunction with several local ensembles.