Project Abstract


On September 12 and 13, 2009 in Milano, Italy, two concerts entitled Due Serata in Sirikata were presented at the Politecnico Sede di Milano, Bovisa as part of the 2009 MiTo SettembreMusica (Milano-Torino) Festival. Four musical works were presented, each featuring the Sirikata virtual environment as a central component, serving both as a networked meeting place for virtual as well as real-world musical performers and simultaneously as a musical controller itself, with client avatar motions and actions controlling musical production and spatialization around an 8-channel ambisonic sound-system. The performances showcased work done primarily at Stanford University as a collaboration between researchers at CCRMA, the Computer-Science Department, and the Stanford Humanities Lab.

CCRMA Team

Juan-Pablo Caceres: composition, programming, music direction
Robert Hamilton: composition, programming, music direction
Luke Dahl: logistics, engineering
Carr Wilkerson: networking, sound

Sirikata Engineering

Daniel Miller, Patrick Reiter Horn, Ewen Cheslack-Postava, Daniel Reiter Horn

Art Team

Chris Platz, Jason MacHardy, Ben Nadler, Hadidjah Chamberlain, Justin Herbst, Will Harper

SHL Team

Musicians

Chris Chafe, Debra Fong, Charles Nichols, Luke Dahl, Visda Goudarzi, Michael Berger
Due Serata in Sirikata a Mixed Reality Performance

Project Overview

Sound and the spatial environments in which sound is realized are intrinsically coupled. Sound in physical space is itself shaped by the spectral characteristics of the room or environment into which it is projected, with individual frequencies ampliļ¬ed or attenuated by the physical resonant characteristics of that given space. The basic tenets of physics dictate the details of this "call and response" relationship and as such, artists and engineers alike have devised performative and analytic methodologies for developing a comprehensive understanding of and expressive control over sound in space.

With the advent of enactive and immersive virtual spaces constructed within powerful networked computer systems, there exists an opportunity to create new paradigms within which sound and space can take on pointedly unnatural and previously impossible relationships. Taking a cue from both existing musical systems and centuries-old instrumental performative techniques, virtual spaces themselves can be transformed into reactive sonic environments wherein avatars controlled by human performers can shape the creation of sonic and musical events. Virtual spaces can become immersive instruments, enveloping their musical performers in an interactive environment where every motion and gesture can create and control sound and music.

At the heart of this presentation is a virtual performance space created within the Sirikata platform, our own open-source platform for the creation of massive multi-user virtual worlds. Using a state of the art three-dimensional graphics engine, beautifully rendered virtual performance spaces have been created that play off the physical characteristics of the concert space itself, in a sense a fantastic virtual corollary to the real-world physical space.

Within the Sirikata environment, multiple virtual performers with the ability to dynamically create musical sounds move and manipulate those sounds through the virtual space. In the physical concert space, a pianist performs, interacting with the virtual musicians, both controlling and manipulating aspects of the virtual world's sound with her playing as well as having aspects of her playing controlled and manipulated by the virtual performers themselves. Live musicians connected over high-speed computer networks from various locations around the world including California and Montana complete the ensemble.

For presentation to the concert and installation audiences, the location and motion of sounds in the virtual space are correlated into location and motion of sounds in the physical concert hall itself through the use of a multi-channel audio system, consisting of eight to sixteen powerful high-quality audio speakers placed around the periphery of the audience. In this manner, an immersive sound world can be created as a fitting complement to the immersive video projection of the virtual space itself.

Learn More...

[+] Playing with Virtual Space

[+] Playing the Network

[+] Sirikata

[+] Jacktrip

MiTo performance: program notes

Mixed Reality Performance is an experiment in which physical spaces and Musicians from different continents encounter one another on-line. Promoted by MITO SettembreMusica in collaboration with the Stanford Humanities Lab and the Stanford Center for Computer Research in Music and Acoustics, this production is an experiment in on-line interaction among musicians in different locations. While a pianist plays their music is manipulated by other musicians on-line; from across the ocean other acoustic realities are manipulated and appear projected in 3-D on walls. Spectators are immersed in a recreated world of sound in an installation that marks the birth of the performance spaces on Sirikata, the most recent open source platform on the web.

Event link: MiTo Festival page Politecnico Sede di Milano Bovisa Aula De Carli

Video by Robert Hamilton

In C   by Terry Riley

This classic of American minimalism is given a new twist, performed with 21st Century technologies and decidedly new ensembles. Members of the Stanfor Laptop Orchestra located in California virtually join a distributed overseas ensemble of laptops and traditional instruments spread across California, Montana and Milan for a rendition of this masterpiece.

Video by Robert Hamilton

Canned Bits Mechanics   by Juan-Pablo Caceres and Robert Hamilton

for three remote disklaviers at CCRMA, a Piano at the Scala, and visualizations in Sirikata.

A new interpretation of mechanical music, the interconnection of musical machines is now explored in the digital age. A local pianist performs with three remote networked Disklaviers. The nature of geographically separated instruments introduce delays that are explored in mechanical patterns between human and machines.

Featuring: Chryssie Nanou on piano (Milan), Juan-Pablo Caceres on remote disklaviers (Milan / CCRMA) and Robert Hamilton on interactive Sirikata performer (Milan).

Video by Robert Hamilton

Dei Due Mondi   by Robert Hamilton and Juan-Pablo Caceres

Dei Due Mondi explores the use of virtual space as both communal instrument and dynamic meeting space, bringing together musical performers from around the globe who interact with the environment as well as themselves as the piece unfolds. Worlds collide and become one as motion and gesture in the virtual world are realized as audible sound and music in the physical world. The audience itself teeters on the edge, positioned between two worlds, as a multi-channel sound system surrounds and envelops the concert hall, placing virtual performers and their musical gestures around and directly within the listening space.