Coretet is a virtual reality instrument that explores the translation of performance gesture and mechanic from traditional bowed string instruments into an inherently non-physical implementation. Built using the Unreal Engine 4 and Pure Data, Coretet offers musicians a flexible and articulate musical instrument to play as well as a networked performance environment capable of supporting and presenting a traditional four-member string quartet. This paper discusses the technical implementation of Coretet and explores the musical and performative possibilities enabled through the translation of physical instrument design into virtual reality and realized through the composition Trois Machins de la Gráce Aimante (Coretet no. 1).

With deliberate reference to traditional stringed instrument performance practices, Coretet was designed as a futuristic '21st Century' implementation of the core gestural and interaction modalities that generate musical sound in the violin, viola and cello. By coupling the real-time fluid interactivity offered by the Unreal Engine 4 game engine with procedurally driven STK physical string models in Pure Data, Coretet offers musicians a flexible and articulate musical instrument capable of supporting improvisation as well as notated solo or ensemble performance.

Coretet also leverages the Unreal Engine's native network layer to create a networked virtual performance environment capable of supporting and presenting a traditional four-member string quartet to performers through head-mounted displays and to audiences through an auxiliary screen or projector. In a concert performance such as is utilised for Trois Machins de la Gráce Aimante this game server hosts each Coretet client instance (representing each performer) connecting across a local ethernet network. Performers in Coretet see one another in real-time with imperceptible latency within a virtual concert space, allowing for the use of communicative visual gesture both of the body and of the instrument and bow.

Fundamentally Coretet is a single instrument which can be shaped and scaled by performers into different configurations. Parameters such as neck length, body size, and number of strings can be manipulated in real-time to either recreate traditional stringed instruments such as violin, viola or cello or to create new and physically impossible instruments.

For ease of use during performance parameter presets for violin, viola and cello can be chosen and recalled instantly as can an experimental spherical instrument configuration known as the orb. Parameter values representing string length and body scale are also used to procedurally generate each instrument's sound affecting the frequency range and timbral identity of the current Coretet instrument.

Performers in Trois Machins de la Gráce Aimante are given a bow modeled after a traditional violin bow which activates the bowed string physical model when it comes into contact with a specific bowing bar on the instrument. A blue outline shown around the bowing bar indicates a collision between bow and bar, and a tracking marker indicating the collision is represented as a white sphere on the bow.

IEEE Computing Now Research Overview: The Industry Perspective: Rob Hamilton

Reference: Virtual Musical Instruments