SonART
- a new multimedia environment for networked collaboration
(This software has been presented to Stanford Software Fair at Apple, 3/30/2004)
Introduction
SonART is a flexible, multi-purpose multimedia environment that allows for networked collaborative interaction with applications for art, science and industry. It provides an open ended framework for integration of powerful image and audio processing methods with a flexible network features.
An arbitrary number of layered canvases, each with independent control of opacity, RGB values, image position, compositing options, etc., can transmit or receive data using Open Sound Control (OSC). Data from images can be used for synthesis or audio signal processing and vice versa.
Applications of SonART include multimedia art, collaborative and interactive art and design, and scientific and diagnostic exploration of data.
Originally created for sound driven data exploration and diagnostic purposes, the software also provides a powerful tool for multimedia performance over a network and facilitates real-time interactive creation, manipulation and exploration of audio and images.
Now it is implemented as a Cocoa-based OS X application. Functionality includes the ability to load and draw multi-layered images, and to send/receive visual and control data to/from other multimedia programs that support OSC.
An arbitrary number of layered canvases, each with independent control of opacity, RGB values, image position, compositing options, etc., can transmit or receive data using Open Sound Control (OSC). Data from images can be used for synthesis or audio signal processing and vice versa.
Applications of SonART include multimedia art, collaborative and interactive art and design, and scientific and diagnostic exploration of data.
Originally created for sound driven data exploration and diagnostic purposes, the software also provides a powerful tool for multimedia performance over a network and facilitates real-time interactive creation, manipulation and exploration of audio and images.
Now it is implemented as a Cocoa-based OS X application. Functionality includes the ability to load and draw multi-layered images, and to send/receive visual and control data to/from other multimedia programs that support OSC.
Features
Image display and processing capabilities
- Intuitive GUI features of audio mixers and of both audio and graphic editing software are preserved.
- Analogies between audio and visual processes are utilized.
- An arbitrary number of images can be simultaneously layered.
- User controlled parameters (i.e., opacity, compositing operations, visibility, location and size) for each layer can be controlled within the program interface or remotely by OSC messaging.
- Will be designed to facilitate plug-ins for any type of image processing
Transmitting and receiving data for sound and image generation and processing.
- Data such as location and RGB values of pixel on selected layers, opacity settings, etc can be both transmitted and received to and from an arbitrary number of network connections using OSC.
- Data can be used for sonification using any OSC enabled synthesis or audio processing engine, or image processing on another host running a separate SonART application.
- Scientific, medical, financial or other statistical data can be received from any networked source for directly mapped sonification and visualization purposes.
Flexibility of connections
Mappings between arbitrary combinations of parameters
- 1:1, 1:N, N:N connections between SonART(s) and sound synthesis programs
- Can be connected within a machine, locally, and/or over network at the same time
OSC Implementations
SonART is the first application that incorporates my WSOSC framework for parsing
OSC
packets; OSC client side is implemented using ObjC OSC library.
Downloads
- SonART (binary executable, OS X 10.3 or newer, zipped: 52[KB])
- final.sonart (document example, zipped: 2.4[MB])
- sonart.template.pat (Max/MSP template patch for using SonART, zipped: 4[KB])