last mod: 08-aug-09 sciss (This is a slightly updated version of the script from the symposium)
What is SwingOSC? Swing ... OSC ...
Why SwingOSC?
Requirements
/Library/Application Support/SuperCollider/Extensions
)
The first step is launch the SwingOSC server. The plain way to do it, is to
/Applications/Utilities/Terminal.app
, on GNOME: in one of the starter menus, on Windows: cmd.exe
)cd
into the SwingOSC installation directory, and launch it with the java
tool--help
, you will see the possible commandline options:OSC comes in two common "transport flavours" :
Using the -u
option will use UDP, while -t
selects TCP. The <portNum>
argument is necessary because there may be different UDP or TCP
services per computer, so they are distinguished by their port (an
integer number). For example, SuperCollider language uses UDP
port 57120, while the audio synthesis server (scsynth) uses UDP port
57110 by default.
For simplicity, we use the ready-made shellscript SwingOSC_TCP.command
(Mac OS X) or SwingOSC_TCP.sh
(Linux) to use TCP on port 57111
. (Sorry the .bat windows script seems to missing at the moment, but you can use the one that comes with Psycollider).
This also uses the -L option which forbids access from remote
computers, so you will need to remove the -L when your GUI server is
supposed to be on a computer different from the one running
SuperCollider!
Now let's see if the server responds. We do it in a very low level way, so it becomes more transparent what's happening behind the scenes. In SuperCollider we need to create a TCP client socket connected to the server:
n = NetAddr( "127.0.0.1", 57111 ); n.connect; // necessary for TCP, for UDP omit this line! n.sendMsg( '/print', '[', '/local', \userName, '[', '/method', 'java.lang.System', \getProperty, 'user.name', ']', ']');
The last line looks a bit complicated, especially if you are not familiar with the java API. Don't worry, the low-level communication will disappear under the hood in a second. Just check that the your user name is correctly printed in the terminal.
Now let's see what we can do with the existing GUI classes of SuperCollider:
g = SwingOSC.default; g.connect; // only necessary when you start SwingOSC without the -h option, so leave it away when using the .command or .sh shell scripts JSCWindow.viewPalette;
The left window is the default look on Mac OS X. The right window was produced by changing the look-and-feel first, using this line:
g.sendMsg( '/method', 'javax.swing.UIManager', \setLookAndFeel, "com.sun.java.swing.plaf.motif.MotifLookAndFeel" ); JSCWindow.viewPalette;
Note: to return to aqua look-and-feel (on Mac), use
"apple.laf.AquaLookAndFeel"
.
Some of the standard gadgets are rendered using the look-and-feel, as you see, for example the JSCSlider
, the JSCPopUpMenu
etc. Others – like JSC2DSlider
or JSCButton
– have been customly written for SwingOSC and look the same on all platforms and with all look-and-feels. (see also: www.javootoo.com)
Note: the variable
g
now holds the instance of the default SwingOSC server. TheSwingOSC
class is modelled after theServer
class (the client representation of scsynth). The methodsendMsg
sends an OSC message to the server.
Note: SuperCollider only knows four types of OSC arguments: Strings (s), Integers (i), Floats (f), and Blobs (b). The mixed use of Strings ("com.java..."
) and Symbols ('/method'
,\setLookAndFeel
) is a mere question of taste here.
The OSC command here follows the pattern'/method', <objectID or className>, <methodName> ... <methodArgs>
See also the file OSC-Command-Reference.html in the SwingOSC folder.
For the above call, consult also the API documentation: java.sun.com/j2se/1.4.2/docs/api/javax/swing/UIManager.html
While you can talk directly (low-level) to GUI java classes with SwingOSC, e.g. instantiate a javax.JFrame
and inside a javax.JButton
,
we are going to use high-level classes in SuperCollider which are more
or less closely linked to java counterparts on the server. So at the
moment we will forget about the details of the java world.
The starting point for every GUI is a window:
w = JSCWindow.new; // this creates the window (it's still invisible) w.front; // this makes the window actually visible
The window implies a container view (it's the socalled JSCTopView
) which can be filled with child components.
Note: I'm going to use the term 'component' synonymously with 'view', sometimes 'gadget' (where 'component' is more general as it can be another container view, and 'gadget' sounds more like it's a button or slider etc.)
Child components can be buttons, sliders, popup-menus, envelope-views etc. Each component is created with a Rect argument to specify its bounds inside the parent view (the window), and gets automatically added to the parent view:
// creates and adds a 2D-Slider inside the window w: x = JSC2DSlider.new( w, Rect( 10, 10, 160, 160 ));
The user can now interact with the GUI, but we need a means to be notified about its actions. Most gadgets allow you to assign an action-function that gets called whenever the user modifies the gadget's state (e.g. drags the slider in the example above):
( x.action = { arg view; // the argument to the action function is the component ("The slider's value is now " ++ view.x.round( 0.01 ) ++ " / " ++ view.y.round( 0.01 )).postln; }; )
There are more specialized action functions that can be assigned: Actions for keyboard typing (keyDownAction
and keyUpAction
), actions for mouse control (mouseDownAction
, mouseUpAction
, mouseOverAction
, mouseDragAction
), actions for handling drag-and-drop (canReceiveDragHandler
, beginDragAction
, receiveDragHandler
), an action when the component is removed (onClose
). Here is an example:
// first create a second slider component y = JSC2DSlider.new( w, Rect( 200, 10, 160, 160 )); // a copy+paste logic: pressing 'c' copies the x and y value, 'v' pastes // the values (try to copy from the left to the newly created right view!) ( var clipboard, func; func = { arg view, char, modifiers, unicode, keycode; var handled; ("Pressed char is '" ++ char ++ "'").postln; switch( char, $c, { "Copy!".postln; clipboard = view.x @ view.y; handled = true; }, $v, { if( clipboard.notNil, { "Paste!".postln; view.x = clipboard.x; view.y = clipboard.y; }); handled = true; }); // if the result of the keyDownAction is not nil, // the key press is 'consumed' (not processed by any // of the component's parent views) handled; }; x.keyDownAction = func; y.keyDownAction = func; )
Another example for mouse control:
( // Colorize the view's background while dragging the mouse [ x, Color.red, y, Color.blue ].pairsDo({ arg view, color; view.mouseDownAction = { arg view, x, y, modifiers, buttonNumber, clickCount; view.background = color; }; view.mouseUpAction = { arg view, x, y, modifiers, buttonNumber, clickCount; view.background = Color.clear; }; }); )
If you are not planning to mix your GUI with custom java components,
are merely relying on the ready-made component classes that come with
SwingOSC, and you are giving away your code to other people, it is
highly recommended to make an abstraction from the actual component
classes (such as JSCWindow
, JSC2DSlider
, etc.).
Instead you use a special factory class called GUI
. Using this class, your GUI code can be rendered with other GUI libaries, not just SwingOSC. For example, on Mac OS X, you can choose to present the GUI using the original Cocoa GUI classes, and some basic classes already exists for an Emacs integrated GUI.
Using GUI is straightforward: To create a window, instead of JSCWindow.new
you write GUI.window.new
. To create a 2D-Slider, instead of JSC2DSlider.new
, you write GUI.slider2D.new
. The names of the components can be looked up the GUI
help file. Here is code from above in the abstracted version; we render
it twice with SwingOSC and Cocoa GUI-Kits (the latter only works on Mac
OS X!):
( [ \swing, \cocoa ].do({ arg name, i; GUI.useID( name, { w = GUI.window.new( name.asString, Rect( 200 + (i * 440), 200, 400, 200 ), false ); 2.do({ arg j; GUI.slider2D.new( w, Rect( 10 + (j * 200), 10, 160, 160 ))}); w.front; })}); )
SuperCollider comes with a bunch of useful built-in visualizations
and GUI-controls. They are accessed by calling special methods on
objects that can be visualized (such as an array of numbers) or
controlled by a GUI. They use the current GUI kit which can be switched
using GUI.swing
or GUI.cocoa
. For example, every object can be "inspected" (all its fields are shown, those with setters can be modified):
GUI.swing; // or GUI.cocoa if you like Server.default.options.inspect;
The inspector shows the current field values using a JSCDragSource (for read-only fields) or a JSCDragBoth
(for read-and-write fields):
You can thus modify the fields with simple drag-and-dropping. Here is a window to select a sampling rate from:
( var rates = [ 44100, 48000, 88200, 96000 ], dragSource; w = GUI.window.new( "SR", Rect( 600, 300, 128, 72 ), false ); dragSource = GUI.dragSource.new( w, Rect( 4, 34, 120, 26 )) .object_( rates.first ); GUI.popUpMenu.new( w, Rect( 4, 4, 120, 26 )) .canFocus_( false ) // disable ugly focus border, we don't need it .items_( rates.collect( _.asString )) .action_({ arg view; dragSource.object = rates[ view.value ]}); w.front; )
Another useful "plusGUI" is browse
which can be called on any class:
JSCDragView.browse; // show the class browser for JSCDragView
To visualize data, plot
and scope
can be used. plot
works "offline" and can be called on an Array
, Signal
, Buffer
or Env
object. scope
is a realtime tool and can be called on a UGen-Graph-Function
, a Server
or a Bus
:
// 1000 samples from the cauchy distribution centered around 0.0 Array.fill( 1000, { 0.cauchy }).plot; // a basic envelope Env.linen( attackTime: 0.1 ).plot; // microphone input signal s.waitForBoot({ Bus( \audio, s.options.numOutputBusChannels, 1 ).scope }) // some synth s.waitForBoot({{ Saw.ar( mul: 0.25 )}.scope })
Sometimes the ready-made components that come with SwingOSC are not
sufficient for your GUI demands. In this case, you have two options:
either you develop a custom Java (Swing) component – something we will be looking at in chapter IX –, or (a bit easier) you develop a custom component in SuperCollider, using the JSCUserView
class. A JSCUserView
at first is a very plain thing. The actual component rendering is performed by assigning a drawFunc
function which utilizes the special JPen
class. JPen
contains methods for painting basic shapes such as lines, rectangles, circles etc. Here is a simple peak meter view:
( // we store the current GUI and it's pen class (e.g. JPen) // in a variable because they might change while the component // exists and would thus produce an error when the Swing // user view tries to render using the cocoa Pen... var gui = GUI.current, pen = gui.pen, pp = 0, numSegments = 8, decibelsPerSegment = 4.5, colors, synth, resp; colors = Array.fill( numSegments, { arg i; Color.hsv( i / numSegments * 0.5, 1.0, 0.5 ); }); w = gui.window.new( "Meter", Rect( 200, 200, 128, 200 )); w.view.background = Color.black; v = gui.userView.new( w, Rect( 44, 4, 40, 192 )) .canFocus_( false ) // so we don't see the focus border .resize_( 4 ) // the view grows vertically when the window is resized! .drawFunc_({ arg view; var bounds, peakSeg; // view.bounds returns the rectangle bounds of the view // relative to the top left corner of its window bounds = view.bounds; // to simplify drawing we shift and scale the coordinate system pen.translate( bounds.left, bounds.top ); pen.scale( bounds.width, bounds.height ); peakSeg = (pp.ampdb.neg / decibelsPerSegment).clip( 0, numSegments ).asInteger; if( peakSeg < numSegments, { (peakSeg .. (numSegments-1)).do({ arg i; pen.fillColor = colors[ i ]; pen.fillRect( Rect( 0, i / numSegments, 1, 0.8 / numSegments )); }); }); }); s.waitForBoot({ synth = { var inp, peakPeak, trig; inp = AudioIn.ar( 1 ); trig = Impulse.kr( 20 ); peakPeak = RunningMax.ar( inp, trig ) - RunningMin.ar( inp, trig ); SendTrig.kr( trig, 0, peakPeak ); }.play; resp = OSCpathResponder( s.addr, [ '/tr', synth.nodeID ], { arg time, resp, msg; pp = msg[ 3 ]; { v.refresh }.defer; }).add; }); // a function that get's called when the window is closed: // stop the metering synthesizer w.onClose = { synth.free; resp.remove }; w.front; )
Note: the view is repainted using v.refresh
. This is placed inside a { }.defer
block in order to make it compatible with cocoa GUI. While swing GUI
doesn't have that restriction, in cocoa GUI (Mac OS X native) methods
on components can only be called inside the AppClock
thread. { }.defer
makes sure its body is executed on that thread.
When designing a GUI, there is a useful pattern that we can follow. It is called MCV = Model-Controller-View because it divides the interactivity process into these three parts:
(public domain via en.wikipedia.org)
The idea is that we have some object that can be manipulated, the model. The model is visually presented by the view and manipulated by the view or any other controller (such as evaluating text in SC, or MIDI input etc.). The crucial point is that the model doesn't know about the view, hence the user interface can be changed or omitted later without destroying the code or loosing functionality.
My suggested way of implementing a MCV like structure in SC is to use a very basic mechanism that is built into every Object: Dependant-registration. It works like this:
~model = Dictionary.new; ~ctrlSet = { arg key, value; ~model.put( key, value ); ~model.changed( key, value )}; w = GUI.window.new.front; ~viewA = GUI.slider.new( w, Rect( 4, 4, 380, 26 )); ~viewB = GUI.slider.new( w, Rect( 4, 34, 380, 26 )); ~ctrlA1 = { arg view; ~ctrlSet.value( \a, view.value )}; ~viewA.action = ~ctrlA1; ~viewA.onClose = { ~ctrlA2.remove }; ~ctrlA2 = Updater( ~model, { arg obj, key, val; if( key === \a, {{ ~viewA.value = val }.defer })}); ~ctrlB1 = { arg view; ~ctrlSet.value( \b, view.value )}; ~viewB.action = ~ctrlB1; ~viewB.onClose = { ~ctrlB2.remove }; ~ctrlB2 = Updater( ~model, { arg obj, key, val; if( key === \b, {{ ~viewB.value = val }.defer })});
The Updater
class calls addDependant
on the model. The model keeps a list of dependants. When the model's changed
method is called, all dependants are notified about the change and can
act accordingly. This way we can add logic that operates on the model
without having to know about all the dependants (i.e. view or the
controller for the view):
~rout = fork { inf.do({ ~ctrlSet.value( \a, ((~model[ \a ] ? 0) + 0.1.bilinrand).wrap( 0 ,1 )); 0.1.wait })}; ~rout.stop; ( s.waitForBoot({ ~synth = { var inp, peakPeak, trig; inp = AudioIn.ar( 1 ); trig = Impulse.kr( 20 ); peakPeak = RunningMax.ar( inp, trig ) - RunningMin.ar( inp, trig ); SendTrig.kr( trig, 0, peakPeak ); }.play; ~resp = OSCpathResponder( s.addr, [ '/tr', ~synth.nodeID ], { arg time, resp, msg; ~ctrlSet.value( \b, msg[ 3 ].clip( 0, 1 )); }).add; }); ) // the model continues to work without the GUI: w.close; ~bang = Updater( ~model, { arg obj, key, val; if( key === \b and: { val > 0.5 }, { "Bang!".postln })}); ~resp.remove; ~bang.remove;
If you wish to integrate other java gadgets for which no
implementations exists in SuperCollider, there is two approaches: The
first one is fast and well suited for presentation-gadgets. Using the JSCPlugView
class, you can easily add new components to a window. The limitation
here is the missing automatic invocation of action functions. The
second approach is to write a proper subclass of JSCView
. Often you can use the first approach to prototype that view.
For example, we might want to have a JSpinner
component. The functionality of JSpinner
is similar to JComboBox
(aka JSCPopUpMenu
),
but it doesn't show a popup menu, instead an up and down arrow allow
the user to step through the possible items. A spinner looks like this:
SwingOSC can be used to rather easily script the java language.
That is, we can create an manipulate java objects through a proxy on
the SuperCollider client side, using the JavaObject
class:
// create an instance of java.awt.Frame ~jframe = JavaObject( "java.awt.Frame" ); // all method calls to the object get forwarded to the // server who tries to find the appropriate java method to call... ~jframe.setSize( 200, 300 ); ~jframe.setTitle( "Schnuck" ); ~jframe.setVisible( true ); // when we are done, we should destroy the object reference // on the server to allow garbage collection ~jframe.dispose; // this is a method in java.awt.Frame! the object still exists! ~jframe.destroy; // this deletes the object reference // to return primitive values to SC, append an // underscore to the method name. Warning: since the // communication with OSC cannot be performed inplace, // the method call must be wrapped into a Routine (that's what 'fork' does)! // // Example: create an instance of java.util.Random ~jrand = JavaObject( "java.util.Random" ); // query a new random value fork { ~jrand.nextFloat_.postln }
JSCPlugView
simply takes an existing JavaObject
and wraps it in a handler that is compatible with JSCView
, so you can use it in the regular GUIs:
~spinListModel = JavaObject( "javax.swing.SpinnerListModel" ); ~spinListModel.setList( List[ "Apple", "Pear", "Banana", "Mango" ]); ~spin = JavaObject( "javax.swing.JSpinner", nil, ~spinListModel ); w = JSCWindow.new.front; JSCPlugView( w, Rect( 4, 4, 200, 30 ), ~spin ); ~spin.setValue( "Mango" );
Using the underscore style, you can query the currently selected value:
fork { ~spin.getValue_.postln };
... but we would rather want to be automatically informed about user actions. We have solved this problem by writing a JSCView
subclass that creates an instance of de.sciss.swingosc.ChangeResponder
,
a helper class that attaches itself to the view and when the user
modfies the value, the change is forwarded to SuperCollider via OSC.
The ChangeResponder ist created like this:
JavaObject( "de.sciss.swingosc.ChangeResponder", this.server, this.id, \value )
this.id
which is the reference to the JSpinner
to listen to
\value
which is the property to query and send back upon user action. So, when the user manipulates the spinner, getValue
is called and the result sent back by the ChangeResponder
using an OSC-Message [ '/change', <spinnerID>, \performed, \value, <currentValue> ]
Here is the full class:
// SIMPLE TEST CLASS FOR DEN HAAG SYMPOSIUM ! JSCSpinnerDenHaag : JSCView { var <items, <value = 0; var acResp; // OSCpathResponder for change listening var model; // JavaObject of javax.swing.SpinnerListModel var changeResp; // JavaObject of de.sciss.swingosc.ChangeResponder var spin; // JavaObject of javax.swing.JSpinner value_ { arg val; value = this.prFixValue( val ); if( items.size > 0, { spin.setValue( items[ value ]); }); } prFixValue { arg val; ^val.clip( 0, items.size - 1 ); } items_ { arg array; items = array; model.setList( items.asList ); } prClose { model.destroy; changeResp.remove; changeResp.destroy; acResp.remove; ^super.prClose; } prInitView { var result; acResp = OSCpathResponder( server.addr, [ '/change', this.id ], { arg time, resp, msg; var newVal = items.indexOfEqual( msg[ 4 ].asString ); if( newVal.notNil and: { newVal != this.value }, { value = newVal; { this.doAction }.defer; }); }).add; spin = JavaObject.basicNew( this.id, this.server ); model = JavaObject( "javax.swing.SpinnerListModel", this.server ); result = this.prSCViewNew([ [ '/local', this.id, '[', '/new', "javax.swing.JSpinner" ] ++ model.asSwingArg ++ [ ']' ] ]); changeResp = JavaObject( "de.sciss.swingosc.ChangeResponder", this.server, this.id, \value ); ^result; } }
... and here some test code:
( w = JSCWindow.new; x = JSCSpinnerDenHaag( w, Rect( 4, 4, 200, 30 )) .action_({ arg view; ("Selected index is " ++ view.value ++ "; item is " ++ view.items[ view.value ]).postln }); w.front; ) x.items = [ "Apple", "Pear", "Banana", "Mango" ];
To access classes that are not part of the Java SE and which are not
in the system class path, you will need to add them to the dynamic
class loader, using the addClasses
method in SwingOSC
. Here is an example for JFreeChart (download from sourceforge.net/projects/jfreechart):
( // assuming you have downloaded jfreechart-1.0.6, add these two // jars to the class path (replace the dictory with your JFreeChart // install dir!) x = "file:///Users/rutz/Desktop/jfreechart-1.0.6/lib/"; g.addClasses( x ++ "jfreechart-1.0.6.jar", x ++ "jcommon-1.0.10.jar" ); )
Now all classes in those two jars should be accessible via SwingOSC. We create a simple pie-chart:
( var data, plot, gen; data = JavaObject( "org.jfree.data.general.DefaultPieDataset" ); Dictionary[ ("Burundi" -> 90), ("Ethiopia" -> 110), ("Democratic Republic of Congo" -> 110), ("Liberia" -> 110), ("Malawi" -> 160), ("Guinea-Bissau" -> 160), ("Eritrea" -> 190), ("Niger" -> 210), ("Sierra Leone" -> 210), ("Rwanda" -> 210.0)] .keysValuesDo({ arg key, value; data.setValue( key,value )}); plot = JavaObject( "org.jfree.chart.plot.PiePlot", nil, data ); data.destroy; gen = JavaObject( "org.jfree.chart.labels.StandardPieSectionLabelGenerator", nil, "{0} ({1})" ); plot.setLabelGenerator( gen ); gen.destroy; ~chart = JavaObject( "org.jfree.chart.JFreeChart", nil, "Ten Poorest Countries", JSCFont( "Helvetica", 24 ), plot, true ); plot.destroy; )
Now display it, using org.jfree.chart.ChartPanel
wrapped into a JSCPlugView
:
( w = JSCWindow( "JFreeChart", Rect( 200, 200, 560, 440 )); JSCPlugView( w, Rect( 2, 2, 556, 396 ), JavaObject( "org.jfree.chart.ChartPanel", nil, ~chart )) .onClose_({ ~chart.destroy }) .resize_( 5 ); JSCStaticText( w, Rect( 2, 400, 556, 36 )) .resize_( 8 ) .align_( \center ) .string_( "(based on 2004 GNP per capita in US$)" ); w.front; )
The result should look similar to this: