UT3OSC is an implementation of the Unreal3 engine built using the Unreal Development Kit(UDK) with an integrated OscPack Windows dll linked to custom UnrealScript classes. First implementation will be used for the 2010 September 7,8 performances of Tele-Harmonium and Perkussionista in Milano, Italy at the 2010 MiTo (Milano-Torino) Festival.

By linking custom UnrealScript classes to Ross Bencina's excellent OscPack Open Sound Control c++ implementation, actions and motions taken by virtual actors in the Unreal3 game engine are encapsulated as OSC messages over UDP and used to control dynamic music and sound synthesis systems in a manner similar to q3osc.

Currently, OSC output from game engine includes:

User XYZ coordinate data
fired projectile coordinate data
Static mesh objects XYZ coordinate data
Static mesh bump event
Static mesh takedamage event

Custom engine features include:

bouncing projectiles (customized OSCProj_ShockBall and OSCProj_LinkPlasma)
global projectile destructor exec command

Overview

UT3OSC is the fourth project in a lineage of integrating virtual worlds and environments with interactive audio and music servers with a common goal of creating shared and distributed musical performance spaces. Like our custom Sirikata implementation for the 2009 MiTo festival, q3osc and Maps & Legends before, UT3OSC combines a game-based virtual world implementation (in this case the Unreal Development Kit or UDK) with customized classes to stream game-data to interactive sound-servers using Open Sound Control messages over UDP. In this manner, gestures, motions and actions from actors in virtual space can be transformed in real-time into control messages for complex audio and musical software systems.

While the UDK offers only free access to the engine's UnrealScript programming layer (leaving the core C++ engine accessible only through significant financial investment) UnrealScript has a provision for binding Windows Win32 .dll's to UnrealScript classes, enabling code such as a C++ implementation of Open Sound Control (in this case, Ross Bencina's OSCPack) to provide bi-directional communication between game engine and sound-server. By then creating mirrored data structures both in UnrealScript classes and in the Windows .dll, data can be passed back and forth between the UDK and any OSC-enabled control system or software, including ChucK, Supercollider, Pure Data (PD) and Max/MSP.

For UT3OSC, a custom Windows oscpack_1_0_2.dll was compiled with specific extern methods and mirrored data structures to communicate with the game engine, both to stream specific game data out as OSC over UDP (including Pawn XYZ coordinate positioning, XYZ coordinate tracking for in-game projectiles, state and XYZ coordinate info for Static Mesh and Interp actor classes) and to send control data from iPad controllers and from the audio-engine itself back into the game to control Global GameInfo parameters such as world Gravity and GameRate, as well as specific positioning information for projecties and static actors.

Custom Classes and Files

UDK\UDK-2010-07\Binaries\Win32\UserCode

oscpack_1_0_2.dll

UDK\UDK-2010-07\UDKGame\Config

DefaultGame.ini
DefaultOSC.ini
DefaultUT3OSC.ini
DefaultEngine.ini

NOTE: Default .ini files generate UDK*.ini files when the engine runs.

UDK\UDK-2010-07\Development\Src\UT3OSC\Classes

ActorFactoryOSCInterpActor.uc
ActorFactoryOSCKActor.uc
ActorFactoryOSCKActorFromStatic.uc
ActorFactoryOSCStaticMeshActor.uc
DefaultGame_QT3OSC.ini
OSCInterpActor.uc
OSCKActor.uc
OSCKActorFromStatic.uc
OSCParams.uc
OSCPawn.uc
OSCPlayerController.uc
OSCPlayerControllerDLL.uc
OSCProj_LinkPlasma.uc
OSCProj_ShockBall.uc
OSCStaticMeshActor.uc
OSCWeap_LinkGun.uc
OSCWeap_ShockRifle.uc
OSCWorldInfo.uc
ut3osc.uc

Exec Methods

Exec methods can be run from the UDK command-line in-game. The command-line can be accessed by pressing "~" for the full drop-down terminal window or by pressing "TAB" for the single-line window-bottom command-line. These are the custom UT3OSC exec methods that have been implemented to this point:

destroyAllProjectiles
destroyAllShockBalls
destroyShockball #

OSCStartOutput
OSCStopOutput
OSCStartInput
OSCSendDeltas

OSC Output

Player output:

/player "bob" 85.474144 -48.000000 -479.850006 [True]

<tag> <player-name> <X float > <Y float > <Z float> <crouch bool>

Projectile output:

/projectile "OSCProj_LinkPlasma_0" "plasma" 141203.968750 13437.806641 -1752.857056

<tag> <projectile-name_instanceCount string> <projectile-type string> <X float> <Y float> <Z float>
* TO ADD: explode bool

current projectile-names: OSCProj_LinkPlasma, OSCProj_SeekingRocket, OSCProj_ShockBall
current projectile-types: "plasma", "homingrocket" "shockball" 
* TO CHANGE: "homingrocket" to "seekingrocket"

Getting Started

Install July version of the UDK from: July 2010 UDK Beta (777 MB .exe)
Follow the Subversion installation instructions below to install files to the correct locations.

There are only 4 locations where files need to be instlled:

UDK-2010-07\Development\Src\UT3OSC: these are the custom OSC UnrealScript classes
UDK-2010-07/Binaries/Win32/UserCode: the oscpack windows dll
UDK-2010-07/UDKGame/Config: the custom default config files
UDK-2010-07/UDKGame/Content/Maps: custom maps go here

Open UnrealFrontend to compile UT3OSC

?game=UT3OSC.UT3OSC should be pasted into the "Extra Options" field
check the "show log" check box to see debug messages in the console

Launch Map

DETAILS COMING SOON

SETUP OSC CLIENT FOR MONITORING

DETAILS COMING SOON

Subversion Setup

Copy UT3OSC data into the Src dir:

cd D:/UDK/UDK-2010-07/Development/Src

svn checkout svn+ssh://rob@ccrma-gate.stanford.edu/user/r/rob/data/mvw/svn/udkosc/UDK/UDK-2010-07/Development/Src/UT3OSC ./UT3OSC/

Checkout oscpack dll:

cd D:/UDK/UDK-2010-07/Binaries/Win32/UserCode

svn checkout svn+ssh://rob@ccrma-gate.stanford.edu/user/r/rob/data/mvw/svn/udkosc/UDK/UDK-2010-07/Binaries/Win32/UserCode .

Checkout ini files:

cd D:/UDK/UDK-2010-07/UDKGame/Config

svn checkout svn+ssh://rob@ccrma-gate.stanford.edu/user/r/rob/data/mvw/svn/udkosc/UDK/UDK-2010-07/UDKGame/Config .

Checkout test Maps:

cd D:/UDK/UDK-2010-07/UDKGame/Content/Maps

svn checkout svn+ssh://rob@ccrma-gate.stanford.edu/user/r/rob/data/mvw/svn/udkosc/UDK/UDK-2010-07/UDKGame/Content/Maps .

Checkout oscpack_1_0_2.dll Source + VS2010 Project [optional]

svn checkout svn+ssh://rob@ccrma-gate.stanford.edu/user/r/rob/data/mvw/svn/udkosc/oscpack/ .

NOTE: also includes .zip of source project

Timeline

8/11/2010

Added crouch tracking in OSCPawn output, and OSCStartInput, OSCStartOutput, OSCStopOutput, and OSCSendDeltas exec functions.

Working on extending "Use" function:

http://forums.epicgames.com/showthread.php?t=717602&highlight=PerformedUseAction

found in PlayerController.uc

/**

* Entry point function for player interactions with the world,
* re-directs to ServerUse.
*/

exec function Use() { if( Role < Role_Authority ) { PerformedUseAction(); } ServerUse(); }

/**

* Player pressed UseKey
*/

unreliable server function ServerUse() { PerformedUseAction(); }

8/10/2010

added initOSCReceiver function to OSCPawn to initiate OSC input polling from the dll.

updated Max/MSP gui OSC sender applet for testing global gameGravity and gameSpeed params over OSC

8/09/2010

Mapped incoming OSC test floats to GameSpeed and Gravity successfully.

Can access WorldInfo and current Game vars and methods stored in ut3osc.uc:

UT3OSC(WorldInfo.Game).fingerTouchArray[0].X=val;

8/08/2010

Code re-written to access OSC params in OSCParams.uc class. Now Hostname and Port can be set through the DefaultOSC.ini

Updated code checked into SVN

8/03/2010

Successful test of OSC messages being sent into UDK via OSCPack dll listener, operating on its own thread.

Pawn motion:

http://www.etc.cmu.edu/projects/coyote210/Docs/undox/Engine.Pawn.html

MoveTo and MoveToward methods

void	MoveTo(vector NewDestination, optional float)
     
// Latent Movement. Note that MoveTo sets the actor's Destination, and MoveToward sets the actor's MoveTarget.  Actor will rotate towards destination

void	MoveToward(Actor NewTarget, optional float)

UDKProjectile

/** Currently tracked target - if set, projectile will seek it */ var actor SeekTarget;

Projectile

Can target be changed just by setting Velocity (Speed * Direction vector)?

/* Init()
initialize velocity and rotation of projectile
*/
function Init( Vector Direction )
{
	SetRotation(Rotator(Direction));
	Velocity = Speed * Direction;
}

Actor (projectiles inherit from Actor)

// The actor's position and rotation.
/** Actor's location; use Move or SetLocation to change. */
var(Movement) const vector			Location;

/** The actor's rotation; use SetRotation to change. */
var(Movement) const rotator			Rotation;

Added "MoveAllProjectiles X Y Z" as exec function: instantly moves all projectiles to a specific location.

Working on

exec function setAllProjectileSpeed(int speed)
{
    local UTProjectile pUT; 

  	ForEach AllActors(class'UTProjectile', pUT)
	{
		pUT.Speed = speed;
	}
}

8/01/2010

Checked in modifications to July UDK to svn at ccrma. Note only modified files/custom files were checked in, including the source for the oscpack windows dll, and these should be checked out into a 2010-07 install of the UDK.

7/31/2010

OSCInterpActor class

- outputs position data on move, bump and damage as well
- Kismet script tested successfully to spit out position data while object moves

7/27/2010

- Moved to July UDK

Current Feature List

- OSC dll in place
- Tracking XYZ data in customized player controller sending OSC data
- Bouncing Projectiles
- Tracking projectile XYZ data sending OSC data
- customized Actor class for OSC on bump and touch events (not tested yet)

6/13/2010

First Words: ?game=UT3OSC.UT3OSC needs to be passed in "Extra Options" in Unreal FrontEnd

OSC message is being sent from Terminal on typing "Test1"

- first impressions are that it's a bit slow... lag/latency maybe UnrealScript?

[COMMANDLET 'UDK.exe make -full' STARTED IN ] June 13, 12:23 AM

Init: Version: 6522
Init: Epic Internal: 0
Init: Compiled (32-bit): Apr  9 2010 12:14:06
Init: Command line:  -full -DEFENGINEINI=..\..\UDKGame\Config\DefaultEngineUDK.ini
Init: Base directory: C:\UDK\UDK-2010-04\Binaries\Win32\
Init: Character set: Unicode
Log: Executing Class UnrealEd.MakeCommandlet
--------------------Core - Release--------------------
Analyzing...
Scripts successfully compiled - saving package 'C:\UDK\UDK-2010-04\Binaries\Win32\..\..\UDKGame\Script\Core.u'
--------------------Engine - Release--------------------
Analyzing...
Scripts successfully compiled - saving package 'C:\UDK\UDK-2010-04\Binaries\Win32\..\..\UDKGame\Script\Engine.u'
--------------------GameFramework - Release--------------------
Analyzing...
Scripts successfully compiled - saving package 'C:\UDK\UDK-2010-04\Binaries\Win32\..\..\UDKGame\Script\GameFramework.u'
--------------------UnrealEd - Release--------------------
Analyzing...
Scripts successfully compiled - saving package 'C:\UDK\UDK-2010-04\Binaries\Win32\..\..\UDKGame\Script\UnrealEd.u'
--------------------IpDrv - Release--------------------
Analyzing...
Scripts successfully compiled - saving package 'C:\UDK\UDK-2010-04\Binaries\Win32\..\..\UDKGame\Script\IpDrv.u'
--------------------OnlineSubsystemPC - Release--------------------
Analyzing...
Scripts successfully compiled - saving package 'C:\UDK\UDK-2010-04\Binaries\Win32\..\..\UDKGame\Script\OnlineSubsystemPC.u'
--------------------UDKBase - Release--------------------
Analyzing...
Scripts successfully compiled - saving package 'C:\UDK\UDK-2010-04\Binaries\Win32\..\..\UDKGame\Script\UDKBase.u'
--------------------UTEditor - Release--------------------
Analyzing...
Scripts successfully compiled - saving package 'C:\UDK\UDK-2010-04\Binaries\Win32\..\..\UDKGame\Script\UTEditor.u'
--------------------UTGame - Release--------------------
Analyzing...
Scripts successfully compiled - saving package 'C:\UDK\UDK-2010-04\Binaries\Win32\..\..\UDKGame\Script\UTGame.u'
--------------------UTGameContent - Release--------------------
Analyzing...
Scripts successfully compiled - saving package 'C:\UDK\UDK-2010-04\Binaries\Win32\..\..\UDKGame\Script\UTGameContent.u'
--------------------UnrealOsc - Release--------------------
Analyzing...
Scripts successfully compiled - saving package 'C:\UDK\UDK-2010-04\Binaries\Win32\..\..\UDKGame\Script\UnrealOsc.u'

Success - 0 error(s), 0 warning(s)
Execution of commandlet took:  22.67 seconds 

[COMMANDLET 'UDK.exe make -full' SUCCEEDED] June 13, 12:23 AM

6/1/2010

basic testing of Windows Oscpack dll writes message successfully over OSC from test app

__declspec(dllexport) double returnDouble(double a);
__declspec(dllexport) void sendOSCmessageTest();

random notes

Shortcuts

Link to Disable Windows key:

http://support.microsoft.com/?id=216893

Hide HUD and Weapon: togglescreenshotmode

http://udn.epicgames.com/Three/CharactersTechnicalGuide.html#Inventory and Weapons: WeaponFired [weapon] [bViaReplication] [hitLocation]
http://www.moddb.com/games/unreal-tournament-3/tutorials/unreal-learning-1-my-first-unreal-tournament-3-mutator

default gravity (game specific) - set in defaultgame.ini
Default value: -520.0

ProjectileList http://wiki.beyondunreal.com/UE3:WorldInfo_internal_variables_(UDK)

Moving to July UDK

Add UT3OSC references to UDKGame\Config\DefaultGame.ini

[Engine.GameInfo]
DefaultGame=UT3OSC.UT3OSC
DefaultServerGame=UT3OSC.UT3OSC
PlayerControllerClassName=UT3OSC.OSCPlayerControllerDll
DefaultGameType="UT3OSC.UT3OSC";

Add ModEditPackages line to section in UDKGame\Config\DefaultEngineUDK.ini

[UnrealEd.EditorEngine]
+EditPackages=UTGame
+EditPackages=UTGameContent
+EditPackages=CastleGame
ModEditPackages=UT3OSC

Exposed OSC Methods

Adding Hostname and Port param to structs to allow for multiple/dynamic OSC targets

sendOSCPlayerState(PlayerStateStruct)
- OSCPawn::Tick

- Hostname   (string)
- Port       (int)
- PlayerName (string)
- Location X (float)
- Location Y (float)
- Location Z (float)
- Crouch     (bool)

sendOSCpointClick(PointClickStruct)

- Hostname                 (string)
- Port                     (int)
- TraceHit                 (string)
- TraceHit_class           (string)
- TraceHit_class_outerName (string)
- Location X               (float)
- Location Y               (float)
- Location Z               (float)
- HitInfo_material         (string)
- HitInfo_physmaterial     (string)
- HitInfo_hitcomponent     (string)

DLL Binding

"A single UnrealScript class can bind to only a single DLL. The DLL to bind to is specified with the DLLBind directive, and the DLL name to bind to is specified in parentheses. Do not include a path or .DLL extension. DLLs can only be loaded from the Binaries\Win32\UserCode folder."
- http://udn.epicgames.com/Three/DLLBind.html