Chunity Progress Blog

Importable ChucK

I've done a lot of work toward making ChucK fully importable as a general library or DLL. This entailed adding a layer of abstraction between various ChucK functions called the Chuck_External class (name not final yet). This class has functions for launching and quitting ChucK VMs, setting callbacks for print statements, interacting with ChucK external variables, adding code to the VM, and calling the audio callback. (Using this class, you are responsible for calling the audio callback yourself.)

To do this, I did a lot of work eliminating the g_vm ("global VM") variable; since there can now be more than one VM, operations that work on a "global" VM are no longer valid. The one remaining area that I have left to address is in chuck_dl.cpp, which is a series of utility functions for chugins (dynamically linked libraries) that want to work with ChucK.

In the future, I'll add some additional functionality to this API, but for now, this is plenty for working with ChucK in Unity and will probably be fine for anyone who wants to embed ChucK in another programming environment as well!

Files and Chugins

It's now possible to use audio files and Chugins in Chunity!

Unity provides a special named folder, "StreamingAssets", for any files that should be directly copied to the filesystem rather than wrapped in a compressed data bundle.

Now, there is an option in ChucK to specify a data directory. This directory will be used to specify the directory of any shreds that are run from a string, rather than a file, and thus whose directory is not known. In Unity, this directory is automatically specified as the StreamingAssets directory, so me.dir() + "filename.wav" refers automatically to any files in the StreamingAssets folder.

The data directory is also automatically scanned for Chugins, so any Chugins placed anywhere in the StreamingAssets folder will automatically be loaded!

There was a small issue with getting some Chugins to work that involved communication with strings. In particular, the Faust Chugin cannot be compiled with the same compiler as Chunity, and the two compilers have different definitions of the C++ class std::string. To address this, I rewrote Chuck_String so that its internal std::string is effectively immutable, allowing it to also store a valid char pointer to its string at all times. Chugins (which are really just DLLs) can access this char pointer and use it to construct a std::string that will be valid according to their own definition of std::string.

This script shows Faust running inside ChucK running inside Unity/C#!

External Variables

There is now a bridge between currently running ChucK scripts and Unity!

I added the keyword "external" to the ChucK language, which can be applied to ints, floats, and Events that are in the global scope.

External ints and floats can be set from Unity, and the values can also be retrieved by passing in a callback function. External Events can be signaled or broadcasted.

External variables are accessed just through their string name and the VM they are in. What this means is that there can't be more than one global external variable with the same name in different shreds in the same VM. Currently, a new shred will display an error message and exit without running if it attempts to declare an external variable when the VM it is running on already has an external variable with that name.

An alternate solution would be to create an Externals public class that all shreds in a VM can access. The downside to this approach would be that the names of absolutely all external variables would need to be known when the class is first declared, which isn't the case for the current solution.

Perhaps a hybrid of these two approaches would be to allow static member variables of public classes to also be external. I will explore this possibility in the future.

This script an example of external variables working. It uses an external float and an external Event. Elsewhere in Unity, the float is set and then the Event is signaled, and then the ChucK script is able to continue on from waiting on the event, using the new value in the external variable.

Seeing ChucK Logs and Errors

I added a compiler flag that, if enabled, redirects all chout, cherr, stdcout, and stdcerr messages to callback functions that are specified externally for each type. This way, a Unity callback that takes in a string and displays it to the Unity debug log can be used to see all ChucK error messages!

This was easy to do for chout and cherr. For stdcout and stdcerr, I had to systematically go through the entire codebase and replace all calls to fprintf and std::cout (and similar variants) with new macros such as CK_FPRINTF_STDERR. These macros perform the original functionality unless the specific compiler flag for the callbacks is set. Then, they instead redirect the output to the specified callbacks.

Here's an example of one of the log callbacks working in Unity.

Chunity on Windows

ChucK is now working in Unity on Windows! There were a few major hiccups that I addressed in getting ChucK to compile within Unity's audio plugin wrapper on Windows:

• Setting up Virtual Studio: it took a certain amount of testing to get the preprocessor flags right.
• pthread.h: I spent a while trying to include a variant of the pthread library, which is used by the liblo osc library, but eventually discovered that the liblo source file that uses pthread (server_thread.c) isn't necessary to build ChucK. It's listed in the Windows makefile but not in the official ChucK Visual Studio project. (Similarly, a regex library is not listed in the makefile but needs to be included in Visual Studio.)
• liblo / cstdint clobbering: When the liblo headers are imported, they define some standard types for themselves. If a library that relies on cstdint is later imported, such as string.h, then the compilation for that file will fail. This happens only in the ChucK source file ulib_opsc.cpp. So, for this file, cstdint must be imported before the lo_types file is imported. Importantly, this import must be only added for the ChucK source file; if it's included for the liblo source files as well then their compilation will fail. This selective importing is achieved with additional preprocessor flags.
• DirectX version: ChucK for Windows uses DirectX 5 to read mouse and keyboard inputs. The earliest version compatible with a 64-bit architecture is DirectX 8. The code required a bit of massaging to fit into DirectX 8, and will require more testing to make sure nothing has been broken.
• Function pointers: on Mac OSX and Linux, a "long" type is 4 bytes on a 32-bit architecture and 8 bytes on a 64-bit architecture, making them a usual choice for storing pointers. However, on Windows a long is always 4 bytes and the 8 byte storage type is "long long". A more stable solution would be to use a more exact type like "int64_t", but for now, I added an extra preprocessor flag that uses long long across the codebase instead of long.
• Unity optimization: I discovered that Unity will disable all audio in a project if there are no audio sources (i.e. widgets attached to game objects that play audio files). It's necessary to connect an audio source to an audio mixer, but it's not necessary to give the audio source an audio file.

Starting Out / API

I spent a week working on creating a ChucK plugin for the Unity game engine, so that ChucK scripts could be constructed and launched live in Unity projects.

This plugin was necessary for a future research area I will get into, and may also be useful for one of the other students in Music 220c who is working in Unity.

To do this, I had to create a Native Audio Plugin in C++, which has access to the ChucK C++ source code and can call functions from it.

The current API is pretty simple to use. The ChucK plugin is instantiated in the Unity Audio Mixer, like any other Unity Plugin. Then, the user exposes the ChucK ID parameter and names the exposed parameter with an id that is used to access the plugin programmatically.

In a C-Sharp script, the user creates a public AudioMixer variable and drags their Audio Mixer into the slot created in the GUI. Then, the ChucK plugin must be instantiated in a Start() function:

After this, any ChucK script can be constructed programmatically and sent to that ChucK instance at any time:

There were three main sources of difficulty in doing this:

• Instantiating multiple ChucKs: I needed to be able to instantiate multiple ChucK virtual machines so that users could route different ChucK scripts through different Unity DSP signal flows (for example, to take advantage of Unity's spatializer for some ChucK scripts but not for ones that make up the background music). This was difficult because ChucK was architected in a way to have many global resources and expects there to be only one virtual machine accessing them. However, I was able to make a pretty stable solution where there are multiple copies of the virtual machine and overall "Chuck System", but only one copy of the compiler, the environment (which is reset between compiling each script), and the "BBQ" (responsible for some audio and MIDI tasks).
• Communicating between the plugin and user code: Unity only allows plugins to be communicated with directly by getting and setting floats. These getters and setters can be customized, but there is a caching layer in the GUI that ignores the customization. So, for example, if you programmatically set a parameter to "5" in Unity C-Sharp code, this 5 will be sent to the GUI cache and then onto the plugin. The plugin can react: "no, 5 is too high, I will set it as 1 instead", and that works for the plugin's internal storage, but the GUI will continue to tell Unity C-Sharp that the parameter setting to 5 was successful and that the parameter's value is 5. I solved this by storing ChucK instances not only in the plugins but also in an external data structure that can be accessed through external API calls. Then, a helper C-Sharp script sets an ID value for each plugin, and that ID is stored on both ends. Then, calls to add ChucK scripts bypass the plugin interface and instead access the auxiliary data structure via the stored ID.
  
• Unity optimization: if there are long periods of absolute silence while the plugin is running, then Unity will disable it completely (not calling its callback at all) periodically. For me, the callback was only being called on the 1st, 3rd, 5th, etc. times of the app execution. So, I added white noise with amplitude 0.00017 to every frame of every signal (0.00015 was too low), and now Unity doesn't disable my plugins anymore. This was very difficult to diagnose, but makes sense perhaps in hindsight. I don't think the white noise is very perceivable.

On top of this, I had no access to a console of print statements from the C++ code, so I had to debug by strategically forcing crashes or infinite loops and observing the stack trace reported by Apple.

Right now, I have a working, crash-free version of the Chunity plugin that works on my own computer! The next steps are to see about compiling the plugin on other architectures. In particular, I expect there will be some difficulty in compiling things on Windows, which is the most important platform for games development.

I will also see about removing the platform-specific audio code from this ChucK plugin, because it's not necessary when ChucK's audio is being pulled from an outside audio callback. This might make it easier to compile the Chunity plugin on multiple platforms. However, as far as I can tell, ChucK's code for audio callbacks is intermingled with its code for MIDI and HID devices, so this idea might not be worth the time put into it.