Neural Networks are a powerful tool for creating scientifically and sonically interesting audio effects. Some examples of existing audio effects that use real-time neural networks include:
When using a neural network in a real-time audio effect, it is common to train the network beforehand, so that in the effect itself, the network only needs to take the audio input, and produce an audio output. This process is known as running inference on a neural network, and requires an inferencing engine.
RTNeural is a C++ library designed to perform real-time audio-rate inferencing for pre-trained neural networks. The main design goals for the library were:
RTNeural supports four backends, to allow for maximum flexibility.
RTNeural contains a simple script for exporting the neural network weights from a trained model to a json format. The json file can then be loaded by RTNeural at runtime.
RTNeural is already being used in real-time audio plugins that are available now!
ChowCentaur is a guitar distortion effect plugin modelled after the famed Klon Centaur pedal. The plugin contains a "Neural" mode that uses a recurrent neural network.
CHOWTapeModel is a physical model of reel-to-reel analog tape. One of the processing modes uses a State Transition Network to help solve the Jiles-Atherton equations that describe magnetic hysteresis.