KeyWI image

The KeyWI

An Expressive and Accessible Electronic Wind Instrument

Matthew Wrighta
Romain Michona,b
Matthew Carena

aCCRMA, Stanford University, USA
bGRAME-CNCM, Lyon, France

NIME2020

Introduction

  • Wind instruments are favored for their ability to create a uniquely intimate connection between musician and instrument. Their ability to capture nuances in a musician's playing is essential to their versatility and expressiveness.
  • However, electronic wind instruments have not always been able to match the capabilities and characteristics critical to acoustic wind instruments.
  • The goal of the KeyWI is to create an intuitive and powerful electronic wind instrument that eliminates these obstacles, while being as open-source and reproducible as possible such that it can be used as a platform for development and be replicated by anyone with access to sufficient tools.
  • The KeyWI's design is based on the melodica, a handheld wind instrument where pitches are selected using a piano-style keyboard and dynamics are controlled by blowing into a mouthpiece attached to the side of the instrument. The use of a keyboard as a pitch controller rather than the more common woodwind-inspired designs aims to be accessible to users who may not have prior experience with the specialized fingerings required for wind instruments, as well as offering intuitive polyphonic capabilities.

History and Related Work

  • Due to acoustic wind instruments’ enduring popularity, there is a long history of electronic wind instruments, which have constantly been influenced by the technological capabilities and ideas of their times:
    • 1936 patent for an “Apparatus for the production of music” [8] that used an electromagnetic pickup placed in a clarinet reed to sense vibrations.
    • 1941 patent for an “Electric clarinet” [1] that featured a simple on/off sensor to detect breath pressure.
    • The 1967 Hohner Electra-Melodica was the first commercial electronic instrument with a breath pressure sensor. Little dynamic range and limited choice in timbre, however, resulted in low sales.
    • The Lyricon controllers [11] began production in the 1970s and featured a sturdier and more sensitive breath pressure sensor and a control system similar to acoustic wind instruments. It outputted control voltage that could be used as the input for an analog synthesizer, offering a wider range of possible sounds.
    • The popularization of MIDI in the 1980s led to the advent of MIDI wind controllers, including the Akai EWI and Yamaha WX (roughly based upon of the design of the Lyricon controllers) breath controllers. These controllers featured higher-resolution breath sensors and could be used with MIDI synthesizers and interfaces. Yamaha also manufactured breath sensors such as the BC3 that could be used with keyboard synthesizers such as the DX7.
    • Today, the WX and EWI lines of wind controllers continue to be the most prominent commercial breath controllers—they are similar in design and control to a clarinet or saxophone and can be used to control hardware or software synths.
  • There are also many non-commercial wind instruments and controllers. The Electronic Valve Instrument (EVI) [14], invented by Nyle Steiner in 1975, is an early electronic wind controller based on the trumpet's valve system. Controllers since have featured improvements and explorations in control systems, sensors, application, and construction [2,3,5].
  • The NIME community has also developed numerous electronic wind instruments and controllers. Gary Scavone's Pipe [13] is a general MIDI wind controller that contains a variety of sensors in a compact format. The Epipe [6] is a wind controller designed after open tonehole instruments that is capable of detecting the nuances in pitch control offered by such instruments. Perry Cook's VOMID and COWE controllers [4] are targeted towards controlling synthesized singing models, and incorporate both pitch and breath control as well as vocal synthesis-specific controls.
  • Generally, electronic wind instruments have developed in two ways:
    1. More accurate, capable, diverse, and reliable sensors and control systems;
    2. Greater freedom in sound and application.
  • However, modern instruments still tend to suffer from some of the same issues as previous electronic wind instruments, including including restricted dynamic range, transient detection, limitations in sound production, and capacity for control by non-wind instrument players.
  • The development of the KeyWI focuses on creating a wind instrument that reduces limitations in control and that is accessible and versatile enough to be used as a platform for development.

Analysis of Current Systems

  • We compared recordings of an acoustic melodica and various modeled counterparts in order to identify the most salient limitations in current technology. A sampling of various electronic wind instrument and breath controller systems were tested—the modeled instrument samples below were generated using a Yamaha VL70-m physical modeling synthesizer with a BC3 breath controller, and were largely representative of the findings.

Four key limitations in the modeled systems compared to the acoustic instrument were identified:

Attacks/Transients

  • Figure 1 compares the spectrograms of representative examples of acoustic and modeled melodica attacks. All samples were recorded with the same user trying to create as fast of an attack as possible, but note the difference in the attack times (time for all overtones to reach within 5 dB of steady state): ~50 ms for the model versus ~35 ms for acoustic. While attack times did vary between instruments, they were all significantly longer than that of an acoustic melodica.
spectrogram attack comparison

Figure 1: Spectrograms of the attacks of acoustic (right) and modeled (left) melodica, both played with as fast attacks as possible

Dynamic Range

  • To test dynamic range, the electronic systems and the melodica were played at five intensities of breath, ranging from as soft as possible to very hard. The limited dynamic ranges of the electronic systems were evident:
    • At low breath pressures, the output fluctuated erratically with clearly audible “steps” in the intensity of the signal, or simply did not register.
    • At the two highest breath pressures, there was almost no variation in the signal at all.
  • Unlike the electronic system, the melodica's output varies significantly at every level of breath pressure, indicating that the acoustic melodica has significantly more dynamic range than the electronic system.

Variance and Expressiveness

  • The timbre of an acoustic melodica is so responsive to breath pressure that the amplitudes of the overtones of an acoustic melodica fluctuate over time, even as a player attempts to maintain a constant level of breath pressure. In comparison, the overtones of the modeled systems are almost completely static. This responsiveness to varying breath pressures also determines the degree to which a user is able to intentionally shape the timbre of an instrument.

Pitch Control Interfaces

  • Many electronic breath controllers, such as the Pipe, Epipe, Akai EWI and Yamaha WX, feature pitch control systems based on acoustic wind instruments such as clarinets or saxophones. These methods of control are tailored specifically to wind instrument players, and do not support polyphony in any practical way.

The KeyWI

  • We developed the KeyWI guided by several overall objectives:
    1. Improving upon the identified limitations;
    2. Being accessible to as many users as possible;
    3. Open source hardware and software;
    4. Modular/adaptable such that it can be used in a variety of applications.
  • Sound is synthesized with Faust [12] running on the Bela Mini processing board [7].
    • Faust is a language that targets high-performance real-time digital signal processing. Its low latency, block-diagram driven structure and performance make it a logical choice for an accessible development platform.
    • Additionally, it has an extensive collection of libraries that include implementations of numerous oscillators, filters, envelope generators and effects, as well as physical models of several wind instruments. The Faust compiler can build directly to C++ Bela programs, which creates a very simple workflow.
    keywi attack spectrogram

    Figure 2: Block diagram representation of KeyWI setup

  • The control system is comprised of two main components: a MIDI keyboard and a breath pressure sensor:
    • The MIDI keyboard used is a MIDIPLUS AKM320. Only the keys, key contacts, and main circuit board from the controller are used; custom 3D-printed hardware replaces the casing, mounts, and other electronics. The keyboard sends key and velocity data to the main board via USB MIDI.
    • An MPXV4006DP pressure sensor measures breath input. It is mounted to a Modern Device MPXV breakout board, which in turn mounts to the PCB via a removable header. One pressure input of the sensor is left unconnected and the other is joined to a short vinyl tube that extends just outside of the casing.
  • A 3D-printed mouthpiece connects either directly to the vinyl tube, such that the KeyWI is brought up to the mouth to be played, or to a longer vinyl tube, so that the KeyWI can be rested on a table. A small hole in the side of the mouthpiece allows air to flow through the mouthpiece as well as augmenting the dynamic range: proportionally less pressure is exerted on the sensor as the user blows harder, so it takes more pressure to reach the sensor's limit.
    • keywi attack spectrogram
    keywi attack spectrogram

    Figure 3: Renderings of 3D-printed enclosure and mouthpiece

  • A custom-printed circuit board serves as the central hub for all connections going in and out of the Bela Mini. It mounts to the top of the Bela cape through headers soldered to the board, and the pressure sensor, output jack, and other components all mount to the PCB. The potentiometers control offset (pressure required to make a sound) and sensitivity (pressure needed to reach maximum), while an LED provides visual feedback when breath input is detected. The entire setup is powered by batteries located in the right side of the KeyWI.
  • The enclosure is comprised of nine laser-cut layers of birch plywood fastened together with four bolts, and the keyboard supports and main circuit board mount to the bottommost layer.
  • To be as accessible to as many users as possible, the KeyWI is entirely open source: project files, Bela software and hardware, and the Faust compiler are all open source. It can be assembled with only access to basic maker tools, common supplies, and Internet.

Results

  • We evaluate the KeyWI in comparison to an acoustic melodica, according to the four limitations of previous breath controller systems identified above in Analysis of Current Systems.

    • Attacks/Transients

    keywi attack spectrogram

    Figure 4: Spectrogram of KeyWI melodica attack

  • The MPX breath pressure sensor exhibits very good response to quick changes in pressure. As seen in Figure 4—captured using a sharp, “tongued” start to each note—the initial attack of the KeyWI melodica (about 25 ms) is much shorter than that of the modeled system tested above (about 50 ms).

    • Dynamic Range

  • The test of five varying breath pressures executed in Analysis of Current Systems was also conducted with the KeyWI melodica. The KeyWI melodica clearly exhibited responsiveness to each level of breath pressure—with the sensor output scaled between -1 and 1 (0 = no pressure, positive values = blowing, negative values = sucking), the pressure sensor returned values of 0.14 at the softest level of blowing and blowing as hard as possible returned a value of 0.91. The sensor also responded smoothly to low pressures, not exhibiting the stepping and unpredictable behavior of other tested systems.

    • Variance/Expressiveness

  • Figure 5 displays KeyWI melodica notes for a synthesized input (60% pressure with 20% amplitude modulation via a 2 Hz sine wave), and with a user attempting to maintain a constant level of pressure. Note how the synthesized modulation results in the modulation of the amplitudes of overtones and the overall signal. With the human input, the sensitivity of the pressure sensor results in natural variance in the pressure input, which is then reflected in a similar manner in the resulting audio (note the variance in the higher overtones and a slight dip in overall volume at about 2.5s).
    • keywi attack spectrogram

    Figure 5: Spectrograms of KeyWI note with synthesized (left) and user (right) pressure input

  • Some electronic systems attempt to add variance by introducing randomness to the sound synthesis algorithm. While this approach may be effective on some level, it decreases the musician’s level of control over the instrument and does not solve the underlying problem: the lack of sensitivity and precise control over the intricacies of the instrument. Since this type of precise control is what attracts many players to wind instruments, it seems counterproductive to mimic this expressiveness rather than giving control to the user themselves.

    • Pitch Control Interfaces

  • A piano keyboard was chosen as the basis of the control system due to its versatility and universality as a pitch controller. Using a keyboard rather than a design similar to a wind instrument allows for intuitive polyphonic control, as well as making the KeyWI accessible to a wider audience.
    • The number of keyboard players worldwide significantly outnumbers the number of woodwind players (in 2018, 1.4 million acoustic and electric pianos and keyboard systems were sold in the US compared to 350 thousand wind instruments [10]).
    • This large community of keyboard players creates a "pluggable community" (as explained by Morreale et al. [9]), from which users can transfer their existing skill set of playing a keyboard to the KeyWI.

    Development Workflow

  • Also central to the design of the KeyWI as a platform is its workflow and ease of development. The Faust compiler can build directly to a Bela C++ architecture, and the resulting file is simply imported onto the Bela Mini. Using the online Faust IDE and browser-based Bela IDE, this entire process can be executed in less than a minute, using exclusively GUI-based systems, without any setup or installations. This creates an accessible and efficient workflow that allows for quick iteration of ideas.
  • The sound synthesis for the KeyWI melodica is based on the symmetrical clipping of a sine wave in order to create the odd harmonics characteristic of the vibration of free reeds in a melodica. When a multiple of the un-clipped sine wave is subtracted from the clipped signal in order to partially remove the fundamental frequency, the result closely models the sound produced by a melodica. The severity of the clipping varies directly with breath pressure to appropriately model the change in timbre caused by changes in breath pressure.
    • Using this synthesis algorithm, the entire program (including data collection, melodica tone synthesis, transient envelope generation, breath noise synthesis, and mixing) requires just 25 lines of Faust code.

Conclusion and Next Steps

  • The KeyWI aims to improve upon several limitations in current systems and provide an accessible platform for more expressive, realistic, and musical instruments. It combines the power and versatility of Faust and the Bela Mini, a control system based on a piano keyboard rather than a wind instrument, and a custom PCB and enclosure for a fully integrated, digitally manufactured design.
  • Though a wind instrument, the KeyWI can be used as a development platform for a wide variety of digital musical instruments; the PCB provides a simplified interface for the Bela Mini that includes USB pins, audio out, various inputs including a breath sensor, and an external power button.
  • There are several possible additions that could expand its capabilities. Adding a microphone would allow for an even greater range of expression, including extended techniques such as singing and playing simultaneously, or speaking syllables into the instrument. A method of providing haptic feedback could also be implemented, which could be used to simulate the vibrations experienced when playing an acoustic wind instrument.

References

[1] L. F. J. Arnold. Electrical clarinet, November 10 1942. US Patent 2,301,184.

[2] G. T. Beauregard. Rethinking the design of wind controllers. PhD thesis, Dartmouth College, 1991.

[3] P. R. Cook. A meta-wind-instrument physical model,and a meta-controller for real-time performance control. In Proceedings of the international computer music conference, pages 273–273. International Computer Music Association, 1992.

[4] P. R. Cook. Real-time performance controllers for synthesized singing. In Proceedings of the 2005 conference on New Interfaces for Musical Expression, pages 236-237, 2005.

[5] T. Henriques. META-EVI innovative performance paths with a wind controller. In Proceedings of the 2004 conference on New Interfaces for Musical Expression, pages 307–310, 2004.

[6] S. Hughes, C. Cannon, and S.Ó. Modhréin. Epipe: A novel electronic woodwind controller. In Proceedings of the 2004 conference on New Interfaces for Musical Expression, pages 199–200, 2004.

[7] A. McPherson and V. Zappi. An environment for submillisecond-latency audio and sensor processing on BeagleBone Black. In Audio Engineering Society Convention 138, May 2015.

[8] B. F. Miessner. Apparatus for the production of music, November 29 1936. US Patent 2,138,500.

[9] F. Morreale, G. Moro, A. Chamberlain, S. Benford, and A. P. McPherson. Building a maker community around an open hardware platform. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, Colorado, USA, 2017.

[10] Music Trades. Music industry census. Music Trades, pages 88–94, April 2019.

[11] R. Noble and W. Bernardi. Electronic musical instrument, October 23 1973. US Patent 3,767,833.

[12] Y. Orlarey, D. Fober, and S. Letz. An Algebra for block diagram languages. In International Computer Music Conference (ICMA), pages 542–547, Gothenburg, Sweden, 2002.

[13] G. P. Scavone. The Pipe: explorations with breath control. In Proceedings of the 2003 conference on New Interfaces for Musical Expression, pages 15–18, 2003.

[14] N. A. Steiner. The electronic valve instrument (EVI), an electronic musical wind controller for playing synthesizers. The Journal of the Acoustical Society of America, 115(5):2451–2451, 2004.