The next step for me is to work towards an interface whose sound generation and manipulation is as an innate property of its design as was with the "Electric Goblets" mentioned above, but whose control is specific enough to allow for repeat performances of given pieces. (Pieces that are based largely on improvisation and/or sweeping and broad stochastic gestures do not require the level of control specificity with which I am concerned. Instead, it is a more traditional repeatability that I am after, as in classical pieces with specific notes and rhythms.) My participation in the Stanford Laptop Orchestra (SLOrk) this quarter is a great opportunity to develop and experiment with an interface of this type, and a short piece with which to perform it.
~

I have connected and installed my new Arduino Diecimila USB Board by Arduino. The software provided on the company website is very slick; it provides a single interface for coding, testing, and uploading programs to the microcontroller.

"Arduino is an open-source physical computing platform based on a simple i/o board and a development environment that implements the Processing/Wiring language. Arduino can be used to develop stand-alone interactive objects or can be connected to software on your computer (e.g. Flash, Processing, MaxMSP)."

"We now ship the new Arduino Diecimila.... Board comes fully assembled and tested with ATMega168 pre-loaded with bootloader...."

Specifications for the specific board are listed here.

Presently I am having some problems uploading software to the board, I receive an error message saying that the board is already in use, and that I should find the other program that is using it and quit that program. This is all well and good except that I am not running any other software that would pay any attention to the Arduino. I will test this further — besides this, the board works flawlessly.

As soon as I have the software working properly (or potentially even sooner, since this next step does not necessitate uploading to the AVR board), I will be looking into buying several bread-boards and building some ribbon wire connectors so that I can connect and disconnect different projects (each on its own bread-board) without rewiring the whole thing. Each project would exist on its own bread-board and would need only to be plugged into and its software uploaded onto the Arduino board to be ready to use. These steps should be relatively simple, but I am always skeptical whenever I think something like this is going to be easy (I cannot be trusted).

I have looked into existing apparatuses that take advantage of and provide resistance to actions that I would like to utilize performatively for this interface, and I have narrowed my choices (for the time being) to ProHands Hand Exercisers.

I have ordered a single "gripmaster" unit that provides freedom of motion to all four fingers as well as general pressure from the thumb at approximately 3 lbs per finger. If the tension (3 lbs per finger) is a good amount and the unit is able to be modified with sensors, I will buy two more (giving one to my significant other because she wants one) and complete modifications to both.
Presently I am still not sure what sort of sensors I would like to use on this unit / which would be best suited. Hall Effect Sensors might work on each finger, but I suspect that they would interfere with each other at such close proximity. Piezoelectric Sensors, I think are not at all suited to this soft of control, but I will look into it as well. The goal of attaching these sensors to the unit would be to be able to sense the exact positions of each of the finger exercisers to the computer and translate that information into meaningful control data. I am open to any and all suggestions as to directions in which I should look to accomplish this, or specific sensors that might work.
Another potential aspect of control that this unit provides (especially when used in conjunction with another unit) is its spatial orientation, motion, and distance to its partner unit. The first of these would require a gyroscope of some sort; these are expensive and I am not overly fond of the gestures required to excite such a sensor in this project's context. Motion may be sensed with an accelerometer; this is not out of the question, shaking the units or moving them slowly in any direction might play well with the more subtle gripping motions required of the pads, and accelerometers are not too too expensive. Proximity of the units may be sensed in several ways (and I am confident in many others of which I cannot presently think): Reflective Optical Sensors, or some wireless technology similar to what was used in Fall 2007's Music 250a Class by Chris Warren's group with their project "Roxx". Here I am open to suggestions too. An option that was also discussed in class (220C), and that would provide more interesting information to any sensors that sensed the unit's proximatey, would be connected the two units with some elastic material (to each other, or to the ground or some other surface, or some crazy combination of these things). The haptic feedback would add a great deal of humanism and musicality to the project, but the potential allusions to the accordion, and/or Michael Gurevich's Acordiotron, make me tentative to commit to this particular element of performative resistance.

More on this when my ProHands arrives, and/or when I get my Arduino working, and/or when I get the bread-board interchanging happening. Hi ho.

I have received my GRIP master hand exerciser and, though it is slightly smaller than I had expected, the resistance is a goodly amount, and, on the advise of my peers, it looks like Hall Effect sensors and magnets should work as a method of extracting control data from the unit.
Today I have purchased (online) 10 SS49E — HONEYWELL S&C — Hall Effect Magnetic Sensors and 25 GRADE N50/N52 MAGNETS as well as an attachment for my Arduino Diecimila USB Board, an Arduino ProtoShield Kit with a Breadboard Mini Self-Adhesive. The latter two purchases will allow me to connect a breadboard directly to the Arduino and so save on space, wiring, general clutter. The Hall Effect sensors are small enough to fit within the GRIP master with very little disruption of its primary grippy function.

I am still attempting to work out the problem with the software uploading on my Arduino board.

I have drawn and digitized a scale sketch of the GRIP master on which I will draft positions and spacings for the sensors and magnets.


~

I now have all my parts, I am going to be soldering my protoshield board together today as per the instructions on this website.
I will also be hardcore attempting to get the damned Arduino working, and will be testing the Hall Effects with my magnets, then attaching them to the GRIP MASTER.
~

With the "help", and by "help" I mean complete and total he-did-it-while-I-passed-on-the-instructions-from-the-website-and-I-watched-avidly, of Baek, my Protoshield is now ASSEMBLED and WORKING!

I have also found the software conflict on my computer preventing the uploading and general use of my Arduino board with the help of the Arduino forums, and have resolved the issue!

I will next test and build the magnet - hall effect sensor aparatus.
~

I have glued the sensors and magnets to the GRIPMASTER, gotten my arduino board up and running with sensors, and prepared a ribbon cable to connect the two (see pictures below). Tomorrow or the next day I will put the wiring on the GRIPMASTER and connect it with the Arduino. I will also have the c code done for the Arduino. In looking online for others using similar devices I have come across Michael Waisvisz at STEIM.
Videos of his hand-controllers may be seen on youtube here and here. I have also downloaded 12 papers on vocal analysis and synthesis to make sure my chuck code will be on track.

Arduino up and running:

GRIP MASTER with 12 magnets, 6 Hall Effect sensors, and ribbon wire:

Ribbon wire breadboard end:

~

I have completed building the thing, and I am now working on the code to get the sensor data through OSC to chuck.
The potential problem that exists at the moment is that the Arduino software does not seem to send OSC data out, and chuck does not seem to take Serial OSC or just Serial data in (to the best of my knowledge (which is not good)). I feel like chuck does do this, and that I am just ignorant of how to do it, so I will continue to investigate. In the mean time I have writtena Max patch that will take the serial information in, and pass it on via OSC. Chuck can definitely hear OSC messages from Max. The max patch will I think have to route the messages into their appropriate OSC directories/routings.

Pictures below:
temp version of max/msp object to rout incoming serial data as OSC messages for chuck.


"finished" grippy thing without a good name yet, the green wires connect the jumper to the signal legs of the
6 Hall Effect sensors, the red wire connects voltage to the first Hall Effect sensor, the Voltage is then passed
(daisy-chained) between all 6 (under the electric tape -- I forgot to photo this), finally the LED at the end
glows green if the ribbon wire is connected properly to the Arduino breadboard, and red if it is not.


Under the red wire is a black wire that completes the circuit to ground. The string holds it all together.


Arduino and protoshield all wired up: red = +5V, black = ground, green = analog out, resistor 330Ω for the LED.


Breadboard with Ribbon cable plugged in.

~

I need to undo the whole thing. I should not have wired the sensor powers serially. Then need to be in parallel. This is my project for today.
It is now done, and it works! I glue-gunned over every solder point to make sure they are protected and insulated. The red wires are +5V, the grey are ground, and the green are the analog sensor readings.


red +5V, grey ground — NOW IN PARALLEL!

green = sensor data!

All plugged in!!! LED too! (green when plugged into board properly, red when backwards):

~

The magnets had to be removed and reattached one magnet per sensor. The numbers I got in from the double magnets were not linear, so exact placement was required. I have done this (see pic below) and am now working on mapping again. I have decided to make two mappings for the 30th (open house): 1) a simple sine wave mapping to each finger pad, where each pad controls the frequency of each oscillator, 2) an adjustment of the code from "I Miss My Uncle Charles" where each of the four finger pads controls the delay time in four reproductions of the code (more on this below in the open house and in the code section). I also thought of a name for the thing:
The GRIP MAESTRO!

With single magnets:

~

Working on polishing my code (see below in code section) and creating my poster (see next entry).

I will include the write-ups from each of my poster sections below. In addition to the mappings discussed below, I am wanting to do a vocal synthesis mapping in the near future (outside of the context of this class of course).

~

Part of the Michael's ongoing research (supported by the Social Sciences and Humanities Research Council of Canada), the “GRIP MAESTRO” is the composer's latest attempt to incorporate real physical resistance and haptic feedback into an electroacoustic performance interface. The goals of this specific project are to take advantage of a simple physical gesture (not previously utilized by an existing acoustic instrument: see figure 1), and, by incorporating physical resistance, create an “easy to learn / difficult to master” interface for the creation of, and control over realtime electroacoustic performance in both a solo and ensemble setting.
The “GRIP MAESTRO” is a modified hand exerciser called a “Grip Master” (see figure 2). By measuring the position of the pads on the device (see figure 3) and carefully mapping them to parameters of music creation (see figure 4), Michael hopes that the “GRIP MAESTRO” will provide the feel, control, and aural feedback necessary to be an effective interface between performers and their music, and between audiences and their performers.
Presently there are two mappings of the control data prepared in the ‘strongly timed’ programming language, “ChucK” (http://chuck.cs.princeton.edu/). Michael is presently in the planning stages of a third, but there are a multitude of potential applications and mappings for which the “GRIP MAESTRO” is ideal (see figure 5).

Figure 1: Movement & Resistance
The action of gripping and squeezing by the hand(s) was chosen early on to be the the central aspect of this interface. Many existing acoustic and electroacoustic instruments rely heavily on the performers hands as driving controlling agents, but few utilize and emphasize the motion of gripping specifically.
In order for an interface to take full advantage of the motion of this activity, it needs to provide physical feedback and/or resistance to the activity consistently for the full range of its motion. It is very important to the composer that any given interface feature a real and innate physical feedback/resistance to the performer’s actions so that he or she may feel as closely in tune with that interface as possible and so that the sounds that are produced by the performative actions relate to the nature of the actions themselves.

Figure 2: “GRIP MASTER”
A ‘hand-exerciser’ seemed the best choice for a physical object that: a)takes advantage of the full range of motion involved with the action of ‘gripping’, and b)provides real physical feedback/resistance to that action.
The ProHands “GRIP MASTER” hand exerciser (available at http://www.prohands.net/) was quickly discovered and decided upon by Michael, who was drawn to several of this specific model’s key features. Among these were: each finger has its own independently but equally resistant pad, the palm has its own moderately rotating resistant pad, the devices are available in a number of different resistances, the small, transparent, yet solid construction makes the device easily upgradable, and its plastic body is naturally electrically insulated.

Figure 3: Sensors & Sensing
In order to translate the position of each of the GRIP MASTER pads to the computer as a number, Michael chose to attach magnets to each pad paired with Magnetic Hall Effect Sensors attached to the central body of the device. A 10-wire ribbon cable is used to connect the necessary Voltage (+5V) and Ground for each of the 6 sensors (each with a wire with which to take readings), and Voltage and Ground for an indicator LED (see circuit diagram and images below). Each sensor and magnet pair needed to be directly aligned (on-axis) to ensure the most linear readings possible. Two sensor/magnet pairs were placed on the palm pad in order to determine the rotation angle of that pad. (The second image below shows the wiring of the “GRIP MAESTRO’s” power before the sensor-reading wires were added, the two magnets per sensor in the image were later replaced by single on-axis magnets.)





Figure 4: Mapping
Utilizing excerpts of Max/MSP code by Daniel Jolliffe, “Arduino2Max,” Michael constructed a Max-patch (Serial2OSC.maxpat) that filters the incoming ‘analog’ readings from the Arduino board, and broadcasts them via the U.C. Berkeley-developed UDP network protocol, Open Sound Control. He then set up two programs in Ge Wang and Perry Cook’s ChucK to receive the messages (since ChucK is presently unable to receive data directly from the Arduino).
The first program (GripMaestro-SinOsc.ck) maps the incoming floating point numbers (0. through 1.) of each of the four finger pads to a different sine oscillator’s frequency, and the palm pad to the sounds’ panning and overall amplitude. This simple example demonstrates the degree of control that the performer has over musical materials with the “GRIP MAESTRO.”
The second program (GripMaestro-Granular.ck) is based off of code by Kyle Spratt from his piece, “I Miss My Uncle Charles.” The original program listens to an incoming audio signal and parses it into small repeating “snippets” depending upon the variables that are given and controlled by the user by editing the ChucK code and replacing the already running “shred.” In this mapping however the parameters of the program are controlled entirely by the “Grip Maestro”: the four finger pads control each of four sets of these original parameters where as the performer depresses a pad the amount of delay time increases, and the palm pad controls the playback speed of the audio signal, the size of the snippets, and the overall amplitude. The use of the “Grip Maestro” allows for much faster and smoother changes of these control parameters, which in turn produces a more personal, potentially emotive performance.

Figure 5: Future & Other
There have been many similar developments in hand-controlled electronic interfaces (of particular interest is S.T.E.I.M.’s Michael Waisvisz’s “Hands”). Many ‘glove’ interfaces lack the physical feedback I have proposed here is necessary to produce a truly personal and emotive connection with the interface and the music it produces, whilst other ‘button-based’ hand controllers may themselves or their controlling-gestures be more complicated than need be for an audience to intuit the performer-interface relationship. It is my hope with the “GRIP MAESTRO” that I have addressed these issues and come to an interesting solution.
The “GRIP MAESTRO’s” simple design allows for strongly coupled performance and sonic result for both the performer and the listener and for many flexible mappings as I hope these examples have demonstrated. The next mapping that I am developing uses the finger pad readings to control the formant information in a vocal synthesizer. Future sensor-additions to the “GRIP MAESTRO” might include a gyroscope or accelerometer to account for the player’s position and/or movement.

~