An Interactive Music Game
Luke Dahl
Music 220C, Spring 2007
Goals
The goal of this project was to explore aspects of music-making and
games, and through this process design an audio-based game or set of simple games.
Some of the questions and themes I hoped to address are:
- What is a game, and how is playing a game different from making music?
- How would a purely audio-based game with no video component work?
- Can the output of such a game be considered music?
- Competition vs. Collaboration.
- Gesture and Movement in the control of real-time musical processed.
As a side goal, I also wanted to get more experience in programming in Max/MSP.
Process
- I did some research on the concept of games in general (see below for some interesting theory), and did a survey of some extant musical games.
- I decided to use the Nintendo Wii Remote as a controller.
It has a 3-axis accelerometer, and a number of buttons and
communicated with the computer wirelessly via bluetooth.
Games
I decided to try to make three small two-player audio games based on three of Callois' game principles:
-
Agon / Competition:
A type of "wack-a-note" game in which users navigate through a
pitch space by gesturing with their controllers. The
game plays a tone at a goal pitch, and the first player to match
that pitch scores.
- Ludus / Complexity and Rules:
A cooperative game in
which the players have to work together to solve a puzzle. There
is a sound whose pitch is some function of the relationship between
each player's gestures. The players' goal is to steer the pitch
to some goal, and as they do so the amount of distortion which is
applied to the sound
decreases.
- Paidia / Improvisation and Play:
In this "game" the idea is to just playfully interact with a
sound making process. There are two rhythmic oscillaters which
sync their tempos to input events from the players. The two
oscillators are also loosely coupled to each other.
Oscillator Details
The two synced oscillators were inspired by Ed Large's models of rhythm
and pitch percpetion which are comprised of a bank of coupled
non-linear oscillators. The rhythmic oscillators I developed are
not an exact translation of Large's continuous time equations, but are
instead designed to have the same functionality.
- Each oscillator has a phase that varies from -0.5 to 0.5.
- Each time step the oscillator updates its phase based on its current phase and period.
- When the oscillator receives a "bump" from a user event it
updates its current phase and period so that it will gradually come
into synchrony with any regular pulse it receives.
- The amount of phase and period adaptation depends on the timing of the user event and the current sensitivity of the oscillator.
- The sensitivity adapts over time. If recent bumps have been
close to the expected period and phase of the oscillator the oscillator
becomes more selective, only responding to accurate input.
However, if over time user events have not beenaccurate the oscillator's sensitivity will become less specific and each bump will result in a larger adjustment.
- The two oscillators are loosely coupled to each other.
Some Last Minute Snags
- I disovered that my computer cannot successfully connect to two
Wii controllers at once, making a two player game impossible. :-(
- I discovered that controlling the rhythmic oscillators by making
"air hit" gestures with the Wiimote only results in a deccelerating
tempo. I suspect this is due to the delay effect that Chris and
others studied.
- When my Max/MSP patch grew complex the oscillators started to grow unstable, often accelerating without end.
Results
Since two-player games are precluded by my current technical situation,
I decided to combine elements of all three games into one single-player
game.
The player can play modal melodies by holding the trigger and moving
the controller. Roll (pronation and suppination of the forearm)
controls pitch, and the '1' and '2' buttons select different scales.
Volume is controlled by the pitch (elevation angle) of the
controller. Occasionally a goal tone will occur and the user can make
it go away by finding the pitch and then hitting the trigger again.
There is a bass tone which can be enabled by pressing the 'A' button.
The bass tone has distortion applied to it. The distortion
can be removed by navigating to the correct controller orientation and
then releasing the 'A' button.
There are two rhythmic oscillators which can be triggered by "air
hitting" either up-down (for the slow oscillator) or right-left (for
the faster oscillator.) Triggering the drum affects the tempo and
also sets a note in an 8-note pattern. After some time the notes
in the pattern will cease and need to be retriggered. Since "air
hits" seem to have too much delay to be useful the 'z' and 'x' keys on
the computer can also be used.
There is also a rhythmic accompaniment which varies over time.
Appendix: Notes from earlier in the quarter
What is a game?
Answers from the class
The idea of the opponent is crucial, whether it be another player or
team, some in-game sprite, or the game world as opponent (must master
the world.)
An arbitrary set of rules packaged in such a way that creates the illusion of meaning.
Something to entertain and excite your mind and body without any real-life risks.
A game is a structured or semi-structured activity pursued for enjoyment (from wikipedia.)
Games are sometimes metaphorical or play situations (playing 'as if.')
Philosophers, Thinkers, Designers
Johan Huizinga in Homo Ludens (or Man the Player),
1944, discusses the importance of the play element of culture and
society, and suggests that play is primary to and a necessary condition
of the generation of culture.
Roger Callois, Man, Play, and Games, 1961:
Play is:
- Free: play is not obligatory, or it would lose its attractive and joyous quality as diversion.
- Separate: circumscribed within limits of space and time, defined and fixed in advance.
- Uncertain: the outcome is unforseeable.
- Non-productive: creating neither goods nor wealth
- Governed by Rules: these rules are different from everyday life
- Make-believe: accompanied by a special awareness of seperate reality.
"Every new game is a totally new beginning, since a game can neither
produce nor found anything. The essence of a game is to cancel out its
results."
http://onlyagame.typepad.com/only_a_game/2006/05/roger_caillois_.html
Chris Crawford:
- Creative Expression -> Art (for its own beauty), or Entertainment (for money.)
- Entertainment, if interactive -> Plaything.
- Plaything -> Toy (no goals), or Challenge (has goals)
- Challenge -> Puzzle (no active opponents), or Conflict (opponent.)
- Conflict -> Competition (can only outperform opponent), or Game (can interfere with opponent's performance.)
Greg Neimeyer:
"the importance of learning by playing is not in encontering new content
by playing, but in testing and exploring new rules through exploration.
Playing a game allows for exploration with indemnity."
Some Extant Music Games
Currently Thinking
- more exploratory than competitive
- possibly cooperative (Neimeyer's "Joint Balance")
- output should be musical, and listenable to as such.
- something that might fit better in a gallery than an arcade.
Thinking about how to make a collaborative game. Considering Neimeyer's Joint Balance as a possible prototype.
Considering the role of difference. Difference is a quality that
exists nowhere except between two other values. One game goal would be
for two players to create or maintain a certain difference between.
Considering how to gradually build skill and complexity. For
example, once two players collaboratively meet goals on one level we
can introduce a new dimension in which they must collaborate.
Should they also maintain the relationship of the lower
dimension, and if they fail they loose the ability to hear the second
dimension, or should they achieve a new level from which they can't
fall back?
Considering Wii-mote as a controller. Ordered one.
Considering Max/MSP as development environment (opportunity to learn it better.)
To Follow Up On
Neimeyer & Chafe, shared vocal control of steering, Organum
Adrian Yennick (sp??), UCSD (Brook)
Wii: Jason, Ben K, etc.
Sile's blutooth controller.
Upcoming show of Neimeyer's students.
Online flash stuff (Brook.)
References
http://en.wikipedia.org/wiki/Game: