This is a response to Chapter 5 of Artful Design,
Interface Design.
I will be responding to Principle 5.18: Re-Mutalize! Input +
Output + Human.
Artful Design states Re-Mutalization as “a
commitment to designing the interface as a whole – and with the
human as an integral part of the system,” (pg. 244). To
explore this idea, let us use the piano as a case study. Used
earlier in Chapter 5, the piano models an interface: “the
membrane of interaction between human and technology,” (pg.
206). An interface contains an input controller, mapping of
input to output, and output (traditionally, acoustic sound
created by a vibrating string or air speed). The piano’s input
controller is fully accessible to the player with 88 different
ivory keys. The keys are in turn mapped to 88 triplets of
strings of various lengths which are responsible for 88
different frequencies which can be played in any combination to
create single notes, or multitudes or chords. The mapping is
abstracted from the player, creating a seamless delivery of
output from input. However, though the mapping is hidden from
the player, it is predictable; a certain input will create the
same output every time. The predictability of the piano is what
allows a newcomer or advanced player to use the same interface
while playing music of varying difficulty. In this essay, I will
propose a different paradigm of interface design.
As stated by Principle 5.1 from Artful Design,
interaction is a loop. A player starts with an intention
– even if seemingly random keys are played in sequence, the act
of pressing a piano key is intention. The product of intention
is action, followed by the perception of our action which acts
as feedback to inform our next intention. However, what if the
instrument learned from the human. What would this entail? In
accordance with Principle 5.5, I am proposing puting
machine learning into the interaction loop! In this
paradigm, I am hybridizing Perry Cook’s
Principles of Design to use an existing instrument to
inform an old controller which manipulates a new algorithm.
Suppose a digital representation of a piano where the input
controller and output mimics that of a traditional piano, but
the mapping is instead a series of disconnected input and output
nodes. As the player begins to tap keys with varying velocities
and combinations, a machine learning algorithm would connect
input nodes with output nodes to yield a complete instrument
unique to its player’s style, cadence, and emotion.
The Re-Mutalized piano is a thought experiment into
holistic interface design.