Leaf Hands in a Musical Environment




Figure 4. In movement 2 of 12 Sentiments, the user affects the music by collecting sunlight with their leaves (hands).

The user's actions create music based on interactions with the environment. The user chooses the way the music unfolds based on which hand they use, and for how long.

This is an example of audiovisual alignment through multimodal design: sound, visuals, and haptic vibration feedback (not shown in the video) all rise and fall in one unified swell.

This design makes space for being in interaction. Its longer time scale enables users to be intentional and reflect while they are taking purposeful interaction in the world. This is one way to achieve a synthesis of doing and being.

(See the main page for the journal article.)