Reading Response #4

From Chapter 4 of Artful Design, I would like to respond to Principle 4.5, which states: DESIGN THINGS WITH A COMPUTER THAT WOULD NOT BE POSSIBLE WITHOUT. The rapid development of computer technologies gives rise to a large number of applications that were unimaginable previously. Most of them start with emulating what human beings are able to do, but those which then exceed the possibilities of human beings are the ones that strike us deeply and make us feel (Principle 3.12).

Take computer graphics as an example: Physical reality is what people pursued initially, and with great effort at simulating the lighting models and tuning parameters for different materials, they manage to render scenes that are almost hard to differentiate from the reality. After conquering the camera-like effect, people then turned to non-photorealistic shading. To illustrate, toon shading, which belongs to this category, is trying to reproduce hand-drawn texture. It uses mainly color quantization and outlining to achieve flatness in a 3D world. There are also various techniques modeling specific strokes such as pencil sketch and oil painting. These two methods alone are just replicating what already exists without a computer, though at a much larger scale with remarkable efficiency. What really makes computer graphics irreplaceable is the precise combination of physically based rendering (PBR) and toon shading. While PBR brings consistency and is good at featuring the future, toon shading establishes the nostalgic mood and shortens the distance between the graphics and the audience. The ability to interpolate these two effects empowers the producers and directors to control the feeling of the audience more sensitively.

Back to the sound domain, with current technology, it is easy to create a self-playing piano, faking the sense of live performing. Yet, the MirrorFugue project, designed by the MIT Tangible Media Group, develops a unique experience that would not be possible without a computer. They record the playing of users, and then replay it on the piano with the projection of the player's face, body, and fingers. It mixes the feeling that the keys are pressed by the image of fingers and the fact that it is just an illusion of light and mirrors. More impressively, with MirrorFugue, it is possible to play a piano duet with ourselves from the past. The design realizes a conversation between now and past, present and absent, evoking a reflection that is inimitable without the help of a computer.