Part I
About Visual Perception
In this part, we isolate three discrete stages of the human perceptual process: Sensing, Interpreting, and Reacting. Each stage is modelled in a pair of simple computing devices that communicate solely via the visual sense.
The installation consists of three stations. At each station an audience member views a simple conversation that illustrates one of the isolated stages, and can interact with the conversation by disrupting the light between the devices.
1. Sensing
The first pair of devices focuses on Sensing. Each device acts as a mirror, attempting to output on its display exactly what it has received as input to its camera.
Due to a variety of factors — distinct locations of the devices, visual noise in the environment, small aberrations in hardware, the time delay between input/output, etc. — the reflection is less than perfect, creating an evolving feedback loop. Visitors can experiment with obstructing the space between the devices to alter the image.
Our eyes focus on a specific area in our field of vision, and convert the incoming light to signals which can be processed by the brain. In the Sensing devices we've designed, the image signal from the camera is focused, warped, and thresholded in order to prepare it for the subsequent stage of perception.
Audience members can reveal these processes by pressing a button.
2. Interpreting
The second pair focuses on Interpreting. When we use our senses, how does our mind derive meaning from the information it receives? Through an accumulation of experiences and memories, we are able to recognize patterns of light and associate them with objects and concepts.
In this stage, the device on the right stores a limited memory of visual patterns. On the left, an audience member can create a simple drawing, and the right-hand device will traverse its memory in search of the pattern that most closely resembles that which it sees.
Though our biological algorithms work in a different manner, this matching of patterns is representative of the way our minds interpret the light we see.
The process of pattern matching is revealed by pressing a button.
3. Reacting
The third stage is about Reacting. Once our minds have gathered meaning from the input delivered by the senses, how do we decide what response to make? In this pair, the two devices play a game of pong, one acting as the ball, and the other the paddle. Each reacts to the other's motion.
This pair is inspired by two strategies we use to react to what we see: logic and prediction. We use logic to make decisions based on what we sense. For example, while driving we stop our car if we see the car ahead of us stop. Prediction is used to anticipate situations in the future. To do this, we analyze the behavior of our environment and we compare it with similar behavioral patterns we have experienced in the past in order to make informed, well-timed decisions.
Pressing the button reveals what's happening behind the scenes.
Installation in The Story of Light festival, Goa (India)
The Story of Light Festival, held in January 2015 in Panjim, Goa (India), brought together 65 artists, scientists, philosophers and teachers from around the world to celebrate light through interactive installations, workshops, and performances. It provided a great venue for audiences of all ages to explore visual perception through the Talking exhibit.