ProjectsInstallations

Passerby

Motion becomes music for visitors at a museum in Mumbai.

We created an interactive installation for ‘Mumbai Art by Artist’ at the Chhatrapati Shivaji Maharaj Vastu Sangrahalaya using Motomi, which turns the movement of passers-by into MIDI and sound.

How it works

A single camera watches the space at 720 × 480 pixels. Motomi compares each frame with the one before it, finds where something has moved and how fast, and turns that into MIDI: where you are becomes which sound, how quickly you move becomes how loud or how bright. Nobody needs to touch anything or learn anything. Walking past is enough, which is why the piece is called Passerby.

In a museum this changes how people behave. Visitors who usually move quietly between objects start to slow down, double back and wave their arms to find out what the room is doing. The sound becomes a mirror of movement in a building otherwise built for stillness.

Context

Turning the body into an instrument with a camera has a lineage in media art. Myron Krueger’s Videoplace, begun in the 1970s, let people interact with computer graphics through their live silhouettes. David Rokeby’s Very Nervous System (1986–1990) used video cameras and a computer to translate the movement of a body directly into music, and Rokeby wrote about such systems as ‘transforming mirrors’. Passerby works in that tradition with an off-the-shelf camera and our own software.

References

  1. Krueger, M. W. (1991). Artificial Reality II. Addison-Wesley. (On Videoplace.)
  2. Rokeby, D. Very Nervous System (1986–1990), interactive sound installation. Link
  3. Rokeby, D. (1995). Transforming mirrors: subjectivity and control in interactive media. In S. Penny (ed.), Critical Issues in Electronic Media. SUNY Press.

Have a space, a sound or an idea?

We design custom immersive experiences and tools for artists and businesses, and we’re always looking for spaces to listen to.

contact@sound.codes