Audio Mosaicing: Material Signatures and the Interactive Surface

Abstract

This paper explores the acoustic potential of everyday materials through an interactive installation that converts physical gestures into sonic expressions. Using a wooden table as a resonant body, we examine the internal vibrational behaviors of materials when subjected to human touch. The installation was showcased at Music Tech Sundays to demonstrate the concept of Material Signatures through a real-time audio mosaicing system.

Introduction

While we interact with our environment constantly, the acoustic responses of these interactions remain largely unnoticed. Every surface, every object, holds within it a latent sonic identity—an internal world of vibration shaped by its materiality and history. This project aims to make those qualities perceptible, using sound as a lens to study, transform, and express material behaviour.

Methodology

The system is structured around four key components, each forming a critical layer of the sonic interaction:

1. The Human Performer

Participants engage with the table using five carefully chosen hand gestures:

• Tap: A light, focused strike.

• Slap: A broad contact surface.

• Bang: A sudden, high-energy impact.

• Knock: Rhythmic, percussive contact.

• Press: Sustained pressure without immediate release.

Each of these gestures evokes distinct vibrational responses, both within the wood and into the surrounding space.

2. The Physical Object

The central material—wood—is not a passive recipient of force. Its cellular composition and structural grain patterns cause internal reflections, absorptions, and sonic diffusions. The chemical makeup—around 50% carbon and 42% oxygen, plus hydrogen, nitrogen, and minerals like calcium and iron—affects how sound travels within it.

We selected a wooden table for its accessible tactility and complex acoustic behaviour. It serves both as surface and instrument.

3. The Activator

At the core of the system is a semi-sensorial interface. Embedded sensors detect physical interactions and pass data into Atom, our custom signal-processing and synthesis engine.

Atom interprets both gesture data and material feedback to shape audio mosaics—real-time sonic renderings based on the interaction's physical energy and its passage through the material.

4. The Acoustic Space

Sound never exists in isolation. The surrounding space influences each auditory event, shaping how it reverberates, disperses, and fades. This layer encodes additional spatial cues about the object's location, the listener's orientation, and the shared physical setting.

Findings

Through extended observation and real-time experimentation, we discovered:

• Each gesture triggers unique resonant behaviors, dependent on force, location, and duration.
• Vibrations within the material travel faster and more subtly than air-borne sound, often creating hybrid effects when both are combined.
• The sonic outcome is not a direct representation of gesture but an emergent behavior—less a mimicry and more a sonification of material memory.

These insights suggest that materials, when paired with responsive systems, can become not only sensors but also storytellers.

Applications & Implications

The framework developed here can be extended into:

• Performative sound art and live installation contexts.
• Museum exhibits where audiences engage with historical materials sonically.
• Material science education through acoustic exploration.
• Expanded HCI interfaces where materials become musical or informative agents.

Conclusion

This installation demonstrates how materials can act as expressive instruments when sonically activated. By mapping tactile gesture to internal vibration and acoustic output, we open new paths for human–material interaction, performative sound design, and acoustic storytelling.

The result is an instrument that is always becoming—a surface whose response changes depending on how we approach, press, or listen.

Keywords

Sound Interaction, Acoustic Materiality, Sonic Gesture, Interactive Installation, Material Signature, Sensorial Computing