Something May Answer

Interactive Projection Installation

Something May Answer is a family-oriented, sound-triggered interactive installation created for a garden bed light trail. Visitors speak into microphones to awaken and summon imagined creatures from the surrounding forest. Projected onto translucent screens nestled among the trees, these creatures appear to emerge from the landscape itself, transforming a simple interaction into a shared moment of discovery, wonder, and magic for children and families.

Credits

  • Creative & Art Direction, Installation Design, Sound & Animation Production: Lois He

  • TouchDesigner and customized software interactive system: Vio Zhu

  • Fabrication Production Partner: Enddle Zheng, Long, Ivor Xiaorui Zeng, Rita Qiu

  • Projection Mapping Consultant Professor Motomichi Nakamura

  • Documentation: Lois He

  • Special thanks to Olbrich Botanical Gardens staff Melissa Jeanne, Mark Penisten and David Antonacci. Judith & Howard for hosting us.

Tools & Materials

  • Creative:

    • Adobe Photoshop, Adobe After Effects, Adobe Premiere Pro, Figma

    • Generative AI tools, AI-assisted animation tools

  • Technical: TouchDesigner, C++ based customized software, AI-assisted development tools

  • Fabrication: Fusion 360, Plywood, PVC, 3D printing, textile fabrication, ShowTex transparent outdoor projection screen

Interaction
Description

Each microphone independently controls a real-time branching animation system. Rather than simply triggering and replaying a single video, the system continuously tracks how long a visitor is speaking and changes the creature’s behavior while the interaction is happening.

As visitors speak, real-time particles immediately respond to their voice. Short interactions may cause a creature to briefly emerge and retreat, while sustained sound moves the system through progressively different animation states—allowing the creature to pause, approach, become more active, or transition into a longer behavior sequence.

1. Idle State:

In silence, the scrims display a tranquil world of magical plants and softly glowing vegetation. Gentle movements, drifting particles, and subtle light create the feeling of a hidden enchanted landscape waiting to awaken.

2. Sound Reactive State:

The moment a visitor begins speaking, a real-time particle system responds continuously to the microphone input. The particles gather and intensify as the voice continues, providing immediate feedback that the system is listening and building toward the next interaction state.

3. Branching Creature Appearing State:

Once the sound threshold is reached, the creature enters a branching sequence determined by the duration of the visitor’s voice. A short interaction may cause the creature to appear briefly and retreat. With longer interaction, it can remain in an idle state before leaving. If the visitor continues speaking, the system branches again into one of several more active animation sequences before the creature eventually disappears.

These decisions are made in real time while the visitor is still interacting—the system does not simply trigger the same pre-recorded video each time.

Creative Process

The visual language of Something May Answer began with the garden itself: its vegetation, natural colors, nighttime atmosphere, and the idea of discovering something slightly fantastical within an otherwise recognizable environment. We designed the magical creatures by combining features drawn from real animals, using sketching, Photoshop compositing, and an iterative AI-assisted workflow to explore forms that could remain believable and grounded in nature while still feeling unfamiliar and imaginative.

Throughout development, we experimented with many different animal references: including cheetahs, owls, butterflies, cicadas, dragonflies, deer, and rabbits. Testing different combinations, silhouettes, proportions, textures, and color palettes. Through repeated iteration, compositing, refinement, and animation tests, we narrowed these explorations into the final creatures, balancing realism with a subtle sense of fantasy that felt at home within the garden.

This visual development was part of a broader site-specific design process spanning the projection surfaces, interaction behavior, sound, animation, physical installation, and real-time responsive system.

Real-time Interactive
Video Playback
System

We created six interconnected animation states for each creature and built a custom playback system in TouchDesigner.

The system continuously monitors the duration of each microphone input and determines which animation state should play next. Because the visitor can stop speaking at different moments, the final sequence is not predetermined: each interaction can take a different path through the available states.

The animation clips were specifically designed with matching transition frames, allowing the system to branch seamlessly between states without visible cuts. This creates the appearance of one continuous responsive animation, even though the underlying playback is being assembled dynamically in real time.

Fabrication
Process

We designed the physical structures in-house using Fusion 360 and worked closely with our fabrication partner to refine the system for transport, assembly, and on-site maintenance.

A four-person fabrication team produced the three modular frames, ShowTex projection surfaces, and custom 3D-printed microphone units, integrating the LED lighting, microphones, and internal electronics over a three-week fabrication period.