The Silent Carnival
Interactive Architecture Projection Mapping Show
Genius Loci Weimar Festival 2025, Germany
This seven-minute projection mapping transforms the building façade into a dramatic stage where Goethe’s Faust comes alive. Rather than abstract visuals, it follows a clear narrative—first animating Faust and Gretchen’s tragic encounter across windows and columns, then turning inward to highlight the very site of Goethe’s pivotal decision. Rendered in a bold red-and-black woodcut style, the piece echoes the printmaking of Goethe’s era while heightening Faust’s passion and moral darkness, ensuring striking clarity on the architecture. At the end of the story, a voting QR code invites the audience to scan and enter a web application to choose the ending. The majority decision will determine how the projection concludes in real time.
Credit
Creative Direction, Art Direction, Storyboard, Music Direction: Lois He
Web App Development, Media Server Integration, TouchDesigner and Unreal Engine VFX: Vio Zhu
AI Video Generation: Lois He, Vio Zhu
Technical Details
To enable real-time interactivity, I developed a cloud-hosted web application where each audience member can scan a QR code to vote for the ending of the projection. When the QR code is displayed at the conclusion of the narrative, the system allows 30 seconds for audience input. After this period, a Python plugin I programmed for the MXWendler media server sends a request to the web application’s server side to retrieve the aggregated votes. The script then clears the results for the current round and triggers the corresponding ending sequence in MXWendler based on the majority choice.
Since MXWendler was the organizer’s chosen media server, I self-learned how to work with it in a short timeframe and extended its functionality by writing a custom Python plugin, enabling the projection to achieve its interactive, audience-driven nature.
Animation Production
The animation was developed through a hybrid AI–human workflow. We began by using MidJourney and ChatGPT-based image generation to create concept art and storyboards, establishing the overall style and narrative direction. From there, we generated numerous start and end frames to lock in the aesthetic language and guide the animation approach. Individual animation segments were then produced with various video-generation AI tools.
To refine and integrate these outputs, we used TouchDesigner and Unreal Engine to add post-production visual effects, enhancing the AI-generated material with dynamic motion and atmosphere. Finally, Adobe Premiere was used to edit, smooth transitions, and shape the animation into a spatial composition—aligning the visual flow with the architecture’s form and creating a seamless dialogue between projection and building.
Voting Results Summary
We implemented a detailed logging system in our web application to track user behavior and participation across three festival days. Over three festival days, the projection mapping was shown 44 times with 371 audience participants. Nearly half voted to save Gretchen, while others chose to side with the devil or leave the ending to fate. About 10% of voters required German-language support, with the rest comfortable in English. Participation rose significantly over the weekend, with total votes on days two and three increasing by 30% compared to day one.
