The Silent Carnival
Interactive Architecture Projection Mapping Show
Genius Loci Weimar Festival 2025, Germany
Trailer
The Silent Carnival is a seven-minute interactive architectural projection that intertwines the fictional world of Goethe’s Faust with a moral dilemma from Goethe’s own life. Both narratives converge around the fate of a young woman accused of infanticide: in Goethe’s real life, while serving in Weimar, he supported a decision that allowed the death sentence of a young woman convicted of killing her newborn child to stand; years later, in Faust, he imagined a different fate for Gretchen, who is ultimately declared “saved” by a voice from heaven.
The projection brings these two parallel histories together across the architecture, transforming the façade into a theatrical stage where reality and fiction, history and imagination, judgment and redemption collide. Rendered in a bold red-and-black woodcut-inspired visual language, the piece references the printmaking aesthetics of Goethe’s era while creating strong graphic contrast across the building.
At the climax, the same moral question is passed from Goethe to the contemporary audience. Viewers scan a QR code to enter a web application and vote on the young woman’s fate. The majority vote determines the ending shown on the building in real time, allowing each screening to conclude differently and transforming the audience from spectators into active participants in the story.
Credit
Creative Direction, Art Direction, Storyboard, Music Direction: Lois He
Web App Development, Media Server Integration, TouchDesigner and Unreal Engine VFX: Vio Zhu
AI Video Generation: Lois He, Vio Zhu
Technical Details
To enable real-time interactivity, I developed a cloud-hosted web application where each audience member can scan a QR code to vote for the ending of the projection. When the QR code is displayed at the conclusion of the narrative, the system allows 30 seconds for audience input. After this period, a Python plugin I programmed for the MXWendler media server sends a request to the web application’s server side to retrieve the aggregated votes. The script then clears the results for the current round and triggers the corresponding ending sequence in MXWendler based on the majority choice.
Since MXWendler was the organizer’s chosen media server, I self-learned how to work with it in a short timeframe and extended its functionality by writing a custom Python plugin, enabling the projection to achieve its interactive, audience-driven nature.
Animation Production
The animation was developed through a hybrid AI–human workflow. We began by using MidJourney and ChatGPT-based image generation to create concept art and storyboards, establishing the overall style and narrative direction. From there, we generated numerous start and end frames to lock in the aesthetic language and guide the animation approach. Individual animation segments were then produced with various video-generation AI tools.
To refine and integrate these outputs, we used TouchDesigner and Unreal Engine to add post-production visual effects, enhancing the AI-generated material with dynamic motion and atmosphere. Finally, Adobe Premiere was used to edit, smooth transitions, and shape the animation into a spatial composition—aligning the visual flow with the architecture’s form and creating a seamless dialogue between projection and building.
Voting Results Summary
We implemented a detailed logging system in our web application to track user behavior and participation across three festival days. Over three festival days, the projection mapping was shown 44 times with 371 audience participants. Nearly half voted to save Gretchen, while others chose to side with the devil or leave the ending to fate. About 10% of voters required German-language support, with the rest comfortable in English. Participation rose significantly over the weekend, with total votes on days two and three increasing by 30% compared to day one.
