WINDOW 窗
An Interactive Multimedia Installation on Quarantine Memory and Dream
Lead Artist: Anna Borou Yu 3D Artist: Jiajian Min Tech Artist: Amo Zeng Year: 2022
2022 Chinese CHI Art Gallery Third Prize, 2022 Chinagraph Electronic Theater Best Visual Effect Third Prize,
2022 Lumen Prize BCS Immersive Environment Award Longlist, 2022 ACM Siggraph Asia Art Gallery, Shenzhen MoCAUP Light
Window is an interactive multimedia installation that reconstructs the Quarantine environment of the artist Borou Yu from the US to China, introducing the audience into the artist’s memory and dreams. While the room was the container of the physical body, the window became the metaphor of hope, spirit and imagination to the outer world. The use of point clouds, sound, texts and the aesthetics of transparency and vanishing delineate the space, which is the representation of the mental projection. With Kinect depth camera, projector and speakers, the project developed an interaction system where the audience would immerse into the soundscape and ocean of texts from the monologue written during the quarantine periods. Window aims to create a poetic digital experience, arouse the audience's empathetic feeling together with the artist during the pandemic, and reflect on the relationship between body and space, the individual and the universe, the physical and the spiritual.
作品荣获2022世界人机交互大会数字艺术作品三等奖，2022中国计算机图形学大会电子剧场最佳视觉效果三等奖，入围2022英国流明科技与艺术大奖，展出于2022 ACM Siggraph Asia数字艺术展，深圳城市光美术馆。
“窗”是一个多媒体交互装置，从空间、声音、日记等多个维度构建出艺术家于博柔从海外回国的旅行隔离经历，将观众带入记忆与梦境之中。在足不出户的日子里，房间是身体的容器，而窗户是与外界沟通的唯一渠道，隐喻着希望、想象与现实。不断变化的空间点云、窗外鸟鸣以及日记文字形成虚幻缥缈的美学体验，将日常空间重构，也是记忆投射的再现拟像。梦境中的房间不断融合，犹如巨大的记忆迷宫。窗户逐渐变成通向精神世界的隧道，仿佛万神庙穹顶的洞口、巴黎圣母院的花窗。通过Kinect捕捉设备，投影仪和立体音响，装置作品创造出一个沉浸式交互空间，观众被淹没在现实与虚幻的海洋中。这是宏大历史中，一份独特却又具有普遍性的个体视角。作品“窗”希望可以创造出一种诗意、有温度和情感的数字体验，唤起观众在疫情期间的共情，建立人与人之间的心灵沟通，同时引发对于身体与空间、个人与世界、物质与精神的深层思考。
There are three different media inputs, the space, the text and the sound. The space of four different rooms in Boston, Tianjin and Changchun were 3D scanned by Iphone Polycam software and exported to Unity as point clouds. The texts are from the diary written by Borou during her quarantine time. And the sound includes recording of exterior sounds from the windows, the monologue, as well as sound effects generated from point cloud images by Melobytes. The overall interactive experience is created in Unity3D. The Morphing Effects between rooms are achieved by Unity Asset FM POINTS. And the interactive contour is implemented by blob detection technique with Kinect. For onsite installation, a 4K short throw projector, kinect 2, Laptop and two speakers will be provided to create a multimedia interactive environment. The projection screen is another window that links the audience with the artist.
ACT I: Window as a Dream
In the beginning of the journey, the audience are placed into the space of floating rooms. The point cloud rooms are flying towards them, facing the windows. They are the boundaries between safety and chaos, and the settler needs to locate themselves within. Viewers could wander through the clusters and when they are closer to the window, the soundscape will be activated and they could hear the bird twittering, the children’s screaming recorded, simulating the adjacency to the outside world.
Then the memory starts to blur and the rooms morph into each other, reflecting a distortion of time and space. After that, the single room merged into a matrix of rooms floating and morphing at the same time. This is the imagination of a nonexisting place, breaking from gravitational spatial logic, where the window becomes the Portal to another ethereal world, like the Oculus of Pantheon or the Rose Window of the Notre Dame Cathedral. The immeasurable scale and overwhelming beauty lead to a feeling of sublime, and the industrial idea of an accessible room transforms into a spiritual concept of intangible rooms.
ACT II: Window as a Monologue
In ACT II, the audience will get back to the individual point cloud room, with the window as the main focus. The x, y, z coordinates of each point on the window were extracted and replaced with a random Chinese character at the same coordinate. Here we use Kinect to track body contour \cite{Filkov} in real-time. As shown in Figure 4, when the participant gets closer, their body will be reflected in the window, with randomly selected, meaningless Chinese characters generated outside the contour, and the revealed messages represent the artist's feelings when she looks at the window inside the contour. The whisper of the artist could also be heard, turning the monologue into a dialogue. After a while, the window shapes destruct and the characters will rush over toward the audience, revealing the hidden overwhelming emotion longing to get delivered.