In a world where emotions are expressed through complex, dynamic patterns spanning the face and body, Mirrored Canvas bridges facial expressions and artistic self-expression. Mirrored Canvas transforms fleeting facial expressions like smiles or raised eyebrows into dynamic visual art. Inspired by how mirrors show us and canvases capture art, it helps people see their emotions in new ways. Participants watch as their expressions become evolving artwork projected onto a translucent canvas, making the invisible visible.

Why face matters

https://pmc.ncbi.nlm.nih.gov/articles/PMC10173880/

Most studies focus only on a few fixed facial expressions, but real emotions are much more complex. People use their whole bodies, not just faces, to show how they feel and if we don’t take that into account we could fail to capture the nuanced, dynamic, and multimodal nature of genuine human emotion.

“People express emotions in nuanced patterns and crucially not everything is in the face... each emotional display is a complex, multi-component, motoric event.”

“The face is not the whole picture. Other aspects, including body movement, gaze, and physiological changes are crucial in our expression and perception of emotions... bodies and faces are almost never perceived in isolation, but rather as an integrated whole.”

Yet, why do we sometimes fail to understand or express emotions?

Mirrored Canvas at work

This project uses a webcam to track facial expressions in real-time with FaceOSC. The data goes to a program built in Processing that turns your expressions into colorful visuals. A projector then casts this moving art onto a see-through acrylic canvas, letting you watch your emotions come alive.

Each expression triggers a unique visual:

• Eyebrow raise starts making the canvas glitch. The more your raise the crazier it becomes.

• Head tilt generates text “I am a body bender” to show your flexibility.

• Jaw drop displays image from a database filled with images

• Your voice pitch controls a flowing paintbrush. Low pitch paints in orange, high pitch in blue.

• The centre of your head acts as the anchor of the brush.

• You can save the artwork by pressing the ‘S’ key.

Making people laugh was a joy. Their laughter quickly became contagious. It felt great to see people embrace a playful, childlike curiosity with their movements while feeling free to be goofy, not shy about making weird voices, and exploring different pitches. It became clear that creating something absurd brings out the absurdity in those trying it.

Iterating expressions and technical details

We wanted this tool of expression to be DADA, weird, absurd, expressive, and intentional to make people be free and perform embarrassing actions without fear of judgement. So, we started by trying to learn how to make as many weird faces as we could while watching ourselves in the mirror, to see what new things we could discover about ourselves using those 43 facial muscles.

Next, we used FaceOSC for facial tracking. We captured eye, eyebrow, mouth, and head movements, mapping those points to the outputs we wanted. We also used voice in the project because it’s another tool we often use to release or convey emotion.

While working with FaceOSC, especially when mapping the eyes, we faced glitches during testing with different users. We found the glitches happened when individual eye parameters were outside FaceOSC’s standard range. This showed us why datasets need diverse representation. As a result, we removed eye tracking and focused solely on eyebrow movements.

Voices from the Canvas

“W*f is that? OMG it’s so funny.”

“Oh niceeee.... I love it.”

Lessons from the mirror

Mirrored Canvas shows that emotions are never still or simple. Our faces and bodies tell stories that shift and blend in real time. This project helps make those stories visible and fun, opening new ways to explore how we feel and express ourselves.