Fish Eye (2020)

Live streaming interactive music, dance and visuals using videoconferencing tools.

Platforms: TouchDesigner, Jitsi Meet, OBS Studio

Get the code on Gitlab: Fish Eye

Fish Eye is an online work created by choreograph/dancer Audrey Gaussiran, composer/percussionist Joannie Labelle and myself, under the moniker [Ctrl+Alt+Del]. It is a live dance performance in four acts with music being played live using Ableton Live and visuals being manipulated in real-time using TouchDesigner. Fish Eye premiered on May 2nd 2020 as part of the first edition of the Festival de Troi(e)s and won the festival’s Audience Award.

Frame from Fish Eye (Photo: Francis Lecavalier, 2020)

Fish Eye is the result of a 72-hour creative sprint hosted by the Festival de Troi(e)s from April 29th to May 2nd 2020. As part of this sprint, three teams of three contestants had to create, rehearse and perform a brand new audiovisual work leveraging videoconferencing tools and live audience interaction. The results were presented on a live Facebook stream hosted by the festival. The main theme of this first edition was “le bocal” (“the jar” or “the fish bowl”).

Because of the COVID-19 health crisis and the resulting worldwide confinement, the Fish Eye was built, rehearsed and performed entirely online. Jitsi Meet, a free and open-source videoconferencing service, was the main tool used to share audio and visual feeds between the performers. The Facebook live stream was handled by OBS Studio.

Frame from Fish Eye (Photo: Francis Lecavalier, 2020)

My role in this project was two-fold: I was in charge of applying visual effects on Jitsi Meet‘s video streams using TouchDesigner, as well as managing the Facebook stream through OBS Studio. One of my main challenges was finding workarounds to make sure that the visuals and audio were in sync. During the sprint, we tried many different routing scenarios, but given our limited time, we still ended up with some minimal latency (~500ms) between Joannie’s music and Audrey’s moves and body percussions. This was considered acceptable, since we utlimately took the creative decision to discard the audio coming from Audrey’s feed.

As a programmer, this project was also the perfect opportunity to further develop my expertise with TouchDesigner. For Fish Eye, I developed a custom VJ setup mapped to a Launchcontrol XL MIDI controller. This allowed me to manipulate visuals in real-time and easily adapt myself to Audrey and Joannie’s propositions, as well as the audience’s reactions.

Frame from Fish Eye (Photo: Francis Lecavalier, 2020)

It is worth noting that an important part of Fish Eye is the embedded interactivity. During the performance, questions loosely related to the main theme (“the jar”) were asked to the online viewers using text directly superimposed on the the video. The audience was invited to write their answers in the live stream’s comments section. This feedback was closely monitored by Audrey, Joannie and I: we adapted and customized every aspect of the performance (choreography, music and visuals) according to the audience’s answers. This creative choice was directly in line with our trio’s previous experimentations with Ctrl:N. This effectively means that every representation of Fish Eye is different, since it is always shaped in a different way.

Detail of Fish Eye‘s TouchDesigner patch (Photo: Francis Lecavalier, 2020)

Fish Eye is set to be further developed and presented again in the course of 2020 and 2021.