Live streaming interactive music, dance and visuals using videoconferencing tools.
Tools: TouchDesigner, Jitsi Meet, OBS Studio
Get the code on Gitlab: Fisheye
Fisheye is an online work created by choreograph/dancer Audrey Gaussiran, composer/percussionist Joannie Labelle and myself, under the [Ctrl+Alt+Del] moniker. It is a live dance performance in four acts with music being played live using Ableton Live and visuals being manipulated in real-time using TouchDesigner. Fisheye premiered on May 2nd 2020 as part of the first edition of the Festival de Troi(e)s and won the festival’s Audience Award.
Fisheye is the result of a 72-hour creative sprint hosted by the Festival de Troi(e)s from April 29th to May 2nd 2020. As part of this sprint, three teams of three contestants had to create, rehearse and perform a brand new audiovisual work leveraging videoconferencing tools and live audience interaction. The results were presented on a live Facebook stream hosted by the festival. The main theme of this first edition was “le bocal” (“the jar” or “the fish bowl”).
Because of the COVID-19 health crisis and the resulting worldwide confinement, the Fisheye was built, rehearsed and performed entirely online. Jitsi Meet, a free and open-source videoconferencing service, was the main tool used to share audio and visual feeds between the performers. The Facebook live stream was handled by OBS Studio.
My role in this project was two-fold: I was in charge of applying visual effects on Jitsi Meet‘s video streams using TouchDesigner, as well as managing the Facebook stream through OBS Studio. One of my main challenges was finding workarounds to make sure that the visuals and audio were in sync. During the sprint, we tried many different routing scenarios, but given our limited time, we still ended up with some minimal latency (~500ms) between Joannie’s music and Audrey’s moves and body percussions. This was considered acceptable, since we utlimately took the creative decision to discard the audio coming from Audrey’s feed.
As a programmer, this project was also the perfect opportunity to further develop my expertise with TouchDesigner. For Fisheye, I developed a custom VJ setup mapped to a Launchcontrol XL MIDI controller. This allowed me to manipulate visuals in real-time and easily adapt myself to Audrey and Joannie’s propositions, as well as the audience’s reactions.
It is worth noting that an important part of Fisheye is the embedded interactivity. During the performance, questions loosely related to the main theme (“the jar”) were asked to the online viewers using text directly superimposed on the the video. The audience was invited to write their answers in the live stream’s comments section. This feedback was closely monitored by Audrey, Joannie and I: we adapted and customized every aspect of the performance (choreography, music and visuals) according to the audience’s answers. This creative choice was directly in line with our trio’s previous experimentations with Ctrl:N. This effectively means that every representation of Fisheye is different, since it is always shaped in a different way.
