Open menu Close menu
August 4, 2024
News List

From Screen to Reality: Digital Art Inspired by Video Games

MUTEK24 20230827 Priori Shoeg Frédérique Ménard Aubin 7182 min

From chiptune (8-bit) music to integrated radios such as FlyLo FM in GTA V, to the Hotline Miami soundtrack, considered one of the best in the industry, electronic music has long been an integral part of the world of video games. Nowadays, this aesthetic extends beyond the soundtrack. Audiovisual performances of electronic music often use graphic elements taken from video games, whether through projections of game sequences, 3D animations reminiscent of virtual environments, or interactive digital avatars.

While many of the performances at the 25th edition of MUTEK Montréal are closely linked to the world of video games, some artists go even further, transforming the stage into a veritable virtual world. These performances don't just imitate the world of video games, they become an extension of it, blurring the boundaries between spectacle and game, and plunging the audience into an audiovisual adventure where every note of music and every projected pixel contributes to total immersion.

Murcof & Simon Geilfus - Twin Color

Looking out over Skyrim from the Throat of the World, you might feel like The Wanderer contemplating a sea of fog, as painted by Caspar David Friedrich. Romantic, surrealist and cubist painters have greatly influenced contemporary aesthetics, and video games are no exception.

This is precisely what Murcof and Simon Geilfus have undertaken with Twin Color: they insert the codes of video games into a contemplative world, exploring a virtual universe imbued with the ghosts of Friedrich and Beksinski. The work combines elements of minimalist electronic music with computer-generated visuals, creating an experience reminiscent of modern video game environments. The visuals projected by Simon Geilfus are distinctive for their geometric precision and dynamism, synchronized with Murcof's music to create a harmonious audiovisual landscape. This world is composed mainly of nature-inspired biomes, such as forests and fields, intrinsically linked and articulated with the music.

Murcof & Simon Geilfus - Twin Color


Twin Color incorporates a game engine, a set of software components used by video games for their geometry and physics calculations, to simulate in real time the suppleness and fluidity of the imaginary world in which the action takes place. Using the open-source 3D image-generation software Unreal Engine, Murcof and Geilfus create a dynamic, responsive world where all kinds of phenomena occur. This multimedia approach, where audio and visuals come together to create a unified sensory experience, is very much present in the aesthetics of video games, where interaction and immersion are key elements.

Twin Color offers viewers the chance to explore a virtual world, where the same environment generates both music and video. Reflection on sound spatialization enables the sonic discourse to match the 3D evolutions of the images generated, further enriching the experience.

A performance to be experienced during A/Visions 2, on Saturday August 24 at Théâtre Maisonneuve, Place des Arts.

Information and ticketing

Meuko Meuko & NONEYE - 無形將軍 Invisible General

In the ever-changing landscape of digital art and electronic music, creative boundaries are continually being redefined. 無形將軍 Invisible General, the second chapter in the Ghost Island trilogy by Meuko! Meuko! and NONEYE, is one example of this convergence. This project explores the interplay between traditional Taiwanese beliefs and contemporary dataism, through the prism of an audio-visual performance that draws heavily on the aesthetics of video games.


Invisible General is a follow-up to Meuko!'s 2018 album. Meuko! ⻤島 Ghost Island, released on Danse Noire, a blend of mythology, acoustic instruments, electronic beats and cacophonous samples, bringing together traditional sounds with contemporary electronics and the harsh noises of modern life. Described by the artists as "anti-utopian", the performance presents sprawling geographies and monuments that evoke a sense of desolation. This visual aesthetic is complemented by beats tinged with industrialization, creating a soundscape that drives the narrative forward. The urban decay and flourishing greenery depicted in the visuals underline the contrast between tradition and modernity, a central theme of the project.

This fusion of old and new is not only thematic, but also technical. The creators use motion capture, face capture and real-time 3D engines to bring their vision to life. The use of these engines enables them to dynamically manipulate the digital environment, creating a living, breathing world that fulfils their artistic aspirations. Invisible General is more than a performance; it's a multidisciplinary production that aims to create an alternative narrative for a post-pandemic, alienated world. By investigating themes of mythology, tech-animalism and video game culture, the project offers a commentary on the role of technology in our lives, and the ways in which it can both connect and isolate us.

By inviting NONEYE, Meuko! Meuko! offers an otherworldly experience to its audience through the creation of futuristic play spaces. Fusing computer graphics and the notion of interactive play, 無形將軍 Invisible General is a performance to be experienced during Nocturne 5, August 24 at the Society for Arts and Technology [SAT].

Information and ticketing

Kode9 - Escapology

Walking through the American wastelands depicted by Hideo Kojima in Death Stranding is an invitation to discover those spaces where human colonies have not settled.

Some performances are directly inspired by video games, such as the Escapology project by Steve Goodman, otherwise known as Kode9, musician, teacher, contemporary artist and founder of the Hyperdub label. This fiction recounts an imaginary Scottish space colonization project inspired by the Kingdom of Scotland's failed attempt to colonize part of Panama in the 1690s, known as the Darien Scheme. As a result, the country fell into bankruptcy, and the catastrophe quickly led to the formation of the present-day United Kingdom. Colonization continued after the Acts of Union of 1707, as Scotland and England took control of half the planet.

photo: Oscar Giralt


Escapology rewrites history in an novel way, using fiction to reinterpret and replay Darien's story, so that it ends inversely with a constructive political paradigm shift. As Kode9 explains, "the Astro-Darien universe is programmed by a Panamanian game designer. She codes it to be specifically 'xenophilic', to ally with foreigners and aliens, to pay its historical debts."

Conceived as a soundtrack for the video game Astro-Darien, the album Escapology, released in 2022, is filled with electronic exclamations, snippets of vocal interpretation and some of the Scottish artist's most deft and intuitive club music. Kode9's return, 7 years after his last album, traps the listener in a video-game-like environment, where touches of footwork and sci-fi jungle are inserted into an unusual melodic figure.

Author of Sonic Warfare: Sound, Affect, and the Ecology of Fear, Kode9 deftly manipulates the affective power of sound, scattering each signal with care despite a body of work marked by dislocation and confusion. The rich timbres, layered textures and slippery pitches of the microtonal electronic phrases respond perfectly to the luminous graphics by long-time Kode9 collaborator Lawrence Lek (as well as the show's visuals by Optigram, Bianca Hic and Kode9 himself). Lek deftly uses simulation and game engines to extract real-world locations and transform them into surreal digital spaces. The project is as visually vivid as it is ambitious as an audio album. For Goodman, "designing a world sonically is much more open-ended than an already visualized world."

Escapology is part of a whole, rather than an album in its own right. It is an essential component of Goodman's most recent work that raises more questions than it answers, but does so in a unique way that few other artists could manage. Kode9's live performance at MUTEK Montréal reconfigures Astro-Darien's atmospheres into pieces of asymmetrical club rhythms in high definition, through millimetric sound design.

To be discovered during Nocturne 6, Sunday, August 25 at the Society for Arts and Technology [SAT].

Information and ticketing

Priori & Jek - This But More

Following in the footsteps of the aforementioned projects where musical artists have teamed up with visual talents, and after his 2023 collaboration with Shoeg, Montreal producer and NAFF label co-founder Priori this year ties up with media artist Jack Anderson aka Jek. More than ever, Priori harnesses the limitless power of partnership on his latest album This but More, a rich collection of 13 ambient and trance tracks that Latreille calls "the biggest collaborative project I've undertaken to date".

On Moonstone, Priori collaborates with Brooklyn ambient producer Ben Boldy to create a liminal experience: reverb-laden vocals come and go over the song's up-tempo trance beat. Basalt Tones is another obvious highlight, and sees Priori working alongside fellow Montrealer Jesse Osborne-Lanthier (half of Orchestroll) in a composition that combines hand drums with industrial noise to produce results both archaic and dystopian.

Wake - Priori ft James | Video: Jek


As an album, This but More finds its strength in thematic cohesion. Although the tracks vary from light and ethereal to dark and menacing, the sounds blend together despite their sonic distinctions to open a portal to a fantastical world. The thematic exploration of healing and the creative process adds a deep layer to the album's narrative. It seems that Priori doesn't just create music, but invites listeners on a reflection, where the complexities of life and art intertwine.

An avid reader, film and video game enthusiast, Priori's music is nourished by his passions, from which flow textured music with thoughtful scenarios and global atmospheres. The collaboration with Jek was an obvious choice, as the New Yorker's work is marked by an organic aesthetic, in which Nature plays an important role. An outstanding technician, Jek uses the Unreal Engine game engine to create rich, lush environments that perfectly illustrate the album's ethereal depth.

The track Wake, Priori's first vocal production with singer-producer James K, is accompanied by a visual clip in which natural landscapes and ruins of imaginary civilizations are superimposed on electro-jungle rhythms and pop vocals. With such aesthetic coherence, there's little doubt that the duo's performance at MUTEK Montreal this summer will be remarkable.

Experience the world premiere of this audiovisual show at Métropolis 2, Saturday August 24 at MTELUS.

Information and tickets

Ubisoft

If any video game studio epitomizes the industry's relentless quest to push creative boundaries, it's Ubisoft, and in particular its Montréal studio. Ubisoft Montréal has not only redefined the video game genre with a bold vision and a willingness to collaborate, but has also broken new ground in development and production.

Following the resounding success of the Prince of Persia license, Ubisoft embarked on an ambitious project that would transform the industry: Assassin's Creed. Released in 2007, this game laid the foundations for a franchise that would become iconic, thanks to its unique blend of story, parkour and combat... Led by Ubisoft in Montréal, Assassin's Creed II was subsequently released in 2009. This project benefited from the expertise of the Singapore and Annecy studios, producing a game that would redefine the action-adventure genre with an unprecedented international synergy.

Developed mainly in Montréal, Assassin's Creed II was a feat in many respects. By integrating high-definition (HD) graphics at a time when video games were aspiring to greater realism, Ubisoft was able to captivate the general public with breathtaking visuals and unprecedented immersion. The integration of high-quality cinematics from a dedicated department added a cinematic dimension that brought the game closer to a true interactive film. Assassin's Creed II didn't just impress with its aesthetics. Inspired by books on secret societies and DNA technology, the game explored complex historical themes. This narrative depth, coupled with open-world gameplay - still an emerging concept at the time - allowed players to explore vast, detailed environments, pushing back the limits of interactivity.

This same quest for innovation and collaboration can be found in another of the studio's flagship projects: Rainbow Six. In 2015, Ubisoft in Montréal took the next step by producing the game's soundtrack with the Orchestre symphonique de Montréal (OSM). This bold collaboration fused the richness of classical instruments with the intensity of tactical action, creating a unique and immersive sound experience. By integrating elements of electronic music, the studio has modernized and energized the soundtrack, while preserving its orchestral depth. The use, for the first time in video game history, of the octobass in 2022, a rare and imposing instrument, added extra power to the compositions, illustrating Ubisoft's desire to push artistic boundaries.

At the heart of the digital age, video games open up an array of possibilities, where collaborations between different industries are favored, cultural transmission takes place and accessibility eliminates barriers to create new opportunities.

Thursday, August 22nd marks the third day of the Forum, titled The Human Connection: New Forms of Storytelling in Digital Art and Gaming, which will heavily feature gaming—a first-time addition to the MUTEK Forum's programming.

Workshop | Gaming and Music Workshop, Presented by Ubisoft

Credit photo: Antoine Salto

The quality of video games often lies in their ability to create immersive experiences through their visuals, soundtracks and sound effects. These elements intertwine to bring each video game to life. Research has even shown that the repetitive soundtracks of video games can affect dopamine levels in the brain, having a real effect on gamers.

Putting the spotlight on sound in video games encourages particular attention to the collaborations that can emerge between different industries. In Montréal, a notable collaboration between Ubisoft and the Orchestre symphonique de Montréal led to the creation of the soundtrack for the game Rainbow Six. Vincent Gagnon, audio director at Ubisoft, chose his instrument of choice: the octobasse, the largest and lowest of the string instruments. With only five in the world, the OSM is privileged to have one on its premises.

"The octobasse was perfect for the game, with its sci-fi feel. It sets a strange tone, because it's not an instrument you often hear." -Vincent Gagnon, Audio Director, Ubisoft

Octobasse OSM Credit Antoine Salto

This massive three string instrument produces exceptional sonic vibrations. Its notes are so low that the octobass can operate below the audible frequency range of the human ear: 16,351 Hz. Its rich deep timbre and its rarity make it a unique sound rarely encountered.This concert is a perfect match for the alienating universe of Rainbow Six.

On Friday, August 23 at 10am, a conference-workshop presented by Vincent Gagnon and his team will delve in more detail into the local composition of the Rainbow Six soundtrack. The discussion will be followed by a workshop giving participants the opportunity to experiment and create their own music, using files from different instruments such as the octobasse, organ and ondes Martenot.

More information

Panel | Beyond the Controller : Accessibility in Gaming, Presented by EA Motive

The video game industry stands out for its innovative use of new technologies towards making the gaming experience accessible to everyone. Today, the broader development of accessibility features in video games includes audio descriptions, customizable controls, visual and auditory adjustments, haptic feedback, and much more. Disabilities can take various forms, it is important to be able to meet all needs.

Behind the softwares aimed at making gaming more inclusive, several consultants, such as Ross Minor, play a crucial role in bringing users first. As an accessibility consultant specialized in gaming for blind people, Ross has now been in the field for over 5 years. His lived experience as a blind individual and his deep passion for video games led him towards this educational vocation. He started out on the Internet with his famous How I Play Mortal Kombat X Completely Blind video. Today, he shares his expertise by collaborating with studios like Ubisoft, Interior Night, and EA Games. He has contributed audio descriptions—essential for blind and visually impaired players—for games such as Sea of Thieves and As Dusk Falls.

This Panel taking place on Thursday August 22nd at 1:30pm will invite Ross Minor alongside Améliane F. Chiasson from JAMTL, Frederic McNamara of Shishi Studios as well as Christian Cimon, Electronic Arts' Experience Design Director in a conversation putting video game accessibility at center stage.

More information

Presentation | Indigenous Innovations in Immersive Technology

Video games can be found in a wide range of genres, and those with an educational purpose require special attention because of the in-depth research they require. Video games possess a unique ability to convey history interactively, enabling players to interact with it firsthand.

The transformative potential of video games as an educational medium has the ability to innovate and recontextualize Indigenous realities—a crucial first step towards the decolonization of video games. These informal educational spaces often carry a colonial bias, all the more glaring when they manifest in virtual worlds inspired by historical events. In this sense, it is necessary to deprogram colonial structures and create universes and narratives representative of indigenous realities. Mikiwam addresses this need to shed light on indigenous perspectives and exemplifies the power of video games as a medium for storytelling and cultural transmission, facilitating the dissemination of indigenous knowledge.

Mikiwam: Solar Punk Herbalism


"Enter into an Indigenous world post-colonization" greets you as you dive into the fictional post-colonial world of Mikiwam: Solar Punk Herbalism, a video game developed by Studio Ekosi, an initiative led by Nehiyaw (Cree) sisters Keara and Caeleigh Lightning.

Studio Ekosi will present their project more in detail, alongside Revital Software, during the Indigenous Innovations and Immersive Technologies panel on Thursday, August 22nd at 4:30pm. This discussion, much like Mikiwam, encourages a reconnection between humans and nature while imagining the diverse forms video games can take.

At the moment, there are approximately 4000 Kanien’kéha (Mohawk) language speakers worldwide. In response to these alarming numbers and concerns of language loss, Revital Software—a language revitalization game studio based in Quebec, founded by sisters Kahentawaks and Wannekerakon Tiewishaw—has a mission to preserve and revitalize Indigenous languages in Canada.

This language learning platform aims to provide alternative resources to the classroom, creating a space for practicing and preserving indigenous languages. According to Revital Software, “These languages are important because they are a cornerstone of Indigenous sovereignty, and they are embedded with the accumulated knowledge & worldviews of our ancestors.” Designed to become an abundant language library, Revital Software uses video games to encourage daily engagement towards language learning.

The first batch of games will focus on Mohawk language and modern retellings of Haudenosaunee legends, expanding its library over time. In the spirit of keeping tradition alive and towards a decolonization of our minds, Revital Software will be presenting their project alongside Studio Ekosi on Thursday August 22nd at 4:30pm.

More information

In the digital age, video games are no longer just about entertainment. They educate, preserve cultures and open up dialogues on inclusion and accessibility. Initiatives such as those by Revital Software and Studio Ekosi show how immersive technologies can be used to enrich our understanding of the world and promote cultural diversity. With collaborations such as that between Ubisoft and the Orchestre symphonique de Montréal, the future of video games and electronic music promises to be rich in innovation and discovery.

At the same time, virtual reality (VR) and extended reality (XR) are transforming the way we perceive and interact with the world. A topic that will be addressed at the XR Salon and the iX Symposium on Wednesday, August 21, as part of the MUTEK Forum.

Writing and general editing: Gregh Chevron
Forum activities writing: Emma Garon
English translation: Lola Baraldi

Sources: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

Copy URL to share Copied to clipboard