Source: Entertainment Designer.
Can interactive technology help audiences to better understand how music is constructed? Can audiences step inside a song and deconstruct the mathematical concepts that make it whole? In an interactive exhibition last month at the New Museum, musician Matthew Dear let audiences experience just that. Microsoft Kinect technology helped create a collaborative performance where the audience played a key role in manipulating a song in an interactive sound environment. The installation explored an immersive environment that blurred the lines between creator and audience to accomplish one goal: playing and enjoying music.
The museum describes the relationship of technology and performance within the musical context: “Throughout history, from Bach to Björk, musicians have sought out technology as a means of adding new dimensions to their performances. Today, technology is also increasingly democratizing music, as the tools to create and share music are more accessible than ever.”
Matthew Dear describes it as a “living breathing sound sculpture.” Dear created an original composition just for the exhibit. He then worked with an incredible team to create the DELQA infrastructure and context that would allow participants to interact with the music. Audience participants manipulate their environment to influence the music. Guests play with different elements of the physical environment, which in turn changes the song or introduces new elements to the music.
The broader team included many players, not limited to Listen which provided the creative development and production. The Principals provided architectural and environmental design. Dave Rife and Gabe Liberti were in charge of spatial sound. The team also included a Kinect technology advisor, a music interaction director, a design director, and a media artist.
Microsoft Kinect is a technology that many people are familiar with as a gaming platform. But the technology is being used more widely, in part for artistic exploration. The team behind this project decided to create a world that would allow guests to step inside Dear’s music. The sonic environment was created by connecting 40 speakers that were linked into a spatial audio system and created what was essentially a 3-D field of sound.
Every aspect of the environments= and every element of the song could be influenced. As the team described, the audience was brought “into the individual musical ideas that make up a song.” Using Kinect, every element of the environment was covered and every surface became a conduit for audience actions.
The team, for example, sourced specific fabric that would allow them to create a membrane -like environment for part of the exhibition. The fabric was opaque to depth cameras in a way that allowed the team to map the surface. But it was transparent to infrared cameras, so the audience’s actions could be seen through the membrane. When they push on the walls to displace the membrane, they changed the quality of the music. Users can change what’s playing or introduce entirely new parts into the music. The goal was to make each action feel powerful and have an impact on the music. For example, when you press on the fabric in a certain place, the density of the drum rhythms could change. At the same time, the other musical qualities were also changed which allowed for a multi-dimensional experience of the sounds.
DELQA is a powerful example of what’s possible when creativity, music and technology merge. The idea of a 3D soundscape that brings music to life as a living sculpture is compelling. Audiences are craving interactive experiences that draw them in in entirely new ways. DELQA’s approach to musical performance is great model for future performances.
Images and video sourced courtesy of PSFK, YouTube