by Aurelio Meza
The showcase video Super Mario Spacetime Organ (illucia & Soundscape) by Chris Novello starts asking, “What is a video game?” We could expand the scope of this speculation and ask, “What is an instrument?” (Georgina Born and Joe Snape finish an article on MaxMSP exactly with that question). These two interrogations bring together Max, customized instruments, and video games into the discussion of devices and instrumentality.
How devices are operated through a physical interface is central to this discussion. Game controllers, just like piano or computer keyboards, can be defined as boundary objects (after Susan Leigh Star)–surfaces of interaction between an instrument and its user (here I’m using the notion of instrument in a very broad sense). In a way they are designed to strain the user, so that they are allowed to enter just a very specific sort of input to the program system (in the NES controller, barely one byte of input). Through this project, Novello skips the default constrains of the traditional NES controller and overlaps video game interfaces with analog synthesizer ones.
I found it interesting that, for the creation of his “controller,” he combines the Moog synthesizer technique of patching (which Max would later use, too) through the patchbay illucia, created by Novello himself, and the heterodox vertical interface of the first Buchla models through Soundplane (a multitouch device by Madrona Labs), which also reminded me of the Trautonium, a keyboard-based instrument developed in Germany during the Nazi regime (studied by Thomas Patteson). Through this “mash-up,” Novello is not only proposing a transmedia work of art, but he is also dabbling in the eternal Moog vs. Buchla discussion. Novello’s “controller” combines techniques that, in the beginning of synthesizer history, were in conflict with each other.
The fact that I’m being so careful about the word “controller” makes me wonder what it means that illucia and Soundplane are used together. Not that it is unheard of to use two different interfaces to operate a video game–we can use the alphanumeric keyboard and the mouse/touch pad for computer games. But the fact that he cobines them complicates the notion of instrument. In his Vimeo account, Novello states, “I directly manipulate the RAM of Nintendo’s Super Mario Brothers to transform it from a game into a strange instrument.” But more than just a “strange instrument,” I think it is an assemblage of techniques and devices that work together in order to get a broader output from the video game. Are these composite aggregates instruments, or do they go beyond this concept? If so, such aggregates would still be instrumental. Must all instruments be restrained to one single device to be considered as such?
Just as I was writing this post’s draft, Patrick told me about Ming Mecca, a similar project by Jordan Bartee, which enhances this discussion. By watching the showcase video, it seems that the “compatibility” between sound and image is much more straightforward. In that sense, Ming Mecca is more image-oriented than sound-oriented (in that sense, more akin to traditional video games), and it takes the patchbay technique to a very different dimension from which it sprang (from sound generation to audiovisual outupt creation).
Are these two artists hacking video games, modding them, or creating something new? This question is more difficult to answer in Novello’s case, as Bartee is doing most of his work from scratch, whereas Novello is using a pre-existing video game, and one of the devices was designed by someone else. I’m so sad I cannot write more than just a blog post for these projects, because I feel they deserve a more properly developed analysis.
