A Feature, not a Bug: Fractals and Video Game Glitches as Nonnarrative Media

In 1975, Benoit Mandelbrot coined the word fractal to describe an iterative curve or geometric shape that can be divided into parts that each possess the same statistical character as the whole (“Stochastic Models” 3825). Taking its etymology from the Latin fractus, meaning broken (“Fractal”), his research sought to concretize a theory of roughness—that is to say, to provide a model for describing the recursive fractioning of the Earth’s coastlines, the structure of plants and leaves, the distribution of galaxies, the biology of blood vessels, and even human recreations such as music, architecture, and the stock market (The Fractal Geometry of Nature). Mandelbrot was not alone in his interests in such patterns: Gottfried Leibniz contemplated the phenomenon of self-similar recursion in the seventeenth century; Georg Cantor proposed the Sets Cantor fractal in 1883; and Wacław Sierpiński proposed the Sierpinski triangle in 1915 (Chengkun 264). These are but a fraction of the proposed sets and definitions that lead up to the nascence of the term and theory of fractals.

Today, fractal research is typically undertaken by various computer programs that process mathematical language, oftentimes transforming complex mathematical functions into striking visual art. Their uses are manifold, from image compression to predicting the growth patterns of bacteria to generating backgrounds and landscapes for films and video games. Brown, Witschey, and Liebovitch make the claim that “fractals abound in archaeology” (1), and that often-disregarded fractal analysis is indispensable to our understanding of non-linearity in cultural dynamics. In this probe, I posit that fractals are equally abundant in media archaeology, and propose that they provide a useful point of entry into thinking about nonnarrative media archaeology as articulated by Wolfgang Ernst.

In the second chapter of Digital Memory and the Archive, “Media Archaeography: Method and Machine versus the History and Narrative of Media,” Wolfgang Ernst proposes “an epistemologically alternative approach to the supremacy of media-historical narratives” (55), a nonnarrative method of media criticism wherein “media themselves, not exclusively humans anymore, become active ‘archaeologists’ of knowledge” (55). For Ernst, media archaeology is a way of analyzing elements of media that elude cultural history precisely because they are non-cultural.

Ernst provides the example of Athanasius Kircher’s water-powered machine for automatic music composition, a “generative set of rules and material that produce the musical impression” (71), and positions it in relation to software that composes audio algorithmically. Both models are nonarrative, algorithmic writing, much like fractal visualization software. Fractal visualizations for Windows Media Player, for instance purport to “move to the music in Windows Media Player Now Playing mode” (“Visualizations”). This process, however, is not the fractal reacting to ambient noise but rather the software producing both auditory and visual output that the human senses recognize as fundamentally different. Software processing is not beholden to the separation of data into visual and auditory channels understandable to the human senses, and Ernst consequently encourages a media-archaeological analysis that looks beyond the human-machine interface of technology. In essence, he encourages us to interrogate what is happening behind the screen and behind the speakers.

Fractal rendering software for Microsoft Windows and Macintosh, such as Apophysis, arguably offers artists and fractal art hobbyists a more hands-on approach to generating fractal flames than a visualization software that takes input from music, but the fractals remain a nonnarrative, mathematical rendering of aesthetics. In Apophysis, the user is able to input variables, select mutations that apply random edits to triangles, adjust the coloring, and directly manipulate the transforms. Despite this illusion of human artistic agency, the software does all the heavy lifting of running scripts and rendering the files as a viewable/printable/editable image. If, as Ernst argues, “media archeology discovers a kind of stratum […] in cultural sedimentation that is neither purely human nor purely technological, but literally in between” (70), it is in this liminal, moistmedia space that we locate fractal visualization.[1] Fractal art must therefore be understood differently from watercolour paintings or cultural texts; this media objects must be looked at in a way that is “enumerative rather than narrative, descriptive rather than discursive, infrastructural rather than sociological, taking numbers into account instead of just letters and images” (70). While we may look at a fractal flower and initially mistake it for a photograph or a digital painting, we must always come back to understanding that it is, in fact, mathematically produced.

As with nature, music, and art, nonnarrative and nonhuman visualization elements exist within games and their consoles, including my object of study this week, the Nintendo Wii and its predecessors. Softmodding a Nintendo Wii—a process I propose to document and interrogate in the coming days—allows one to run homebrew applications through the Wii menu, such as, for our purposes, a fractal generator by Krupkat called Mandelbrot. The mathematical application allows one to generate and modify fractals from the Mandelbrot set with the use of the Wiimote to point, zoom, cycle through palettes, and alter the number of iterations. While not as refined as the fractal flame renderers for Microsoft Windows and Macintosh, the Mandelbrot app nevertheless offers an interesting opportunity to intentionally generate and manipulate images on what is marketed primarily as a video game console. The program therefore raises compelling questions about the material implications of being able to generate fractals on a console rather than through the expected PC, and the impetus to consider what the Wii is capable of when not used as Nintendo explicitly directed or intended. Wiibrew, a wiki dedicated to homebrew on the Wii, lists five more homebrew applications alongside Mandelbrot in the category of mathematical applications, including a full-features calculator and a Pi calculation project.

More native to video game consoles, however, are the nonnarrative visualization elements that we call glitches—often fractal in nature—which generally break down into glitches of textures, models, graphics, physics, or sounds. Like Ernst’s technical artifacts, glitches “only disclose their essence when operating” (58)—they cannot exist without the calculating processes behind the game software, but, once powered they “actively generat[e] sensual and informational presence” (58). Narratively, glitches interfere with the playthrough of a video game as a cultural or archaeological object. From a nonnarrative perspective, however, glitches are not so negatively charged; instead, they offer insight into the composition of a video game, exposing hidden textures, model compositions, and a bank of sounds that are inaccessible in a glitch-free playthrough. Reading glitches through Ernst’s media-archaeological perspective directs players’ attention to the “noncultural dimensions of the technological regime” (61), paradoxically contributing to our understanding of video games as cultural objects. If media archaeology is interested in calculating memory rather than narrative memory, glitches offer insight into video games as objects by suspending human judgement about the aesthetic qualities that prevent us from perceiving beyond the surface. In such a way, it is fruitful to consider fractal output and glitches as being created by a media subject, expressions of the machines that are “active archaeologists of physical realities that are often inaccessible to human senses” (58).

As a result, we might ask ourselves questions about the agency of a video game console, the console’s preservation of an archive, and its ability to store and process data beyond human comprehension. Moreover, we may also consider the difficulties of capturing the variations inherent in digital-based arts and glitches—how can not only the visual output of a glitch or fractal but also the process leading up to it be preserved? While we can screen capture a visual or physics glitch, replicating the glitch after a hard reset may be impossible. For fractal art, for each rendered flame, there are infinite possibilities that are lost to us as archivists and, furthermore, the very act of rendering a fractal creates a different type of mathematical file entirely. So how can a given technology alter and present data from its archive?  These questions, raised by Ernst’s conceptualization of media archaeology can provide fruitful avenues for examining media that often goes unacknowledged or is dismissed as a malfunction.

Note

[1] Roy Ascot’s theorization of moistmedia sees dry silicon computerization systems and wet biological systems converging to produce a new art, design, performance, and architecture. While moistmedia typically combines biotechnology, engineering, and telematics, Chengkun makes a case for understanding fractals as moistmedia through the study of the visualization of social information.

 

Works Cited

Ascott, Roy. Art, Technology, Consciousness: Mind @ Large. Intellect Books, 2000.

Brown, Clifford T., Walter R. T. Witschey, and Larry S. Liebovitch. “The Broken Past: Fractals in Archaeology.” Journal of Archaeological Method and Theory, vol. 12, no. 1, 2005, pp.37-38.

Chengkun Tang. “From Experiments in Visualizing Fractal Theory to Rethinking Social Networks as Moistmedia.” Technoetic Arts: A Journal of Speculative Research, vol. 14, no. 3, 2016, pp. 263-273.

Ernst, Wolfgang. “Media Archaeography: Method and Machine versus the History and Narrative of Media.” Digital Memory and the Archive. Ed. Jussi Parikka. Electronic Mediations 39. Minneapolis: University of Minnesota Press, 2013. 55-80.

“fractal, n. (and adj.).” OED Online. Oxford University Press, March 2017. Web. 21 May 2017.

Madelbrot, Benoit B. “Stochastic Models for the Earth’s Relief, the Shape and the Fractal Dimensions of the Coastlines, and the Number-Area Rule for Islands.” Proceedings of the National Academy of Sciences, vol. 72, no. 10, 1975, pp. 3825-3828.

— The Fractal Geometry of Nature. W. H. Freeman and Co. 1982.

“Visualizations for Windows Media Player.” Microsoft Support, 9 Sept. 2016, support.microsoft.com/en-us/help/17878/visualizations-for-windows-media-player.