Depart/Transcend: An Audio-Visual Web Installation
Depart/Transcend is an audio-visual installation accessible by navigating to https://www.depart-transcend.com on a modern web browser[1].
produced by: Armando González Sosto
Introduction and Background
Depart/Transcend is a collaboration between Andrés León and me. This piece is our second audio-visual collaboration following Duality[2], presented at Berklee College of Music in 2018.
León composed the music to Depart/Transcend as a reaction to the death of a mutual friend in 2019. With a duration of 34 minutes and 22 seconds, the music provides space for reflection and meditation through the usage of repetitive motifs performed on a piano and processed with generative granular audio processing techniques.
With León and I being in different countries during the Covid-19 pandemic, it was clear that the internet was going to be the primary form of presentation for this piece. We decided to move forward with an installation where our presentation space would be a web browser, running a custom site I'd develop from scratch.
I'm interested in working with the internet as a form rather than as a medium: instead of using the web as a means of sharing a pre-rendered piece, I’m interested in situations where the internet is a protagonist that shapes the development of the work. This idea brought forward a series of important artistic questions I wanted to explore.
Existing autonomously on the web
A core feature of Depart/Transcend is that it lives entirely on the web. In order to experience the work, an audience member must visit its web address, which doesn’t present a recording or a transmission of the performance, but rather is the performance.
Artists tend to use the internet as a means of distribution for work that derives its shape from its historic origin on other media. This can bring a layer of difficulty when bringing it into a new context, which became especially clear during the early stages of the Covid-19 pandemic, where many artists drifted towards live-streaming solutions to find alternatives for live performances. The limitations brought on by these solutions were accepted by audiences mostly due to the unprecedented nature of the period, but as social distancing became the norm, what made these performances initially tolerable began to fade away.
I attribute the gradual loss of interest and relevance in these types of performances on the lack of structural autonomy they can achieve. With structural autonomy, I’m alluding to the ways in which the structure of a medium constrains an artistic expression, and whether work developed under these constraints tends to yield an autonomous artistic statement. To understand the structural autonomy of a piece, it's important to determine the role played by the medium: is the medium a pre-requisite of the artistic statement, is it a functional distributor, or does it stand in the way?
Working with a web-based installation allows artists to bring a degree of structural and artistic autonomy to their work, as the work is being channeled through the medium it was conceived and constructed for. Naturally, this can extend to mean a variety of things, and many forms of expressions have been borne into the reality of the internet and can stand as structurally autonomous artistic statements. However, in order to have the degree of flexibility I was looking for in Depart/Transcend, I built a web site from scratch using a variety of technologies to bring into question what the definition of an installation can be.
Web Installation
The web is interestingly amorphous: its physicality is hard to pin-point, and, as put by Umberto Eco when describing nets in The Encyclopaedia as Labyrinth, one of its main features is that “...every point can be connected with every other point, and, where the connections are not yet designed, they are, however, conceivable and designable”[3].
The web is amorphous as its connections are innumerable, its content and form can be constantly re-shaped, and the ways in which it is mediated are constantly in flux. However, it can be somehow intuitively thought of as a space, especially when interfaced through a web browser. I believe this is the case since browsers bring a degree of dimensionality to an abstract concept by providing a visual frame to access it, and in combination with the traditional window metaphor used by operating systems, it stimulates this vivid image of the web having properties shared with spaces.
However, dimensionality is not the only concept that browsers bring to the potential spatiality of the internet, which is important since spaces, especially in the arts, have more properties than those associated with physical dimensions. As Heidegger states in Building Dwelling Thinking, “the bridge does not first come to a location to stand in it, rather a location comes into existence only by virtue of the bridge.”[4] Relating this concept of locality to the spatiality of the web, the web browser creates a space by virtue of connecting multiple locations - “spaces receive their being from locations and not from “space”.” The spatiality of the internet can be derived in part from the visual dimensionality provided by its standardized mediators (i.e. browsers), and by the locality brought by its inherent connectivity.
Spatiality is an important component of web installations, and the fact that a web browser can be thought of as a space provides the opportunity to use its spatial characteristics as artistically modifiable parameters. However, the dimensionality of the web and web browsers isn't limited to the Cartesian plane presented on browser windows, or even to the 3D projections that are possible using modern systems. Its dimensionality is more subtle and is dependent on its unique attributes of connectivity and mutability. The exploration of the dimensionality of web-based art and web-installations is one of the core concepts explored in the piece.
It’s also important to consider the importance that space and site-specificity have when thinking about installation work. Juliane Rebentisch articulates this concept in Aesthetics of Installation Art when describing site specific art, by stating that the “concept of space...is always implicitly at stake when we speak of site-specific art, as the latter opens up to its concrete environment - the space surrounding it.”[5]. The fact that the piece is presented on a specific website brings into focus the question of a webspace. This connects with the previously mentioned ideas of spatiality. Ultimately, we can think of a website as a space, as it has dimensionality, and it has a location.
Concepts Explored
All the discussion on form is ultimately irrelevant if there’s no attempt at expressing an artistically and personally meaningful idea. The intention for this piece is to be able to develop an art piece that can stand as an autonomous artistic statement that leverages web technologies to make a special point about a its specific concept.
The concept of death is difficult to navigate and understand. As humans, we obviously understand the evident aspects of death, but attempted answers to the central questions personally never feel satisfying. I believe that obvious-to-ask questions contain intriguing ideas to work with, especially when they tend not to yield satisfying answers, as these often are the questions that fundamentally follow us around through life. Depart/Transcend is centered around one of these ideas: what is death?
After asking this question, the obvious becomes self-evident: it is impossible for the living to understand death. Given we can’t truly understand death, perhaps through a reflection on the nature of time we can begin to form a relationship to an underlying structure of existence. An understanding of this underlying structure can help us begin to get closer to the vessel of time that propels us towards death, and perhaps this can help us understand what our temporality means to us as individuals that will eventually perish and decay. Life is subject to time. The only certainty in life is that the passage of time brings death closer. Thus, time became a central concept in the development of this piece.
Time also becomes an interesting concept to intertwine with the previously mentioned ideas of networked art, especially when exploring the notions of centralization and localization that networks can provide. Specific explorations of these ideas will be explored when discussing different components of the piece.
Technical Configuration
Despite being accessible through a single URL, the project consists of two servers which run distinct programs. This separation provides an interesting insight into the obfuscation of transparency inherent to internet architecture. This obfuscation provides an interesting artistic resource, as it allows for the exploration of centralized and localized processes.
Both servers are written in the TypeScript programming language, a version of JavaScript with added optional typings[6]. For the sake of simplicity, TypeScript and JavaScript can be used somewhat interchangeably, as their difference lays mostly on the experience had by a developer writing code. Additionally, TypeScript is ultimately always transpiled (converted) into JavaScript which is then interpreted by JavaScript engines, such as those found in Node.js and in web browsers.
Technical Configuration: Front End
The first server runs a front-end, client-side website. This website’s code is downloaded and executed locally on an audience member’s computer when they access the URL. This code has dependencies across a variety of libraries, which will be explained in order to contextualize the structure of the site.
React
The front-end is centered around React.js, a “JavaScript library for building using interfaces''[7]. The code for Depart/Transcend was initialized using create-react-app[8], a command-line tool which allows programmers to create boilerplate code to create a functioning React app. React.js is an open-source library created and maintained by Facebook, which, among other things, allows developers to create reusable blocks of code in order to build interactive and reactive user interfaces.
In the context of Depart/Transcend, React.js is used as a “container”. This enables the coexistence of a multitude of libraries contained in a centralized page-centric architecture. This means that as an audience member navigates through pages, there is a high degree of similitude in what they’re seeing and in how the code is structured, which in turns allows for complex navigation patterns which are easy to manage from a coding perspective.
p5.js
p5.js is a JavaScript library used to develop interactive graphics with a very simple syntax heavily indebted to its precursor, Processing[9]. In this project, p5.js is used as the main tool for graphics, and a collection of sketches was developed using 2D canvases and 3D webGL graphics.
Socket.io
Socket.io is a communications library that leverages technologies such as WebSockets to create full-duplex communication between a server and a client[10]. This library is the main way in which the back-end server and the front-end server communicate. Unlike the standard HTTP protocol, WebSockets allow communications to flow continuously between nodes in a network without the need for constant requests. Socket.io mostly works with WebSockets, but it also automates HTTP requests for cases where WebSockets are not usable.
Figure 1 shows how the two servers communicate between each other using Socket.io:
Figure 1 - Structural Diagram
Other libraries
material-ui - Provides reusable User Interface components (buttons, switches, toggles)
i18next - Provides tools to manage internationalization and translation
matter-js -A physics engine
react-audio-player - A lightweight audio file player for React projects
react-router: navigation framework
Technical Configuration: Back End
The backend consists of a simple application built using Node.js, a JavaScript compiler that runs independently of a browser[11]. Node.js allows developers to build stand-alone command line applications that can be accessed and interfaced through networks. In the case of Depart/Transcend, the technical configuration of the back end is extremely simple and will be explored in depth when examining the sketches of the piece.
Technical Configuration: Architecture
Figure 2 presents a diagram that describes the structure of the project with more elaboration than Figure 1. The intention behind showing this diagram is to explain the structural dimensionality of the piece. This can be useful to have in mind when attempting to understand how the different sketches are presented to an audience as the piece develops.
Figure 2 - Structural diagram
The structure of Depart/Transcend consists of a series of p5.js sketches which are automatically interchanged through the usage of a score – a JSON file which contains an array of objects describing the duration of a sketch and the ID of the sketch that follows it. These sketches are used as vignettes which reflect on a focalized concept derived from the central ideas of temporality the piece is looking to allude to.
The approach of presenting visual concepts through vignettes offers an interesting juxtaposition with the long form of the sound of the piece. The long form of the music in of itself brings into question notions of temporality, largely in part to the meditative effect that is achieved through the constant repetition of slightly modified phrases. This temporal environment is further developed through the construction of different temporal envelopes within its own structure. This creates a multiplicity of timelines and temporal environments that bring into question what the nature of time can be in of itself.
Vignettes of time
Questioning the nature of time is the core concept explored in the vignettes that constitute Depart/Transcend.
On one hand, I was looking to explore how networked systems bring an interesting perspective into the nature of different temporal mechanisms. By working with centralized and de-centralized time keeping methods, I would be able to explore how ideas of temporal relativity take part in day-to-day life, also bringing into question notions of spatiality due to the networked elements of the piece.
Additionally, after reading A Minor History of Time without Clocks by Joshua Foer[12], I also became interested by the realization that the nature of time-keeping mechanisms and the conceptualizations that propel their use are inherently tied to the time under which they were constructed. This means that the definition and construction of time is fundamentally dependent on the moment in time under which they were constructed. Time can be arguably seen as a recursively definable concept, as the understanding of time is dependent on the time in which it is being attempted to become understood.
One of the features this approach of understanding time brings forth is the fact that different time periods will bring forward different approaches to conceive timekeeping. This led me to become interested in the commonalities and differences that can be found between different attempts at keeping time. Given that defining time itself is incredibly complex, I’m mostly interested in seeing what different timekeeping methods say about time. Perhaps, an understanding of time can be found by looking at the overlaps between different timekeeping methods.
A brief exploration of some of the vignettes developed for Depart/Transcend will be realized to explain the ways in which these ideas materialized in the context of this project.
Vignettes of time: Landing page
Figure 3 - Landing Page
Figure 3 demonstrates the first page shown when accessing the piece. This page is shown in part as a requirement for using audio on the web – in order to play audio from a website, a user must first engage with the site in some form, in the case of this site, by clicking Enter. This page also features additional visual elements, including a wandering platonic solid, and two strings of incomprehensible text – one on the top of the screen and the other in the bottom of the screen.
These two strings of text are generated through the usage of a genetic algorithm, a type of algorithm which seeks to express the nature of evolution found in genetic systems. In this specific case, the genetic algorithm is used to evaluate the suitability of a given string of text in relation to a desired target string. If a given string is deemed suitable, its “genetic material” is passed onto the following generation after having its genes combined with another suitable “parent”[13]. This process keeps repeating itself until, at some point, a given string perfectly matches the desired target. As it might be evident, the target in this case is the title of the piece.
The reason this algorithm is presented twice in this page goes beyond the aesthetics associated with the visual symmetry of the page. The string of text presented on the top of the page is generated remotely in the back-end server, and every single visitor of the site is seeing the same string of random text presented on their browser. The genetic process is being performed whether a visitor is active on the landing page the page or not, and through the usage of Socket.io, it’s certain this value is consistent across all nodes connecting to this site.
On the other hand, the string that is presented on the bottom of the screen is being generated locally on the computer of the viewer, and as such, is different for every audience member. The locally generated text takes the current state of the external evolution when accessing the site as its point of departure and continues evolving from there. The two strings of text are evolving using the same algorithm, but their evolution ends up being significantly different.
The reason I became interested in working with genetic algorithms is because I see evolution as the longest temporal envelope in which living organisms are involved, and as such, I believe it is one of the purest articulations of the concept of time we can access.
Evolution is subject to time, and the evolution of our species is what made it sensitive to temporal processes. I wanted to explore evolution as an elucidation of time, an idea I was able to explore further by networking the development of genetic algorithms to central and remote evolutionary nodes. Time seems to flow differently when its centralized or localized, and this vignette attempts to allude at this.
Vignettes of time: Ancient Timekeepers
Figure 4 - Gnomon
Of all historic time-keeping mechanisms currently known, the gnomon (or sundial) is one of the earliest examples. By showing the gnomon on the sketch demonstrated in Figure 4, I was looking to allude to the earliest system of time keeping as a way of alluding to the beginning of the piece. This sketch uses WebGL to create a 3D environment on the web that I can then rotate and manipulate.
This begins the process of looking at time keeping mechanisms as a way of exploring similarities and differences between different systems used to keep time. This trend was continued in the sketches demonstrated in figures 5 and 6, which demonstrate pendulums and hourglasses to allude to time.[14]
Figure 5 - Pendulum Vignette
Figure 6 - Hourglass Vignette
I think it can be powerful to allude to these symbols of timekeeping in sketches that then modify their shapes as they contain a heavy cultural legacy associated with their imagery. This cultural imagery is later juxtaposed against other timekeeping mechanisms, but it also can stand as a statement in of itself, because despite being artistic vignettes, most of these sketches can actually track the progress of time, in one way or another.
Vignettes of time: Modern Alternatives
After alluding to some of these traditional methods for timekeeping, I looked to create sketches that would also serve as timekeepers in an alternative way. These sketches are not seeking to keep time in a useful manner, but rather, they look to present ways of keeping time that would result in a reflection of temporality.
For example, figure 7 demonstrates a trigonometrically inspired sketch that shows distorted sine and cosine waves being plotted on a Cartesian plane, as an uncontextualized value for “age” is presented in the center of the screen. These periodic waves are distorted not out of a design choice implemented through code, and there is no deliberate, software-induced noise generation.
However, these waves are generated in the backend server and communicated over a network of varied reliability, with the induction of noise being a result of the latency and distortion induced by communicating information from a centralized point. The age value displayed in the center of a screen simply displays the sum of frames counted by the server, which has been counting since it last was booted up.
I believe this sketch shows how periodicity can be easily distorted and warped as it is communicated between different nodes, bringing focus into the relativity of time. Granted, the noise found in these waves is probably brought on due to the technical limitations of current network protocols, but I still believe that the fundamental idea is clear: this timekeeping method is being actively constructed on two nodes, and its construction is fundamentally distorted by the fact that it needs to be communicated between remote locations.
Figure 7 - trigonometric timekeeping
Another alternative time-keeping method used in this piece can be seen in the Calendar Vignette, shown above. This sketch shows a collection of random dates cumulatively being added in random intervals of time. A date is generated randomly, if this given date took place before the date in which the sketch is being seen, a synonym of the word “past” will be displayed in the center of the screen. Conversely, if the randomly generated date takes place after the date the sketch is being seen, a synonym of the word “future” will be displayed in the center of the screen.
The goal of this sketch is to reflect on the varying perceived speeds of time as life unfolds. This sketch centers temporality on the time in which the sketch is being seen, with both the past and the future being non-existent concepts that are reduced to a simple adjective. My focus during the development of this sketch was to center time around the idea of now, as every other moment that can possibly exist is predicated on the existence of now.
Figure 8 - Calendar vignette
Conclusion
Depart/Transcend marks a significant piece in the development of my computational work. On one hand, I’m satisfied with the technical achievements accomplished in this project, as I believe it shows a degree of familiarity with the established and cutting-edge technologies of the web. This enables a lot of creative possibilities for future projects, which is exciting. I’m proud of the quality of the code written, and I believe it’s a stable system I can continue to build on in the future, allowing the piece to grow and evolve as time goes by.
The level of familiarity with technologies I’ve obtained is important to me because it allows me to be able to articulate ideas on the internet in a way in which I believe can be unique to its given context. I’ve often thought of the internet as a confusing tool for artists – sometimes it’s hard to fully understand how to properly express oneself in it in a manner that makes the most out of the idea and of the medium. I believe this is a struggle that will continue to exist as I move forward, but this piece will mark an important point in how I relate to the notion of creating art that will mostly be presented on the internet.
The writing of this document also allowed me to explore ideas in a manner that I find interesting and important. During my experience in the MA Computational Arts program at Goldsmiths’ College, it’s been interesting to discover how the world of arts criticism and arts theory is structured, because I feel it is extremely different to how the world of music criticism and theory work. Coming from a musical background heavily focused on the technicalities of jazz harmony, I find the philosophical concepts put forth by Heidegger and Adorno (to name a few) to have a highly enriching effect on how I understand my role as an artist and my relationship to the media I work with. It’s not to say that I’m not grateful for my knowledge of jazz harmony, but I’m mostly trying to point out the artistic benefit of being acquainted with a new approach of conceptualizing what art theory means.
Finally, I believe this is a project that our friend would’ve enjoyed. Without wanting to resort to emotionality, I believe it’s worth stating that his warm support and friendship during my time as a music undergrad were extremely meaningful. He was a lab assistant at my undergrad university department, and he was a central part of the communities that developed there. Upon contemplating the bizarre thought of his absence, the words of Einstein bring warmth following the bitter taste of his departure:
Now he has departed from this strange world a little ahead of me. That signifies nothing. For those of us who believe in physics, the distinction between past, present and future is only a stubbornly persistent illusion.[15]
Reference:
[1] The piece was tested on Chrome v90 and Safari v13.1.3. HTML5 compatibility is required. It is
also recommended that only this tab is active on the browser, and no other programs are running
in the background.
[2] https://www.youtube.com/watch?v=Vr1huJUVVsk
[3] Lars Bang Larsen. Networks [Theodor W. Adorno, Pamela Allara, Lawrence Alloway ...].
London Whitechapel Gallery Cambridge, Mass. Mit Press, 2014.
[4] Rebentisch, Juliane, et al. Aesthetics of Installation Art. Berlin, Sternberg Press, 2012.
[5] Rebentisch, Juliane, et al. Aesthetics of Installation Art. Berlin, Sternberg Press, 2012.
[6] “TypeScript - JavaScript That Scales.” Typescriptlang.org, 2015, www.typescriptlang.org/.
[7] Facebook. “React – a JavaScript Library for Building User Interfaces.” Reactjs.org, 2019,
reactjs.org/.
[8] “Create React App · Set up a Modern Web App by Running One Command.” Create-React-
App.dev, 2019, create-react-app.dev/.
[9] “P5.Js | Home.” P5js.org, 2019, p5js.org/.
[10]“Socket.IO.” Socket.IO, 20 Oct. 2019, socket.io/.
[11] Node.js Foundation. “Node.js.” Node.js, 2019, nodejs.org/en/.
[12] Groom, Amelia. Time. Cambridge (Mass.) ; London, The Mit Press ; London, 2013.
[13] Shiffman, Daniel, et al. The Nature of Code. United States, D. Shiffman, 2012.
[14] Groom, Amelia. Time. Cambridge (Mass.) ; London, The Mit Press ; London, 2013.