Nostalgia Mode
An idiosyncratic musical performance device, made as a meditation on nostalgia and an audio-visual performance system inspired by hauntology.
produced by: Benjamin Sammon
Introduction
Nostalgia Mode is a blurred contemporaneity, looking at nostalgia through the lens of the present. Edison originally created the phonograph as a way of preserving voices of the dead, now we have the power to create voices for the non-living.
Concept and background research
I wanted to create a temperamental instrument evoking a reflection on the fragility of memory. I was heavily inspired by Kim Cascone’s The Aesthetics of Failure which discusses the technical failures apparent in 1990’s digital music production and describes how a group of musicians incorporated mistakes in their art. I was also inspired by Rosa Menkman’s Glitch Studies Manifesto which suggests using glitch as a method of bending and breaking the medium until it becomes something new. I decided to utilise the mistakes the marble run makes as an artistic choice; the erosion of a ‘norm’ makes those ‘mistakes’ the new status quo and the idiosyncratic nature of the device becomes its own character.
I played around with the idea of shared agency which fascinates me, deciding to become less of a performer and more of an operator of sorts; giving the marble run an extent of agency to perform in its own way.
I wanted to make the marble run modular and switchable, specifically by exposing the circuitry so you can easily switch elements of the board out; expressing the fragility of the piece. It is a reflection on nostalgia as sometimes we think we remember something, but memory is fallible.
Nostalgia mode, coined by Frederick Jameson, means a nostalgia for the present, and this nostalgia uses anachronisms that are out of joint. I was very interested in the idea of time being out of joint, so I endeavoured evoke this concept literally by time stretching a Delia Derbyshire song. Jim Jupp sums up my process eloquently in Mark Fisher’s book Ghosts of my Life “It’s revisiting old textures and old imagined worlds with new tools.”
In terms of aesthetic design, my project was inspired by Oskar Krajewski's neon and raw miniature futurescape, Recycled Future and the musical instrument is similar to Graham Dunning’s Mechanical Techno which uses piezo microphones to trigger drum beats as they rotate on a record player.
Conceptually, I was strongly influenced by the ideas of hauntology discussed by Jacques Derrida and developed further by Mark Fisher and Simon Reynolds. Hauntology focuses on a temporal disjuncture where modern society is haunted by cultural spectres of the past. Musically, sampling is used a great deal in hauntological pieces, especially sounds which are evocative of deep and sometimes forgotten cultural memory. My piece features a time stretched and reverb-soaked sample of a Delia Derbyshire song which creates a haunting ambient backdrop, reflecting the lost future of the BBC Radiophonic Workshop. I also see the process of recycling a piece of technology (the marble run) and re-purposing it as hauntological.
Technical
For this project I used a series of 3 Arduino’s, one with four piezos, which I utilised as force sensors, and three potentiometers. At the project’s conception, I had to work out which input device would be best triggered by a falling marble. I tested a number of solutions including velostat, FSR’s, soft potentiometers and infrared sensors. The velostat and soft pot did not fit the desired purpose and the FSR was too large to fit the tube alongside the marble. I decided to try to set up a small infrared transmitter and receiver either side of the marble run, sending data as the marble moved through the pipes. This was also a failure and proved harder to mount in an effective way. Going back to the drawing board, someone suggested that I use piezos, which turned out to be the ideal size. This initially seemed like the perfect solution but after damaging 15 of them I realised that they required iteration to improve sustainability. I decided to hot glue a layer atop each piezo, holding it together more effectively, whilst providing extra padding to protect against impact with the marble. These piezos sent raw analogue data to Max/MSP which then converted any value above a certain threshold into a bang, randomly triggering a musical note.
I also used two Arduino’s to control the NeoPixel lighting display imbedded within the marble run. I decided to use two as I had two different lighting setups and sending both to a single Arduino caused lag issues. The first 3 LED strips faded in and out when certain notes were triggered in Max/MSP. The second 3 sets of strips each contained 8 LEDs which corresponded with one of 8 notes played in Max/MSP.
In Max/MSP I had 3 different musical parts setup. Firstly, I used granular synthesis to time stretch and rearrange grains in the aforementioned Delia Derbyshire sample. Secondly, I had a Karplus-Strong algorithm synth, controlled by the piezo inputs. Which could be switched out for the ‘arpy’ sample instead. Finally, I had a series of 8 metro objects which determined the tempo of each note in the final section of the song. These metros provided an equally spaced tempo, which would cause each note to gradually go out of sync and create interesting polyrhythms.
I setup a system where Max would receive serial data from the piezos/potentiometers and send serial data to the LED strips. It was then connected to Unity via OSC allowing the audio from Max/MSP and potentiometers to affect the visuals in Unity. I decided to use Max specifically because it works perfectly across these tasks; it can handle my music and data wrangling with no discernible lag.
Visually, I wanted to create an interesting superstructure made up of marbles to reflect the project. I had previously created supershapes in Processing but decided to port this concept over to Unity. The lighting and shader options would be far superior, and I found the OSC communication worked more fluidly than with Processing. Plus, scripting different elements in separate C# scripts made the workflow much easier.
I created a number of different supershapes but ended up with a formation that was more of a sphere. This worked best on the 3-projection setup in the SIML space as you could more accurately see what was happening in real-time. I used an iridescent shader that would simulate a marble type pattern and my system was setup so that certain triggers would change elements of the supershape. Different sensors in the run would trigger different visual effects upon impact with the marble. These triggers affected the camera perspective and hue of the shader, whilst the potentiometers affected the supershape structure and fading from/to black. Finally, the granular synthesis affected the scale of the individual spheres.
To avoid lag, I decided to run Unity and Max on separate computers. I acquired a router from creative facilities and used that to communicate via OSC between the two machines, sharing the load of the processing power. I used my laptop for Max/MSP as that would be the lighter of the two processing and I used the Mac Pro to run my Unity app as it is a more substantial machine.
Future development
I feel this project is a good workable iteration, but it could be developed further. Firstly, I would like to replace using Max/MSP on my laptop with a Raspberry Pi using Pure Data. I would also like to develop a headless Pi solution, potentially with more physical computing elements, replacing my need to use my laptop live.
Secondly, I want to design my own 3D printed marble run parts. With bespoke parts I could make the system even more modular with the ability to replace parts on the fly, with each element making a different sound. This will also enable me to build a larger and more robust marble run and use it in a similar way to Eurorack style modular synthesisers.
Finally, I think this project can be developed into an installation the public can interact with, developing their own sounds. This could consist of a series of marble runs that work effectively as different instruments. There is also potential for a mechanical marble run that plays itself.
Self evaluation
The network solution I created, using 3 Arduinos, 2 computers communicating between serial and OSC using Max and Unity all worked well. It wasn’t without issue but by iterating I managed to solve most of the problems. I think all of the visual elements worked well in unison, but I would improve on the actual performance. If I made a few amendments, the intensity of the visuals could better match the intensity of the audio.
My initial goals were to create an audio-visual performance involving an unconventional performance device. I believe I achieved this, as I aimed to go against the curve of proprietary made MIDI controllers, like conventional keyboard style setups. I wanted something that wouldn’t create robotic and perfectly quantized sounds but would do the opposite. In a way it has its own personality and agency, which can be both a detriment and a strength.
In future, I would improve on some issues with the marble run. As previously mentioned, 3D printing my own parts could alleviate issues cause by the limitations of a pre-made marble run. Sometimes marbles would get stuck because of the LED strips or the piezos. This was largely ironed out but if I want to develop this into an installation in future, I will have to work on these issues to make sure that it is more user friendly.
References
Cascone, K. 2002. THE AESTHETICS OF FAILURE: 'Post-Digital' Tendencies in Contemporary Computer Music. Computer Music Journal 24:4 Winter 2002 (MIT Press).
Derbyshire, D. 1984. Mattachin. BBC Worldwide Music.
Derrida, J. 1994. Specters of Marx. Chapter 1: Injunctions of Marx. Routledge.
Fisher, M. 2013. Ghosts of My Life: Writings on Depression, Hauntology and Lost Futures. Zero Books.
Jameson, F. 1991. Postmodernism, or, the Cultural Logic of Late Capitalism. Duke University Press Durham.
Menkman, R. Glitch Studies Manifesto. Amsterdam/Cologne, 2009/2010 http://rosa-menkman.blogspot.com
Reynolds, S. 2011. Retromania: Pop Culture's Addiction to its Own Past. Chapter 10: GHOSTS OF FUTURES PAST Sampling, Hauntology and Mash-Ups. Faber and Faber, inc.
Max/MSP Links and Tutorials
- Norris, M. 2016. spindrift~ granular synthesis Max external. http://www.michaelnorris.info/software/spindrift
- Max Karplus-Strong synth tutorial https://www.youtube.com/watch?v=Yb8JuFHJxQQ&feature=youtu.be&fbclid=IwAR33cNTXvjzMpKRQvUTqQa7Axx6KAm683_LpN-U9bb_XPNtuwsyYNOqkYCg
- MAX MSP OSC - https://www.youtube.com/watch?v=bupVHvMEAz0
- MAX MSP Arduino - https://www.youtube.com/watch?v=6bT3G4Mep7E
Arduino Links and Tutorials
- Setting up a piezo with Arduino - https://www.sparkfun.com/tutorials/330
- Adafruit NeoPixel Überguide - https://learn.adafruit.com/adafruit-neopixel-uberguide
Unity Links and Tutorials
- Richard Hawkes' "Creating Supershapes in Unity" tutorial - https://www.youtube.com/watch?v=2uOnB2FtB88
- Daniel Shiffman's Coding Challenge #26: 3D Supershapes - https://www.youtube.com/watch?v=akM4wMZIBWg
- Paul Bourke's Supershapes (Superformula) website - http://paulbourke.net/geometry/supershape
- Reza Ali's 3D Supershapes - http://www.syedrezaali.com/3d-supershapes
- Iridescent shader by SMKplus - https://github.com/smkplus/Iridescence
- Tutorial on changing object transparency in Unity - https://www.youtube.com/watch?v=hXUMdWXr2so
- Unity OSC - https://thomasfredericks.github.io/UnityOSC