Voice Visualisation
Voice Visualisation is a visual interpretation of a piece of music I composed for “Shiloh", a film I scored in 2017. I wanted to capture the music's textured, meditative and ethereal feel using dynamic polygons. It can be read as an abstract visual score. I was inspired by the diverse ways through which music can be portrayed, interpreted and communicated sensorially.
Produced by: Freya Berkhout
I drew artistic inspiration from graphical scores, which are often striking yet hold great and intricate detail, as exemplified by György Ligeti, Krzysztof Penderecki, Peter Sculthorpe and Björk’s Biophilia app. These works present one of many ways through which sound can be captured visually, and offer the viewer an alternate emotional experience outside of, or in addition to, an aural context. Visually, I was inspired by minimalist and geometric art by Manfred Mohr, Carmen Herrera, Nasreen Mohamedi and Wassily Kandinsky.
Detailed technical guidance came from Manuel Kretzer’s Generative Design Tutorial on Sound Mapping. Initially, I introduced the Minim library, which I discovered can only run monophonic sound files, and began to map the song to the space. The algorithm tracks the frequency and amplitude of the music, creating filled and unfilled polygons and plots them across the canvas in real time. As the amplitude rises, the size of the polygons and the number of vertices on the polygons increases. If the frequency and amplitude peak simultaneously, a faint polygon encapsulated in an ellipse will be drawn. I was originally working in portrait aspect ratio, and it took me a lot of editing to re-create the desired effect in landscape, which ended up looking pleasingly spacious.
The work could be simply expanded to create different shapes and colours in real time, potentially as a series. It would be interesting to try to capture specific frequencies, for example the sound of the human voice, and create bespoke visual imagery for that element of the music as well as utilising the amplitude and frequency. I would also be interested in potentially mapping the song in a more random and non-linear way to create unexpected, more chaotic works.
Aesthetically, I wanted the work to look minimal and bold from afar, yet more detailed when examined close up. The faint polygons and ellipses provide a sense of depth when you are close to the piece, which is a relatively apt visual reference to the subtle movement and undercurrents in the composition.
References
1. Manuel Kretzer’s Generative Design Tutorial on Sound Mapping (https://www.cs.buap.mx/~daniel.valdes/docs/tutorial-06_processing-soundmapping2.pdf).
2. György Ligeti, Krzysztof Penderecki, Peter Sculthorpe and Björk’s Biophilia app.
3. Manfred Mohr, Carmen Herrera, Nasreen Mohamedi and Wassily Kandinsky.
4. IS71016B Programming for Artists 1 (Lior, 2017 - 2018) (https://learn.gold.ac.uk/course/view.php?id=8957).
