Spatial
“Spatial” is an interactive audio-visual performance that lies in the intersection between sound and space, and explores the bidirectional dynamic between these two forces. The performance makes use of 12 speakers, strategically dispersed around the space, which come together to form an immersive surround sound system. The physical space, with its geometric visual compositions projected onto three walls, in turn further distorts their perception. With its playful mind tricks and sensory manipulations, “Spatial” invites its audience on a journey that takes place entirely within the confines of four walls.
produced by: Yu Hsuan Liao
Full perfomance video: https://youtu.be/nrKADbw9Xus
Concept
In a multilayered spatial composition, not only does the sound move around the listener at a constant distance, but it can also move as far away as we can imagine and also come extremely close.
— Stockhausen Klavierstück
The notion of space itself, as it is used in music – as well as in many other fields – has many senses; as the French writer Georges Perec would say, there are »spaces of space«. There is, for instance, an opposition between physical space and metaphorical space (or space as a tool for representation);there is also an opposition between virtual and real space; space can be used as »immersion« or, on the contrary, for constructing localizations;and so on.
Listeners don’t have eyes in the backs or tops of their heads, they tend to rely on vision more to perceive the scene in front of them, and on sound more to deal with things behind and above them. In some cases the blending of cues and the diffuseness of sources leads to a general perception of space or ‘spaciousness’ without a strong notion of direction or ‘locatedness’.
This project sees music as architecture and space as melody, morphing the physical space into different shapes and texture. Explore how two sensations trick human brain perception to believe into.
Technical
On the interaction: Assigning two speakers (top and bottom) as one channel, the sound comes from different directions in each performance. Constructing a space by different matrix of visual and audio. For example, sound can come as the opposite side, above , below of the visual. There are so many possibilities with three walls showing visuals and 12 speakers. Each performance is different based on the movement positions of the audience. No seats are provided on the site, audiences are allowed to stand, walk, sit, or even lay down in the space.
Visual Technical: I made the work mainly in tounchdesigner, constructing a virtual architecture in it by using geometry instancing in main file. And instance audio information(amplitude/ sound effects filtering) as scale/ rot/ pivot parameters. The visuals are pre-rendered in different touchdesigner files as video. And I apply it as textures on the geometrical shapes/ lines. At last, I edited all of my visual content in premiere pro as 5760*1200 movie file.
multi-channel compose: I composed the audio in ableton, by using E4L envelope to locate the sound binaural in virtual space with headphone. For the performance, I used max msp to assign the channels to the speakers in g05. And I used midi- controller to manipluate audio panning, reverb, delay, echo in live.
Audio-visual sync: Initially I planned to do both visual and audio live, yet limited by the GPU, the frame rate become extremely slow when I run touchdesigner and ableton, manipulated by midi controller at the same time. So instead, I imported in the audio I composed into touchdesigner and use filter/ envelope/ resample/ math/ range to filter out different sound effects. And map the parameter to scale/ pivot/ rotation of xyz coordinates.
When conducting live performance in G05, I imported the pre-rendered visual in Madmapper. By using max patch to send OSC from ableton on my laptop to trigger the play button in Madmapper on the pc that make both sync without latency.
Generating the Visuals
Conceptual compositions came naturally through the process of manipulating noises and randomizing geometry. The bigger challenge has been finding ways to generate shapes and lines that align with the texture of audio. I began by experimenting with methods of creating silhouettes. I’ve found some success by manipulating 2D and 3D primitives with fine tuned noises. Once a pattern is found, the previously mentioned process of adding detail is applied to reinforce the illusion of a representational object.
In this project, I play with noise a lot to create the organic textures of visual. A lot of audio visual reacts corresponding to beats and amplitude, aside from making it audio visual, I want to create a more organic touch into the visual.
How Colors Change the Perception of Interior Spaces
Interior space study points out there are alter important variables like artificial lighting, natural light, proportions, or materials - with all such elements influencing the experience that occupants will have in a space. Having three walls as physical space in my peroformance, I create visuals that based on this study to create space that is enlarged, compacted, lowered, stretched, widened, narrowed, shortened or highlighted by playing with colors, portions, compositions and textures.
Audio setup
Self-evaluation and Future Development
In this project, I construct simple geometrical architecture, yet there can be more element can be added into this construction. For example, the gravity of objects, the construction of the architecture, the space is endless for compose. The first experiment in physical space confirm my techniques and work flow I have been developing that is reusable in the future. They have also contributed to my style in ways that I was not expecting.
The biggest challenge I encountered in this project is finding the perfect balance for best experienece of audio and visual. As an immersive audio visual performance, consistency is an extremely important element to make people feel immersive throughout the experience. The speakers in G05 are fixed so I have to constantly adjust the scale and position of my visual back and forth to find the sweet spot of best experience of audio panning/ reverb/ delay effect. Yet at the same time enable audience to move freely without being constrained to a specific position. With this experience, I found physical space condition have a huge impact on the digital work and how it influence my pipeline and workflow.
For future development, different physical space is also an element that I would love to experiment. The texture, shape, dimension of the space will completely change the effect of audio and visual. After my first experiment of projecting on three walls, I would love to expand this project by experimenting with different physical conditions and seeing how virtual space meets physical space.
Deeply influenced by Greek-French composer, music theorist Iannis Xenakis, math has been an topic that I planned to explore yet ran out of time to apply on my audio composition for this project. Fibonacci numbers have been found in many fields, for instance, music, architecture, nature, everywhere! I would love to extend my dissertation on how math playing as a hidden correlation factor in different fields, especially in music compositions and space. Composing music in mathematical precision and mathematical aesthetics to see how it works in physical space and explore the hidden secret behind the harmony of two forces.
References
Taruskin, Richard. Retrieved April 7,2018. "Chapter 2 "Indeterminacy."". Oxford History of Western Music. New York, USA: Oxford University Press.
David Gutkin. 1959. Drastic or Plastic?: Threads from Karlheinz Stockhausen's “Musik und Graphik,”
Makis Solomos. 2015. The Complexity of Xenakis’s Notion of Space
Britannica. Archivedfrom the original on 6 May 2008. Retrieved 28 April 2008."Space – Physics and Metaphysics". Encyclopædia
Jonathan Cole. Friday, 22 June 2007. Music and Architecture: Confronting the Boundaries between Space and Sound Transcript.
Rob Wannamaker. January 2012. Mathematics and Design in the Music of Iannis Xenakis
Jimmy Stamp. JUNE 5, 2013. 5 1/2 Examples of Experimental Music Notation https://www.smithsonianmag.com/arts-culture/5-12-examples-of-experimental-music-notation-92223646/
How Colors Change the Perception of Interior Spaces
https://www.archdaily.com/935067/how-colors-change-the-perception-of-interior-spaces?utm_campaign=trueAnthem%3A%20Trending%20Content&utm_medium=trueAnthem&utm_source=facebook&fbclid=IwAR1otMF0eklh9m3717xaaOGqPz25sXlfA1t1O0OWDbXe_E4pHaGJ0a0CEhs