A Beautiful Temptation
This is an interactive projective narrative live performance, talking about a soul that likes fantasy and falls into it, but reality often breaks in, then she is struggling, wandering and entangling between the fantasy and reality…
produced by: Jinyuan Li
Introduction
The performance is based on my own experience, interspersed with narrative speaking:
“I like fantasy, which is beautiful and free. Unlike reality that always makes me confused and helpless. I fall into the infinitely vast fantasy, but reality often breaks in. So, the fantasy disappeared, I want to catch it, I want to fall into it again, but the reality always wakes me up. Then, I am struggling, wandering and entangling between fantasy and reality.”
from peaceful and lively music to gradual enhancement, it ended abruptly at the climax.
I think that fantasy is a beautiful temptation, it can be a spiritual habitat for self alone hiding from the world, but also can be like Siren’s singing, making people unknowingly lose their minds and motivation to struggle in real life. Facing reality is inevitable. After all, we are walking through it, that’s why it often breaks in, fantasy disappears, the reason wakes me up. But the beautiful, free, romantic, unfettered fantasy is much more absorbing than the heavy and tired, plain and boring of real life. Catching back the fantasy and falling into it again is driven by my emotion. Struggling, wandering and entangling between fantasy and reality is the result, maybe there is no solution, I might always balance the relationship between them.
The work is my reflection and introspection on myself. I like to indulge in unrealistic fantasies to escape the regret, frustrations and difficulties in real life. Maybe some people will have the same experience with me, as people like infinite fantasies that are both true and illusion, impractical, and make people yearn for them. Struggling between fantasy and reality, temptation and restraint might be a state of existence that everyone has. Fantasy can be used as a spice for real life, but cannot replace life. In the temptation of fantasy, we must learn to restrain.
FULL VIDEO
Concept and background research
Kurokawa emphasized the symbiotic relationship between sound and image in his "syn_" work. It neither emphasized the interpretation of images by sound nor the concretization of sound by images, but treated the two as a simultaneous whole, the audience present. The audience does not need to think rationally, but directly feels intuitive through listening and seeing, and “immerses” into the context of the work.
Early in the period of modernism, artists such as Kandinsky, Chagall, Paul Klee, and Mondrian were fascinated by the concept of "synthesis", especially Kandinsky. In his "On the Spirit of Art", he puts the correspondence between image, color and music analyzed in detail, and how the two achieve "universal unity" with surging passion and romantic spirit. The synaesthesia and synesthesia that Kurokawa emphasized in his Audio-visual works are the contemporary version of Kandinsky's vision.
This is also what I want to show and express. How do we feel about our surroundings? Our experience of this world is not from a single sensory experience, but a mixture. The senses are integrated with each other, and we get a comprehensive experience. We experience our environments (the world) as a unified bundle of sensations from multiple sensory modalities. That’s how I composed my music and the visuals. Music followed the visuals, emotions, and the visuals, emotions interacted with music. Synesthesia is hearing colour and seeing sound.
As for performance, I am thinking about how to integrate with computer vision. What’s the difference between the interaction with a mouse and the interaction with body movements? At first, I thought the body's movement interacts with the screen simply. And what shows on screen, I used an animation brain to think of the patterns, which are very concrete, but hard to finish by codes, I got stuck on that. Then I found some works from Adrien M & Claire B. I got some inspiration, what is computer vision projection with performance and dance. The patterns on the screen are props. Although they are two-dimension, they are not static dead objects. They respond to every body’s movement. The dynamics of the human body and the screen are a process of mutual dialogue. Together, they elaborate and convey the information to be expressed to the audience. The patterns on the screen may be abstract or even simple geometric figures, but because they can interact with performers, it makes the audience have very concrete associations with these extremely abstract figures, and the scene is thus produced. I think that is what I am looking for and it’s the main point for this interaction.
PROCESS VIDEO
Technical
First part / Enjoying
After the tidal sound, I fall into fantasy, with soft and lively music mixed with sounds from the jungle and underwater singing, I sink deeper and deeper into fantasy. The fantasy is like Siren’s singing. Let me immerse myself in her illumination.
"I like fantasy, which is beautiful and free. Unlike reality that always makes me confused and helpless. ”
I used machine learning - styleGan2. I think its neural network is very familiar with our consciousness fluid. We imagine something based on our real world, but imagine maybe twist, polish, decorate the real into a similar but more perfect and ideal world. I chose thousands of multiple categories of photos from our real world: nature(ocean, followers, forests , etc.), universe (planet, star sky, etc.), different objects’ textures (like marble, pigment, light, etc.), kinds of creatures ( insects, transparent ocean creatures, fish, etc.), human moving silhouette, etc. I picked them following my aesthetic ( this may be subjective, but the whole project is mainly to express my inner emotion), put these different photos together, then trained them, the machine found their similarities, and made a connection between them. The generated images surprised me, it’s beautiful, a bit abstract, but can let me imagine what it is.
Second part / Losting
But reality often breaks in with the sudden drum and strings sounds, then fantasy shattered and disappeared.
“But reality often breaks in. So, the fantasy disappeared, I want to catch it, I want to fall into it again, but reality always wakes me up.”
The disappearing images are from styleGan2, which are 2-D images. I set a threshold of brightness, went through each pixel, and picked the eligible ones, then used a geometric transform produced into 3-D forms, consisting of lines and particles moving / diffusing in Brown Movement.
Third part / Struggling, Wandering, Entangling
I wake up, find myself underwater, I struggle, desperately go upstream. After the blue ocean scenario, I go back to reality, where I find another self is looking at me, it is like a shadow following me. I feel lost, I want to fall into the fantasy again, then the pink fantasy scenario appears but disappears quickly. Because reality always wakes me up, then I am struggling, wandering, entangling between fantasy and reality.
“I want to catch it, I want to fall into it again, but reality always wakes me up. Then, I am struggling, wandering and entangling between fantasy and reality. ”
This is the mainly interactive part, the C++ code runs at real time combined computer vision with Kinect X360Box. My body movements changed colours and shapes on the screen. There are marching squares algorithm and shader that I used to express the two-selves talking with each other and emotion.
I also recorded my movement with Kinect firstly, combined with shader and motion graphics (both of them made in C++ code), made two contradictory scenarios: I stand at the reality talking with another self at the fantasy and I stand at the fantasy talking with another self at the reality.
The music gradually became more and more enhanced, the shining green and blue colours interacted with me, the entangling lines appeared, very quick fantasy images flashing and shining black and white colours interactively showed and ended suddenly in a dark space with the music ending at the climax, leaving a silence.
Others
Setting up in G05 space.
Future development
This is my first try to do an immersive performance. Everything is fresh and new for me, I just want to see how far I can go, and it doesn't seem bad. Because it is my first time, there are still lots of things that could be polished and progressed. Like how to do an immersive sound surrounding? How to express my thoughts with more clear and better aesthetic visuals? How to let people become more immersive into the live show at that little tiny time? Focus what I really what to express is always the main point for every piece of work. I hope I can constantly review my own work and get further, challenging myself everyday.
Self evaluation
This is my first time trying to combine visual, sound and body movement together. I want to show a comprehensive immersive experience to the audiences, and from their words, looks like I succeed. I need to specially talking about the music piece, which I composed for the first time, although it is not computational ( I haven’t learnt Max MSP yet, but I will learn it next year), it is the second important thing in my work, as it lays the mood of the whole work. When I composed it, I followed the visuals, thought about the connection between colors and sounds. I just followed my intuition, ask some questions to myself, in those scenarios, what you will hear and see? The styleGAN images helped me do both the visuals and the music. As for the interactive piece, it needs more emotional expression, with the help of my movements, to express more clearly. I thought that there are two-selves talking, even arguing with each other, how to express that contradiction? Maybe use treble and bass. The character in fantasy is good at overcoming reality strength with softness, what kinds of instruction / sound is suitable? What kind of visuals can show my inner emotions? I found myself that the progress in music comprehension and the progress in visual expression of personal things. This piece of work is a start for me, I see the potential that I can develop, and realise one of my little dreams.
Thanks
Thanks for everyone who come to see my performance, I am so grateful!
Special Thanks
Thank Katie, Oliver, Robert, Rita for doing a critic group together and giving supports.
Thank Theo, Atau, Andy, Rachel for giving me feedbacks and offering supports.
Thank Rob, Chris, Pete for helping set up and offering supports. .
Thank Nathan, Yuting, Armando, Sunny, Karen, Kris, in G05 Group for helping set up and offering supports.
Thank Cheng, Hoyin, Hongye, Siyuan, Xi, Haotong for helping record shows.
Thank Caterina for helping.
Many thanks to my family!!
References
softwares:
openframework, MadMapper
Libraries:
ofxSyphon,
ofxKinect,
ofxOpenCV,
ofxCV
Artists:
https://junkiyoshi.com
http://www.ryoichikurokawa.com/
https://www.am-cb.net/
Books / Channels / links:
Patricio Gonzalez Vivo and Jen Lowe: The Book of Shaders.
Jamie Wong (2014) : Metaballs and Marching Squares.
Michael Hadley: Basics of Generating Meshes from an Image, ofBook.
Daniel Shiffman: Chapter 6, Autonomous Agents, the Nature of Code.
Daniel Shiffman: Chapter 4, Particles Systems, the Nature of Code.
Artificial Image channel on Youtube.
week 12 computer vision (part 1), Workshops In Creative Coding 1 (2020-21).