Traditional Chinese narrative in computational arts
Produced by: Shuai XU
1. Introduction
1.1 Background
Digital media art provides a new way to explore art, and therefore more and more people are paying attention to it. With the development of technology, digital technology is also continually present in daily life. Therefore, for a creator exploring how to combine computer technology and art, it is particularly important to find a good balance between computer technology and computer art.
1.1 Research topics
The main focus of this project is how to use computational arts to convey narratives of traditional Chinese literature. I pose three questions to encourage deep consideration of the role of computational technology:
1. How can computational arts enhance traditional Chinese narratives?
2. How can traditional Chinese narratives be conceptualized, visualized and integrated into computational arts?
3. How to use these studies and show them in your work?
1.3 Theories and Contexts
For my project, I want to choose traditional Chinese narrative to expand my research; the ancient elements of this literature can give me a bridge to create a dream space, which can employ computational technology to immerse people in the space, as well as allowing audiences to interact with the installation and be part of the stories. At the same time, I am looking for fusion of and contrast between Eastern and Western culture, coupled with the collision of ancient and modern art.
2. Traditional Chinese narrative
2.1. What is Traditional Chinese narrative?
According to Anthony C. Yu,(Chinese Literature: Essays, Articles, Reviews (CLEAR)
Vol. 10, No. 1/2 (Jul. 1988), pp. 1) "
In the study of traditional Chinese narrative it is customary to speak of the developmental dependence of prose fiction on history, or more precisely, on historiographical writing." Yu believes that traditional Chinese narrative is a process that as an exhaustive and complex sociological report and defines the key characteristics of Traditional Chinese narrative as analysis, structure and composition. Chinese narrative is one source of information about China, which includes valuable history and culture and also describes the identity and values of citizens and the ways they conceptualize reality.
2.2 A new viewpoint of Traditional Chinese narrative
As Olga Leontovich claims in his article (The world of Chinese fictional narratives: content, characters, and social impact, (06 November 2015), for traditional narratives, structural narrative analysis is mostly used, with special attention paid to narrative mechanisms, recurring factors, themes, and text patterns. The combination of these patterns affects the character of the entire narrative. Data can be analyzed from three perspectives: time, space and society. Thus, at the same time the reader (or listener), the characters, the sequence of events and the causal relationship between them, the ending and the narrator's attitude towards being mentioned are essential narrative elements. These things are identified and explored as elements that form a complete narrative framework.
2.3 Elements of Traditional Chinese narrative
When we have a good understanding of the narrative itself, then we can extract elements from different types of Chinese historical records.
The first is mythological literature; for example, Shan Hai Jing (The Classic of Mountains and Seas) uses different kinds of texts and images to record a strong narrative of the relationship between the physical world and the mythical world. Many of the myths and legends in Shan Hai Jing are regarded as essential material for religious studies. These myths reveal the activities of the elves and at the same time present the beliefs and ways of worship of ancient peoples.
The second type records traditional Chinese philosophies such as Daoism, Confucianism and Buddhism. These philosophies were the spiritual pillars of traditional society and have continued to this day, having significant influence on contemporary society. Their spirits have left many records in ancient China, including sculpture, painting, poetry, temples and even dramas recording the daily life of traditional China.
The third type is the traditional Chinese poetry of 5,000 years, whether written by professional poets or created by civilians; it is an essential part of Chinese literature. This poetry has many forms, including the well-known Tang dynasty poetry, Song dynasty poetry, Yuan dynasty opera and Qing dynasty opera.
Next, we have traditional Chinese opera, a form of musical record. The roots go back to the early period in China and gradually advance over more than a thousand years, maturing in the Song dynasty (960–1279).
Opera in China includes many elements, such as music, song and dance, martial arts, acrobatics, costume, and make-up art; these constitute an essential part of Chinese literature.
2.4 Immersive Traditional Chinese narrative space
Barrett explains the term “immersive” as performance that allows the public to exist within the performance area and in which the audience should influence the performance (Figure 1). It is the empowerment of the audience in the sense that they are put at the center of the action; they are the pivot on which everything else spins. It is the creation of parallel theatrical universes within which audiences forget that they are an audience, and thus their status within the work shifts” (Machon, 2013).
In an immersive environment, it is important for the audience to be allowed to participate in the elements instead of just experiencing them. These immersive elements include sounds, lighting, physical experience, computational technology, and space, all of which can affect audience experience and perception (Figure 2).
Figure1: Light, Space, Rhythm – A conversation with Florence To [WWW Document], n.d. URL https://www.creativeapplications.net/environment/light-space-rhythm-a-conversation-with-florence-to/ (accessed 4.20.19).
Figure2: Feng, Q.（22 September 2017）. Inner Awareness. Interactive Architecture Lab.
3. Computational Arts and technology
3.1 What technology can be used in classic performance？
Nowadays, there has a captivating and conceivably relationship between performance and interactive computational design. Therefore increasingly designers are chosen interactive elements in their artworks; also pay attention to the status quo of culture. With the wide-ranging availability of computational technology in theatrical design and performance, increasingly people are paying attention to them. The audience also has stood out from the past performance forms.
In the interactive model, every action is a response to the previous action, as a result of which each audience experiences a series of unpredictable and unique effects; in computational arts, this effect is also known as generative art. This effect can be combined with a host of specialized instruments, such as 2D and 3D projection mapping, robotics or a variety of other mechanisms and tracking systems, which, in combination with machine learning, make tools more flexible.
Because of this diversity of instruments and the differences between them, they must be integrated into the immersive environment in the most effective manner with full consideration of their respective capabilities and usage environments (Figures 3 and 4).
An understanding of computational technology and how to use it is imperative in order to increase the interaction with the audience. This project brings together various technological forms, which can help produce a more interactive and immersive performance (Figure 5).
Figure3: Dökk （03 January 2017）– Live media performance by @fuse_works → [WWW Document], n.d. URL https://www.creativeapplications.net/environment/dokk-live-media-performance-by-fuse/ (accessed 4.21.19).
Figure4: Dökk （03 January 2017）– Live media performance by @fuse_works → [WWW Document], n.d. URL
Figure5: Dökk （03 January 2017）– Live media performance by @fuse_works → [WWW Document], n.d. URL https://www.creativeapplications.net/environment/dokk-live-media-performance-by-fuse/ (accessed 4.21.19).
3.2 Spaces and Embodiment
Space: With the development of computational technology, the number of artists using these technologies has increased, which in turn has contributed to the promotion of live performances. Because of this, the role of space, whether physical or virtual, has gradually become a significant aspect of live performance art. The combination of live performance and virtual performance combines a large number of digital technologies, which are employed through different performance styles; it also new technologies, forms, and spaces creating and reinforcing the interaction between individuals in the region and their respective environments.
These processes shift the focus of art from pure literature to performance and to the combination of embodiment with technological and production elements. In the interactive virtual reality installation “Osmose” (1994-1996) (Figure 6) Char Davies claimed her audience were "immersants” (Figure 7). This Installation, however, consisted solely of sight and sound and did not include touch, taste, and smell. According to Merleau-Ponty (1962), these latter senses engage the perceiver; the body is the medium that perceives the connected world, and Deleuze and Guattari’s (1980) notion of “haptic space”, which negative antithesis between the senses (Nelson).
Embodiment: Two propositions are used to illustrate the process of implementation to prove that embodiment is a vital issue in the field of computational digital performance. On a theoretical level, through (Platonic) dialectics, we tend to consider that there is a divergence in the "survival" condition before the principle of digital simulation, especially in material reality. In postmodernism, Freud believes that there is an incredible double standard in the existence of the world, or a simulacrum (Baudrillard); in the 1980s and 1990s, where academic and artistic backgrounds are popular, as a hyperreal copy of the complicated an existing relationship with the original.
Figure6: Bill Seaman and Otto Rössler (2011). Neosentience - The Benevolence Engine [WWW Document], n.d. URL http://www.immersence.com/publications/2011/2011_BSeaman_ORoessler.html (accessed 4.22.19).
Figure7: Bernier, Robert (1995). Char Davies Immersive Space Perceptive Space [WWW Document], n.d. URL http://www.immersence.com/publications/1995/1995-RBernier-en.html (accessed 4.22.19).
The virtual world and the real world have a dialectical relationship in the digital ear serves. In this case, we are no more extended discovery ourselves to deal with the dichotomy of embodied/disembodied, real/virtual. According to “Postscript on the societies of control” (Deleuze), humans are no longer enclosed in traditional communication spaces. The scope of digital performance that embodies self is expanded, such as in Marshall McLuhan’s well-known chiasmic reversal of media through the extension of human prosthesis; this expansion also includes motion capture: for example, Jeremiah uses the computer to generate a hand (Figures 8, 9 and 10), and an embodied avatar is developed using surveillance technology (Broadhurst, 2007).
Figure8, Figure9, Figure10: Jeremiah - The face of Computer Vision [WWW Document], n.d. URL http://personal.ee.surrey.ac.uk/Personal/R.Bowden/jeremiah/jeremiah.html (accessed 4.22.19).
4. Space Design Research
4.1 Ideas in White Moonrise (from the Book of Songs)
The Book of Songs has played a role in spheres far beyond literature and had a lasting influence on Chinese civilization, in the same way that Homer's epics took hold in the West (Kern, 2018).
The Book of Songs is the oldest Chinese poetry still in existence. It is mentioned by Confucius (551–479 BC) in his Analects. It is one of the "Five Classics" and has been memorized and studied by Chinese scholars for over two millennia. Chengyu (four-character classical idioms) from the Book of Songs still form an essential part of the discourse and everyday language of contemporary Chinese. Confucian education seeks to teach moral subtleties through music, rites, and poetry (Figure 11).
The poetry in the Book of Songs is in two parts: the Air of the States, which uses concise lyrics and simple language and is largely folk songs and hymns speaking of what love and courtship are and the responsibility of soldiers, and telling people how to satirize politics and protest.
With regard to White Moonrise, Mao Shi Xu thinks that it is a satire of a ruling king, Lust; the Chinese philosopher Zhu Xi's (Figure 12) Poetry Collection claims that "these words tell of a man falling in love with a woman." Gao Heng's Explanation of the Book of Songs describes the story of the king of Chen killing a handsome man for a beautiful girl. Nowadays, we think it is a love poem of Acacia.
The poem tells of a man watching the moon and thinking of the beauty of a girl,
of her face, her figure and her posture; the more he thinks, the more he worries, and the more he worries, the more he thinks. With the deep acacia and the beauty of the moonlight, this constitutes a moving scene and becomes a poetic painting (Figure 13). This is an English translation of White Moonrise (Barnstone):
White Moonrise
The white rising moon
is your bright beauty
binding me in spells
till my heart's devoured.
The light moon soars
resplendent like my lady,
binding me in light chains
till my heart's devoured.
Moon in white glory,
you are the beautiful one
who delicately wounds me
till my heart's devoured.
Translated by Tony Barnstone
Figure11: Emperor Zhou the Dragon Slayer, as shown in a wall painting at Yixing Temple in Jiangsu Province (Credit: Alamy)
Figure12: Zhu Xi – Chinese philosopher, politician, and writer of the Song dynasty – proposed looking at the words of the songs themselves (Credit: Alamy)
Figure13: Illustrated Edition of the Book of Odes Handwritten by the Qianlong Emperor, Qing Dynasty 《御筆詩經圖》 清 高宗御筆寫本
4.2 Image Generating in Computer Programming
In ancient Chinese narrative, landscapes usually contain small figures of people, which concordantly assimilate into the vast world around them. People and nature interact with and complement each other and therefore attain a condition of balance and harmony.
There is a spiritual element to White Moonrise, demonstrating the feeling of missing the moon. First, the poem describes a beautiful girl under the moonlight, the moonlight far off and the moonlight close by on the face of the girl. Next, with the movement of the moonlight, the girl's graceful gesture is revealed. Finally, the moonlight is used to express the love of the heart and the end of the feeling of self-satisfaction – a total of four parts, with four scenes as the main narrative events (Figure 14).
Computational programming provides opportunities to create and generate visualizations, which will change the way that giving as many visual results as possible to the computer rather than making a video according to people's requirements and scripts. Now an important aspect is a visualization of movement will be generative. (Figures 15, 16 and 17).
Figure14: Light move made by Shadertoy
Figure15, Figure16, Figure17: Light Moon made by Shadertoy
4.3 Physical Installation design
The design of the space is based on the storyline and the classical Chinese garden environment, using the structure of the garden to create an abstract space device while using multiple projectors to construct an immersive environment. Throughout the entire presentation, the projected digital image interacts with the performer.
Throughout the space, the intention is to use multiple application techniques, such as multilayered screen projection, Kinect motion tracking, machine learning, projection mapping, computer interaction and so on, to realize the interaction between representation and environment. The audience can find resonance with the performance, immersing themselves in the fantastic scenery and interacting with it (Figures 18 and 19).
Figure18,Figure19: 3Dsketch by C4D
5. Conclusion
5. 1 Conclusion and Further Studies
This project, "Traditional Chinese narrative in computational arts," aims to find a different way of conveying traditional narratives, motivated by the computational arts. Through the use of computational technology like 3D projection mapping, body sensor tracking and machine learning, I have tried to redefine the relationship between traditional narrative and computational arts by creating an interactive and immersive show.
This thesis consists of three parts. The first part focuses on a new viewpoint of traditional Chinese narrative while aiming to explain the relationship between traditional elements and framework perspective.
The second part focuses on computational arts and technology, through an understanding of what spaces and embodiment are, and ultimately their combination in mixed-computational environments with embodied and improvisational self-expression.
The last part involves ideas for the installation, the context of which is the new form of computational arts alongside a re-imagining of the traditional Chinese narrative White Moon. To accomplish this, an immersive environment was designed by computational programming.
In the future, I would be willing to write a more specific guide to computational theory and another article. This time, I only have time to complete a demo of this story. I would wish to spend more time on this project, combining computational theory and computational technology to put my thoughts into effect and examining these concepts more critically in practice.
Reference:
Anthony C. Y, (Jul. 1988). History, Fiction and the Reading of Chinese Narrative. Chinese Literature: Essays, Articles, Reviews (CLEAR) Vol. 10, No. 1/2, pp. 1-19
Leontovich, O. (06 November 2015). The world of Chinese fictional narratives: content, characters and social impact. Int. Commun. Chin. Cult 2, pp.301–317.
Machon, J. (2013). Immersive Theatres: Intimacy and Immediacy in Contemporary Performance. Macmillan International Higher Education.
Light, Space, Rhythm – A conversation with Florence To [WWW Document], n.d. URL https://www.creativeapplications.net/environment/light-space-rhythm-a-conversation-with-florence-to/ (accessed 4.20.19).
Dökk. (03 January 2017). – Live media performance by @fuse_works → [WWW Document], n.d. URL https://www.creativeapplications.net/environment/dokk-live-media-performance-by-fuse/ (accessed 4.21.19).
Bolewski. (n.d) – template.pdf [WWW Document], n.d. URL https://dspace.lboro.ac.uk/dspace-jspui/bitstream/2134/21689/3/Bolewski-template%282%29.pdf (accessed 4.21.19).
Nelson, R. Lavender, A., Bay-Cheng, S., Kattenbelt, C. (2010). Mapping Intermediality in Performance. Amsterdam University Press.
Bill Seaman and Otto Rössler (2011). Neosentience - The Benevolence Engine [WWW Document], n.d. URL http://www.immersence.com/publications/2011/2011_BSeaman_ORoessler.html (accessed 4.22.19).
Bernier, Robert. (1995). Char Davies Immersive Space Perceptive Space [WWW Document], n.d. URL http://www.immersence.com/publications/1995/1995-RBernier-en.html (accessed 4.22.19).
Jeremiah. (n.d) - The face of Computer Vision [WWW Document], n.d. URL http://personal.ee.surrey.ac.uk/Personal/R.Bowden/jeremiah/jeremiah.html (accessed 4.22.19).
M, C., 2015. White Moonrise (From “Book of Songs” — China’s most ancient poetry anthology). Medium.
Kern, M., n.d. The Book of Songs: Poems that helped shape Chinese thought [WWW Document]. URL http://www.bbc.com/culture/story/20180525-the-book-of-songs-poems-that-helped-shape-chinese-thought (accessed 4.27.19).
Excerpt from The Anchor Book of Chinese Poetry [WWW Document], n.d. . Penguin Random House Canada. URL https://www.penguinrandomhouse.ca/books/8873/the-anchor-book-of-chinese-poetry-by-edited-by-tony-barnstone-and-chou-ping/9780385721981/excerpt (accessed 4.27.19).
National Palace Museum [WWW Document], n.d. URL https://www.npm.gov.tw/en/ (accessed 4.27.19).
Computational Arts-Based Research and Theory (2018-19, Term 2)