Augmented Creativity
This work experiments a co-creation system of art making by man, machine and nature, expressing an entangled relationship between them. By comparing the contributions of different participants, the unique ability of human beings in creative creation is further verified and understood.
produced by: Yanyi Lu
Introduction
This work explores a new process of creativity generation under the guide of Jordanous’s Four PPPPerspectives (2016) and speculates intertwined relationships among multi-contributors in computational creativity. This work can also be seen as an experimental multispecies storytelling on creativity.
This experiment collected the images from OpenProcessing community as training samples and fed them into styleGAN (a novel generative adversarial network (GAN) introduced by Nvidia) to generate many images. Then these images are postprocessed as environment-driven interactive moving images by optical flow algorithm.
In this computational system, it is speculated that humans are inspiring themselves and that all other nonegos are used as bridges and catalysts in a closed loop, to some extent. I hope this work can motivate audiences to think about the definition of creativity and reflect on human’s unique ability on creation by comparing with machine and nature’s creative abilities.
The final outputs of this project are web page (Augmented Creativity Gallery https://augmentedcreativitygallery.wordpress.com/) and dual-screen video installation.
Concept and background research
1. Literature Review
Creativity can be classified as big creativity (big C) and small creativity (small C) based on the level of breakthrough; creativity can also be defined as personal creativity (p-C), historical creativity (h-C) and social creativity (s-C) (Gómez de Silva Garza, 2019). Creativity is made up with a degree of freedom and constraints at the same time, which means it can be evaluated by two important criteria, novelty and value (Al-Rifaie, 2016). It is said by López de Mántaras Badia (2013) that creativity is a historial-cultural scheme and result of cultural inheritance and experience summary. Although creativity sounds more perceptual rather than rational, it is true that creativity can be seen as a logic form of problem solving involves cognition science so that it may be possible to repeat by machines or programs. According to the definition given by Varshney et al. (2013), a valuable creative idea consists of generative step and selective step, which means creativity is a coexist state of generation and assessment. On the other hand, creativity is a process of massive exploration and breakthrough of familiar conceptual spaces (Duch, 2006). Therefore, computational creativity can be thought of as consisting of transformation and exploration operators, convergence and divergence processes, a balanced and evolutionary algorithm (Gómez de Silva Garza, 2019).
2. Concept and Process
There are four perspectives on computational creativity from Jordanous (2016). He raised that a work should consider the creativity of person (producer), process, product and press (environment). This theory is the foundation of my artistic practice of Augmented Creativity.
2.1 Person (producer)
It was once generally accepted that creativity is a uniquely human ability. As described by Guckelsberger et al. (2017), creativity exists in any living being, not just human being. More broadly, many of us are also talking about the creativity ability of machines nowadays. With the development of technology, people raise some questions about the creativity of machine in the field of art. Can machines also be creative and produce new artistic ideas that have not been discovered by human? I will include human, machine and natural objects as contributors in this practice.
2.2 Process
The generative process is divided into three steps and the relationships between contributors are changing all the time. I chose images from OpenProcessing community as training samples because at this stage people treat machine as a programming tool to create artistic graphics. Almost all the samples I selected were human artists who used algorithms to imitate some natural substances or get inspiration from natural phenomenon, such as the flow of water and the breaking of particles. It looks like about 90% creativity is from human and about 9% is from nature, while machine just contributes less than 1% creativity to assist human in realizing their ideas.
However, at the second step, with the intervention of machine learning (machine creates new artistic ideas with the feed of works from human), the output of machine seems to be out of control. In the experiment, I am surprised to see some amazing images and it reminds me the contribution to creativity from machine can never be just 1%. The relationship between human and machine changes from slave and slaver to friend and collaborator with the emerging of machine learning. The third step is about the impact of environment in creativity, which will be described in next section, Press.
2.3 Press (environment)
Press is an important but often overlooked factor. An artwork's meaning is sometimes given by its context, worthless in one context but a valuable idea in another. For example, the work will inspire the audience and the audience will have different interpretations of the same work due to their different experience and background. This is a bidirectional interactive process (Jordanous, 2016). In this step, I simulated the process of interpretation by using the optical flow method to realize the interaction between the audience and the generated images. Furthermore, I also included other contributors such as natural life and machines for participatory creation. I took videos such as waves moving and projected them onto the images. Some projection attempts are successful, but some are not. These successes and failures also indicate the specificity and importance of press in creativity.
2.4 Product
Product means the outcome of a creative idea. For this practice, I hope to use a multi-species storytelling video to give the audience some thinking of creativity.
3. Reflective Thinking
As said by Boden (2007), both the failures and successes of ideas raised by computers prompt our human to have more clear insight about our own unique creative powers. This art practice implies some reflections on the role (also weakness & strengths) of each contributor in process of creation.
3.1 Human: the thinking mode of human is modularity and the ideas are used to be development gradually based on the experience and knowledge (Varshney et al., 2013). The role of human is like a leader in this system. It is hard for machine to assess an idea and make a decision, but human doesn’t.
3.2 Machine: as opposed to the way humans think, the thinking mode of machine is bouncing (Varshney et al., 2013). It will list enormous possibilities and test them all. The generation of ideas is discontinuous and much more random. Therefore, the strength of machine is to explore the unknown and help jump out the box. That’s why I call this work as Augmented Creativity.
3.3 Other natural elements: many other collaborators or contributors can be grouped in this category, for example, animal, plant, natural phenomenon. Human are a part of ecosphere. In the long history of evolution, we learn from nature and try to imitate it. We feel intimate with nature and seek for aesthetic ideas in it. The ecosphere is like an invisible teacher, providing one of the greatest libraries of inspiration and context for your work. Our sense of beauty emerges from nature and returns to it again now.
All in all, the relationships between them three in creation system seems to be a closed dynamic loop. Can we also speculate that humans are always inspiring themselves, and that machines created by humans are only bridges and catalysts in this closed loop? We use the technology as a tool, not as a purpose. Technology is a great tool for augmenting creativity but not for generating countless fake creativity without context.
Technical
The first step is training styleGAN model in runway. The samples are downloaded from openprocessing community and cut to 1:1 size.
The second step is about mapping the movement in video to the static generative picture by optical flow in openframeworks with addon ofxopenCV. The color of line is defined by the color of pixel in picture and the length of the line is decided by the degree of change within two buffers.
Future development
This research project is still on the conceptual stage, raising a possible model for computational creativity. The next step is exploring the application scenario and making it as a real valuable system to assist artists or designers’ creation by the power of multi-contributors. I will continue this research and present it in future exhibition or conference.
Self evaluation
This was the first time for me to get in touch with the field of computational creativity. At the beginning of the project, I did a lot of literature reading and found some worthy points which has not been studied as my research topic. For me, I took thesis project more like a practice-based academic research than making new media artwork. Due to the lack of equipment/space at home and the form of online exhibition, I chose to display my current research outcome through a web page (Augmented Creativity Gallery) and dual-screen video installation. The final display is not as perfect as initial plan before Covid, but I will continue this research work based on the feedback and then conduct a field exhibition in the future.
References
Research Reference:
Jordanous, A. (2016). Four PPPPerspectives on computational creativity in theory and in practice. Connection Science: Computational Creativity, 28(2), 194-216.
Gómez de Silva Garza, A. (2019). An introduction to and comparison of computational creativity and design computing. Artificial Intelligence Review, 51(1), 61-76.
Al-Rifaie, Mohammad Majid, Cropley, A, Cropley, D, & Bishop, Mark. (2016). On Evil and Computational Creativity. Connection Science, 28(2), pp. 171-193. ISSN 0954-0091
López de Mántaras Badia, R. (2013). "Computational Creativity". Arbor, 189 (764): a082. doi: http://dx.doi.org/10.3989/arbor.2013.764n6005
Jordanous, A. (2016). Four PPPPerspectives on computational creativity in theory and in practice. Connection Science: Computational Creativity, 28(2), 194-216.
Duch, W. (2006). Computational Creativity. 2006 IEEE International Joint Conference on Neural Network Proceedings, 435-442.
Guckelsberger, C., Salge, C., & Colton, S. (2017). 'Addressing the "Why?" in Computational Creativity: A Non-Anthropocentric, Minimal Model of Intentional Creative Agency'. In: Proc. 8th Int. Conf. Computational Creativity, 2017. Atlanta, United States.
Boden, M. (2007). Creativity in a nutshell. Think, 5(15), 83-96.
Varshney, Lav R., Pinel, Florian, Varshney, Kush R., Bhattacharjya, Debarun, Schoergendorfer, Angela, & Chee, Yi-Min. (2013). A Big Data Approach to Computational Creativity. arXiv:1311.1213
Code Reference:
Workshops in creativie coding (week13), optical flow example by Theodoros Papatheodorou