Evolve and Mutate
Exploring the use and possibilities of generative systems to create visual artworks, this piece uses genetic algorithms, which are left to evolve and mutate autonomously in an attempt to discover new visual artefacts.
produced by: Jakob Glock
Introduction
In this project I am exploring how mathematical functions can be used to plot points on a two-dimensional plane, when stringing these functions together in various combinations, different forms and structures begin to arise. Through countless hours of experimentation and live coding sessions it became apparent to me that the scope of possible outcomes within this system was vast and started to devise a way in which I could automate the process of generating these images, allowing me to navigate the space in a more intelligent and efficient manner. The route I chose to explore and navigate this space was genetic algorithms, which borrow ideas from evolution such as selection, crossover and mutation. Genetic algorithms allow me to rank, mate and evolve images, for the ranking procedure I decided to use my own aesthetics, which I defined within a fitness function using a set of simple image analysis methods. Once I had a system which would run consistently and reliably, I started to execute and leave my system running for days on end generating, ranking, mating, evolving and saving my images, slowly building up an extensive catalogue. During this time I was also able to fine tune and experiment with the settings of the system, varying the mutation rate and weightings within my fitness function, and seeing how these would change and influence the output of my system and the images it created from population to population.
Concept and background research
In this project I was interested in exploring how aesthetically pleasing images could be created using combinations of mathematical functions and how this idea could be extended into a machine which would be able to act autonomously and have the ability to explore and search out the best possible images. Much of my inspiration into the creation of computational machines stems from my undergraduate study, where I became obsessed with the generative music of artists such as Brian Eno, Autechre and the music evolution project Darwin Tunes. Visually I have been developing my aesthetics over the past year and have become more and more interested with high performance graphics programming and have been delving into the shader scene and websites such as ShaderToy where I have discovered visual artists such as Inigo Quilez who have been a huge source of inspiration both aesthetically and technically. Another major source of inspiration within this project was the work by Scott Draves and his fractal flames, whilst researching online I came across a paper which explained the algorithm for creating his fractal flames, but more interestingly at the bottom of the paper I found a series of mathematical functions which I decided to experiment with and use within my system, Draves also explored genetic algorithms and evolution with the project Electric Sheep, which utilises multiple computers across the internet to allow users to vote and influence the evolution process. In my approach I was interested in using my own aesthetics to evolve my system, staying away from external inputs from users and attempting to enclose my own judgement process into a single package. Further to this I was also more interested in using a genetic algorithm to see the evolution taking place over a period of time and wanted to stay away from the more traditional idea of using it solve and make available to me the perfect solution to a given problem.
Technical
My main technical challenges:
Language and Shaders
Within this project I have decided to used OpenFrameworks and the programming language GLSL (OpenGL Shading Language), by using GLSL I am able to write and execute small programs directly on the GPU (Graphics Processing Unit) and process large amounts of data in parallel, much faster than if I was to use the CPU (Central Processing Unit). In my program I am mainly utilising fragment shaders, these allow me to access either every pixel on the screen or the pixels within a two-dimensional texture which I have uploaded to the GPU, I am then able to process the data stored within and output this to the screen or a set of points. When processing data you can manipulate multiple values at once, this can be achieved by storing various values in the different channels of the texture, this allows me to store and process both the X and Y components of my points. Within my GLSL program I have defined a number of functions which I am combining in various ways to plot my points on the screen and draw my image, every new image has a unique combination of functions and a new program needs to be written and uploaded to the GPU to run and display it. Thankfully, when you are running GLSL within OpenFrameworks there is no need to recompile the whole application and the GLSL program can be loaded straight onto the GPU from within the application. The ability to load these programs dynamically helped me to automate my process using genetic algorithms, allowing my OpenFrameworks application to rewrite my GLSL code for every new image, enabling it to combine, mutate and evolve individuals from the previous population to create a new child GLSL program, never having to recompile my main application.
Fitness Function
When implementing my genetic algorithm I had to devise a method in which the computer could analyse each image and then assign a value to it which related to its fitness. This fitness defines how aesthetically pleasing each image is, the higher the value the better the image should be, and the higher the chance for this image to be chosen to mate with another to create the next generation. To calculate this value I looked at my own aesthetic judgement and defined the key aspects of what I call an aesthetically pleasing image, the first aspect is, if the image was spread across the whole screen then the fitness would be high, compared to a spot in the centre which would score lower, this was calculated by detecting areas of interest within the image and then summing the distances between them, the larger the total sum the higher the fitness would be. The second aspect is an image which had a varying texture this would score higher than an image with a less varying texture or a single colour screen, here I borrowed a method from statistics called variance and looped over all of the pixels within my image to calculate this value. The final aspect was the number of areas of interest detect within the image, here I used OpenCV to calculate if pixels were above a certain value and counted how many of these 'blobs' where in the image, the more areas of interest the higher the fitness of the image. When combining these three aspects together within my fitness function I multiplied each value by a number, the higher this number was the bigger the influence was from that aspect.
Exhibition
When displaying my piece in the exhibition I had to take into account and figure out the best way to communicate my project and ideas to the viewer. Whilst in the final stages of my project I decided to take a step back from my work and started to dissect and analyse both my the process taken and work which I had created, and it was within this period where I realised that my project had three distinct parts. The first part was the process taken by my program to draw each individual image, here I decided the best option was to have my program running live and in high definition, this allowed the viewer see how the images took form and hinted at the generative methods by displaying the code within the GLSL program on the screen. The second part to my project was evolving and mutating my images which allowed me to create a catalogue of 50,000 images. To communicate this to the user I decided to write a small OpenFrameworks program which looped through my catalogue, displaying around 30 images a second and stopping briefly on randomly selected images, by speeding this process up it allowed the evolution taking place within the collection of images to be seen by the viewer. The third and final part to my project was the high definition images themselves, here I decided to select eight which I found the most aesthetically pleasing from my catalogue and have them printed on to A3 paper and hung on the wall allowing the viewer to really inspect and get up close to the fine detail within.
Future development
Within this project there is a huge amount of scope for further development and exploration of the possible outcomes of the system I have created. Defining more mathematical functions to process and plot my points is one area which could be further developed, another is leaving my system to act autonomously for longer periods of time, this could be achieved by porting my system into JavaScript and running it continuously on a server and as a website, allowing viewers to dip in and out of the system and delve into the archives of the images created. Another area which could be developed is the fitness function within my system, here I could explore different methods for ranking the images, one idea for this is by using an input from a camera and judging how alike each output image is to this input image, this would introduce some interaction into my system allowing the viewers to choose what is considered to be an aesthetically pleasing image instead of basing the system on my own aesthetics. One final extension to this project would be to utilise the power of multiple computers to speed up the evolution process with this I could then create a Twitter bot where images are posted online, here I could post the best image from each generation allowing people to follow and keep up to data with the best current images.
Self evaluation
Overall I am happy with the final outcome of my project, I think the images created with my system were engaging and aesthetically pleasing to look at, whilst retaining a high standard in the quality of the graphics produced and I feel the reaction I got from talking to people during the exhibition reflected this. During my exhibition it became apparent that I could have labeled my prints with the population, generation and individual numbers, allowing the user to see at which stage they had come from within the evolution process, I think it might have also helped if I had labeled my computers to show the difference between the two. In terms of my genetic algorithm and the evolution process I have attempted to create, I discovered that the starting population or the seed to the system plays a massive role in the evolution path which is taken, thus I think the larger the population is at the beginning and the more variation within the system would allow for a better a evolution path taken. Whilst my system does work, this needs a considerable amount of work and time to explore the different possibilities and approaches to this algorithm, to improve both the selection process and fitness function, I can see this as being a source of research and interest for years to come and is an area I will explore and develop further.
References
- Mathematical functions from this paper on page 16 onwards were converted from equations to code by me.
- My GPU particles was inspired by this OpenFrameworks example.
- Algorithm for calculating variance ported from here.
- Much of my work is inspired by Inigo Quilez both aesthetically and technically.
- Electric Sheep by Scott Draves is a similar project which utilises the users aesthetic judgement.
- Darwin Tunes again uses the users aesthetic judgement in the evolution process.
- Fractal Flames have been a big influence, visually.
- Inconvergents aesthetics have also been inspiring within this project.
- The music used in my video.