Generative vs. Generative
I set myself an ambitious goal to create a voice controlled sketch using the Wit.ai api. 40 days later (give or take), I am not very far from square one. Instead I made the executive decision (some would say 38 days too late) to focus on something else as time was running out.
produced by: Omolara Aneke
Introduction
With a background in Fashion and Print design I am intrigued by the possibilities of generative pattern design. Designing in this generative way allows for rapid prototyping of ideas as well as the room for new and more inspired creations. Merging all my interest, I looked further into Cellular Automata and Conway’s Game Of Life for visual inspiration on how I could attempt to do this. I initially envisioned creating a generative design that could be printed onto fabric for matching durag and top sets.
Concept and background research
As always, my starting point was to head to those same titles in Daniel Shiffman's Nature of Code book and the correlating youtube pages. During which I discovered a channel, Softogloy, who creates 3d cellular automata designs. Enticed by the intricacy, flow, dimension and colours of his work, I decided to create a 3d cellular automata sketch. But instead of endlessly pursuing my original idea to make it voice controlled, I decided to do the opposite. Where are I would usually make a program and then put msuic to after wards, i thought i would try and generate music as the visuals unfold on the screen.
Image : 3D History Dependent Cellular Automata, 2018, Softology via Youtube
Technical
As mentioned previously, I set out to use voice control as my one type of interaction but that proved more difficult than i had originally anticipated. Not wanting to use the machine learning example we used in class, I attempted to try to stretch myself and use the Wit.ai api
I faced two major hurdles consecutively but Openframeworks alway complied and ran so I assumed I was doing something right and with time and tweaks I would eventually get there. Audio then computational speed. Getting the correct response back with the speech end point was trying. Trouble came sending a live stream which elicited 1 of two responses, 1. referring to ‘under/overflow detected’. And 2. {entities: , value: }. All very promising as it had not crashed or thrown up any error signs. inested , as the http request pointed to one file, I attempted to constantly record over and send the one wav file. All of this was absolutely wrong as it needed the keypressed function to record and stop recording and it took up too much computation power showing animations at a slow and jittery speed.
Instead, I used the same animation but used to output. Using the ofxMaxim addon, I managed to set up envelopes of sound that are triggered when looping through the matrix and creating randomly generated 3d boxes. Along with adding frequency changes depending on each state change, the result is a type of generative musical sketch.
My first attempt at this was to have one sound wave running through the whole program, then when the states are changed, a different frequency is called. This took up too much computational power, and the sound was choppy but not in an artistic this-is-intentional-kind-of-way. I thought maybe the sounds were interfering with each other so i attempted to add multiple maxiOsc objects and call the different waves, but this was a worse solution. As a result I turned to MaxiEnv’s which allowed me to trigger a very short sound. In addition I slowed down the layer generation inturn slowing down how fast the sounds are produced.
Creating the visual was a task, but also a great feat. I ran into trouble using vectors to make a 3d matrix and abandoned the workshop example in favour of an array implementation of a grid like structure. The biggest problem here was balancing visuals, speed and sound with computational power. This meant that something will suffer as a cost so I sacrificed speed as the 3d structure increased in layers. This means there is a slight wiggy room for a not smooth program.
Future development
Further developments involve finding a solution to the Wit.ai live streaming audio problem as it would be a good tool to utilise. I would like to look further into creating designs more similar to that of Softology, instead of makeing larger cube shapes, I would like to look into other structures like spheres or pyramids. I would also like to figure out a way to structure the code to make the visuals run at a constant speed.
Self evaluation
My time management skills need work. Far too many times during the creation project did I contemplate leaving the path but stuck it out in favour of perseverance. I do not think this was a successful attempt at showcasing my effort level, in hindsight I should have moved on a lot earlier or used the Machine Learning example we used in class as it would have been a simpler way to implement handsfree voice control, leaving more time to develop and research rules to create more interesting 3D automata visuals. To add, the sounds generated are reminiscent of child learning to play piano and they lag slightly from the visuals. With more time to research, both of these things could have been improved.
References
Elliott (2018) eio/osc-controlled-cellular-automata. Available at: https://github.com/eio/osc-controlled-cellular-automata (Accessed: 5 May 2020).
Game Development (2020) Generate Random Cave Levels Using Cellular Automata, Game Development Envato Tuts+. Available at: https://gamedevelopment.tutsplus.com/tutorials/generate-random-cave-levels-using-cellular-automata--gamedev-9664 (Accessed: 5 May 2020).
Guy Birkin (2020) ‘6-Colour Block Cellular Automata’, Aesthetic Complexity, 9 March. Available at: https://aestheticcomplexity.wordpress.com/2020/03/09/6-colour-block-cellular-automata/ (Accessed: 5 May 2020).
Learn.gold (no date) IS71014B (19-20): Game of Life. Available at: https://learn.gold.ac.uk/mod/page/view.php?id=720848 (Accessed: 5 May 2020).
Learn.gold (no date) IS71084A Stream A (19-20) : Lecture recording – Week 9. Available at: https://learn.gold.ac.uk/mod/page/view.php?id=745128 (Accessed: 17 May 2020).
openframeworks (no date) ofBook - Sound. Available at: https://openframeworks.cc/ofBook/chapters/sound.html (Accessed: 5 May 2020).
Ovkaric, S. F. (no date) stanko16/CA3D, GitHub. Available at: https://github.com/stanko16/CA3D (Accessed: 5 May 2020).
Parametric Online (no date) Processing Tutorial: 3D cellular automata. Available at: https://www.youtube.com/watch?v=dfzdSl9zYog (Accessed: 5 May 2020).
Sahas, C. (2020) ChitlangeSahas/3D-Cellular-Automata. Available at: https://github.com/ChitlangeSahas/3D-Cellular-Automata (Accessed: 6 May 2020).
Softology’s blog (2020) Softology’s Blog, Softology’s Blog. Available at: https://softologyblog.wordpress.com/ (Accessed: 5 May 2020).
stackoverflow (2020) c - Please explain what this code is doing (someChar - 48), Stack Overflow. Available at: https://stackoverflow.com/questions/3195028/please-explain-what-this-code-is-doing-somechar-48 (Accessed: 5 May 2020).
The Audio Programmer (no date) ofx Maximilian & openFrameworks Audio Tutorial 01- ADSR Envelope & Triggering. Available at: https://www.youtube.com/watch?v=tRTlc2NVes8 (Accessed: 5 May 2020).
Weisstein, E. W. (no date a) Elementary Cellular Automaton. Wolfram Research, Inc. Available at: https://mathworld.wolfram.com/ElementaryCellularAutomaton.html (Accessed: 5 May 2020).
Weisstein, E. W. (no date b) Rule 30. Wolfram Research, Inc. Available at: https://mathworld.wolfram.com/Rule30.html (Accessed: 5 May 2020).