CyKink
This is an interactive program which runs on Openframeworks, open your camera and try to get shapes and colors you want on the 3D abstract model!
produced by: JINGYI CHEN
Introduction
The program is interactive by movements of limbs, using addon ofxGUI, ofxMaxim and ofxRapidLib, to control a bird-like spaceship in 3D space like the characters in sci-fi fiction.
Concept and background research
I am mainly inspired by Japanese animations watched when I was a child. In many animations like Neon Genesis Evangelion, characters drive transformers to fight against enemies. They use handles and keyboard to manipulate transformers and transformers are easy to transform various forms. So, I envisioned using handles via machine learning to create continuously changing abstract 3D models in Openframeworks. Also, I am affected by the spaceship which is ridden by aliens from another galaxy civilization in the Chinese sci-fi fiction Three Body. It's a high dimensional spaceship therefore its mapping projection on a three-dimensional world is a continuously changed 3D geometry. Accordingly, I decided to create a fluid-like machine in 3D.
Technical
I originally want to use Gametrek to collect manual data, but at this time in my hometown, I can't find one. Under the circumstance, I used TouchOSC, OSCulator and Wekinator to draw gestures and train from them. But during the process, I find delivering data by OSC message caused jams in rotation. In addition, to consider showing the project online in the future, and OSC is only on iOS/macOS platform and it is not free, so I decided to discarded OSC and to use the camera and ofxRapidLib to do machine learning.
As for the 3D model part, firstly I download an obj. model from the Internet and modify it in Blender and exported it by ply. form in Openframeworks. In terms of the shape, I am inspired by abstract sculptures created by Alberto Tadiello.
The base code I used in C++ is Particle Lab by Andy Lomas and by the variables like amplitude, bias, octaves I can control the size and the frequency of change. By transferring each Vertex of ply. model to 3D space and connecting them as a mesh then draw triangles over the mesh. In addition, I used Shader (vert. and frag. files in the data folder) to make its visual effects. Like the sound, I used add-on ofxMaxim and the example of addictive synthesis to use many arrays of data to create sounds that can be felt like sci-fi. By using ofxGUI I can adjust the parameters during the process. By ofxRapidLib and the example of imageClassifer, I can easier to make classification on Openframeworks.
Now by tapping 'r' to recording and training the positions of the hand I can run my project and try to control my fluid-like machine and listen to the mix of different sounds. It is interesting during the progress to find which position the machine can recognized the class most accurately and to see the flicking of the shapes.
Future development
It achieved preliminarily. If I have more time, I'd like to use game handles like Gametrek or Logitech Extreme 3D pro as well as create a physical interface. Moreover, I will make it smoother during the change like mercury and use space HDRI to make it looks like in outer space. I have used it in the file but haven't figure out temporarily why it can't be rendered in this program. It is good in other programs. It also worth spending more time adjusting parameters of the Shader, sounds and the camera even to create a system and fiction story for it. I need to watch more interactive artworks and get inspiration from sci-fi poems, novels and films to create a series of sci-fi, abstract and meditation works. I also want to transform it into an online interactive work and design more delicate motions.
Self evaluation
I have met a heap of problems during the process but also learned more. I tried API, json, wit.ai, Shader and other things in OpenGL, have a deeper understanding of C++ and can fix some bugs which can't solve before. But I have wasted a lot of time in libraries and examples which are irrelevant and can't settle down my idea and make a demo at the beginning. Also, I need to buy and try the projector and other devices as soon as possible.
References
Related artworks:
Alberto Tadiello, Untitled (Adunchi), 2010 https://www.pinterest.com.au/pin/409546159865249372/
Libraries:
OF addon: ofxRapidLib, ofxMaxim, ofxGUI
Shader version 150, OpenGL 3.2
Referenced Examples:
WekinatorPenguin, addictive synthesis, Particle Lab3 by Andy Lomas, image Classifier
Softwares:
Openframeworks
Blender
Model:
Tydirium T-16 Shuttle 3D Model
https://free3d.com/3d-model/tyderium-t-16-shuttle-47589.html