Responsible Shadow
Installation that uses computer vision to track viewer’s silhouette and incorporate it into the projected system where all elements influence each other; proof of concept.
Project by: Anna Alfut
When defining the experience design, I came to think about it as inspired by ‘The North Wind and the Sun’ Aesop’s fable. In the story, Wind makes a bet with the Sun as to who will make a Man remove his coat. Wind tries to achieve the results with force, causing of course the opposite effect, while the Sun calmly shines on the Man leading him to remove his coat.
I wanted to introduce a world with particles system that are enjoying their life bouncing around. When user's silhouette is detected particles are scared and run away from it, unless the user stays still, which results in particles calming down and curiously gravitating towards them.
Needless to say, it didn't turn out as I planned.
Technical development
Addons: ofxKinect, PS3EyeGrabber, ofxOpenCV
Set up: I used PS3 Eye camera and the Bouncing Ball-OOP-system and Monster Shadows class examples as a starting point. I haven’t solved the Monster Shadows task before. My code was going to have four parts: the main class, particles system, contour tracking code and the background environment files (I did not use the background at the end).
Particles system: After I integrated my objects, I prototyped particles behaviours based on the mouse coordinates. Particles have three motion types depending on whether the mouse is in the view, and if it is moving or standing still. The logic uses couple of boolean variables. Every 50 frames I am checking distance between x and y coordinates and set flags based on that.
Computer vision: The starting point was to draw outline around blobs, blob detection code was provided, I adjusted its parameters. Afterwards I extracted vertices from blob polyline and checked their normals. That was the baseline of polyline that I used to draw the final shape path. I wanted the blob to be visibly larger than the person that it was based on. Unfortunately, despite smoothing and resampling the intermediate polylines, I didn’t manage to get a smooth shape. My final step was to calculate the average centroid from all underlying blobs. This is the point that I used to control the particle system. The contour part was difficult for me to reason around and I did use pair programming support from more experienced C++ coder, who helped me research documentation and talk my ideas through with me.
Putting it all together: For legibility, I decided to move contour code together with the PS3Eye setup into separate class. It became problematic later when I wanted to re-direct my particles system from mouse coordinates to the blob average center. I had to access this point in all of my classes. After trying several approaches: trying to pass value to the main class, creating separate function within blob class, or even adding extra class to keep the value in it, in a moment of despair I settled on the non-recommended extern line at the top of each cpp file. It is not ideal, but it works. At this point I ran out of time and the system was functional enough to demonstrate the concept, so I decided to stop here.
Further development
I want to continue working on this project during the summer to complete its functionallity but also to make it look good. I need to smooth out the contour shape and apply styling with the ofxGLSLSimplexNoise shader or other methods that would give me a similar effect. I also want to refine particles movements. As of now they feel too mechanic. They could also sit on the blob more like a halo/cloud instead of gravitating to its center.
My original concept also had a genetic algorithm component to it. It would be triggered when the user is standing still, the colour of the blob should change to indicate to both user and ‘metaphorically’ to the particles the change of state. Particles that touched the blob would also ‘collect’ this colour and pass it on to the background system so that when the user exits the view there would be a new plant added, personalized to this user’s blob. That is the ultimate goal and a pay off to the user, to positively contribute to this environment.
Reflection
My idea was too large and complex for the timeline and my current technical abilities. I was still learning, re-architecting code and adjusting the idea on the go. I do not see this submission as a final piece more a proof of concept. I also feel that due to my lack of experience, my approach was reactionary and somewhat intuitive rather than deliberate. At times a solution would be obvious only to completely escape my mind a minute later. I also got used to pair programming and find it difficult to work on my own in this medium. Lots left to learn, it was a humbling process.