Miscommunication
Miscommunication is a project that uses computer vision and face tracking to detect presence of multiple faces in front of the camera. The visuals increase in complexity when more and more faces are introduced, while the typefaces transforms by the movement of the faces.
produced by: Eddie Wong
Introduction
Miscommunication reimagines the interaction between more than one viewers in real-time through the eyes of the machine. It starts off with detecting one face. With the introduction of more and more faces, a series of lines are drawn to connect one face to another. The idea is to encourage viewers to gather in the periphery of the camera and 'connects' physically and also virtually. The more faces that are detected, the more complex the visuals get as it leaves traces on the screen. While all this is going on, the word 'miscommunication' morphs from a sans-serif typeface to an organic typeface to the direction of where the face is looking.
Concept and background research
The work was an exploration of interesting dynamics in communication between humans and humans through machine mediation. My initial idea was to convey 'miscommunication' by morphing between one word/sentence by 'tweening' between one contrasting font with another ie. English to Chinese. Next, I've introduced face tracking to control these transformations. The idea soon evolved into using the default face tracking mesh as part of the visual design itself. The addon can detect more than one face so the more faces that appear, the more connected lines are drawn turning it into a 'paint-by-face' program.
The graphics used during the live shows of the band Disclosure was one of my reference. The design of their face projection looks eeriely similiar to the default mesh of the face tracker addon. I was also interested in achieving the effect in this video for the morphing effect for the text. A great project by Zach Liebrman - Más Que la Cara and Machine Pareidolia by Greg Borenstein was also an inspiration.
Technical
I build the project using Openframeworks. I used the ofxFaceTracker2 addon which proved to perform faster than the ofxFaceTracker addon which it was based one. The key feature of it had was the ability to detect more than one face simultaneously which I have exploited to my design. I even tested it out using photos of faces that are relatively small and it worked relatively well. The performance tend to have dropped when detecting more than 3 faces. However, I have included that performance limitaton into my design idea by turning off Auto-background mode to create trail of faces with every appearance of a detected face.
For the type animation, I've manipulate a set of points that make up the letters to blend into the movement of the face. The source code was adapted from the organic typography lab assignment we did in Term 1.
Future development
I could do alot more to further develop this project with face tracking but ultimately ran out of time. I would have each feature on the face to control the appearance of a text. I'd also reworked the arrangement of the peices that conveys the idea of two views communicating. Perhaps also, the appearance of a second face takes on a different mesh design when new faces are introduced. I'd also focus more on the text, perhaps creating a series of characters instead of just one sentence.
Self evaluation
Overall, I am satisfied with the work in the face of my rudimentary coding skills. The idea have been changed from what I initially proposed due to the scope and time constraints. It'd be useful to have exhibited the work to get audience's live reaction as the work was meant for a small group of people (by leaving traces of the their face on screen). I'd have finesse the idea more to incorporate a meaninful interaction between the face detection method with the points on a type.
C++ still proves a challenge for me especially with integrating addon features to the things I've coded. I'd also used more object oriented programming and organise my code better.
References
- ofxFaceTracker2 - https://github.com/HalfdanJ/ofxFaceTracker2
- ofxCv - https://github.com/kylemcdonald/ofxCv
- Organic Typography Lab - https://learn.gold.ac.uk/mod/page/view.php?id=600210