A E I O
Are you my French buddy?
by Sabrina Recoules Quang
This application is not really a game to play with, it is more of an encounter between you, the computer and your extended bodies on screen, on your face, or on someone else. Projecting yourself on one of them will therefore reveal a new body extension.
(E, I ,A and O) Exploring Inter and Action with the computer and the Other.
As Eric Shouse wrote in reference to Massumi's work (Volume 8 Issue 6 Dec. 2005 , http://journal.media-culture.org.au/0512/03-shouse.php)
“L’affection is each such state considered as an encounter between the affected body and a second, affecting, body … (Massumi, Plateaus xvi)
In my work as a shadow puppeteer, I explore encounters between extensions of myself as a mask performer, with a puppet on stage, and of course with shadows.
Shadows are not always black, they can disperse into an endless spectrum of colours if projected on a screen. The screen will reveal them.
Of course, there are multiple variables to take into account… Will the projections be on a flat screen or onto a performer? Is it an introspection, an interaction? Various types of screens can bring in many different characters of encounters. In this application, the computer vision helps the process of revelation. The computer adds a new dimension, a new inter-mediation between ourselves and our extended bodies.
Instructions
This application is using FaceOsc tracker from Kyle McDonald and is designed for one or two users. Only one user will interact with the computer and puppet the shapes with his/her face.
To start , keep your expression as neutral as possible , mouth close. Look straight in front of the webcam. This first position will be remembered by the computer as the neutral position, from where the code will interpret your movement and the vowels you will tell.
This application is set for a French speaker and will follow your command only if you articulate properly your vowels in a French way ! Have you noticed that native French speakers articulate far more than others?
If one user, the encounter starts with a small creature on screen that you can move with the movement of your face, she will mimic the raise of your eyebrow and respond to the tilt of your face. Soon , there will be a blue creature following the first one. Following them, a red shape , the “body” will appear with which the two creatures could play hide and seek.
If you smile enough, yellow particles will flood over the screen. The particles will react to their move.
Say “A” to trigger the apparition of a red mask which follows your face movements too
Say “O” to draw small red mouths dancing in a flower shape choreography above the red body.
Say “E” to erase the particles
Saying " I" in French should put a smile on your face ! If you smile enough the particles will appear again,
If you too far from the screen, the particles will go off the screen , stop reacting to the creatures movement. Smile again and they will play again together.
Find the right place to make your instructions taken. This place is where the encounter happens, between the mask you and the red body.
If two users, the first one will draw a story with all the shape characters on the other.
You can help each other by connecting your face movement.
Your encounter is helped by the computer which introduces a playfulness while keeping you connected together.You can play with the movement of your face, your mouth and eyes. Saying vowels will puppet the drawings too.
If you look at each other eyes ,the mask will be on your partner face and the body and the face of the mask will be reunited.
Experimentation , Evolution
Experimenting with this application was very joyful and I found myself talking very deeply with the users , even complete strangers. One user told me , using only face to draw or play with the screen opens a new perspective for people who lost the use of their limbs but still want to be creative. I like the possibility to use the machine to help interactions and communications between humans, and want to explore the therapeutic side of the work. Further developments would be to investigate about differences between cultures in the way you speak or interact with objects .
In term of coding, I will experiment with machine learning and speech recognition to improve the letters recognition. I could not succeed in implementing "U" instructions as this letter was to near "E" or the neutral position of the mouth.
Borrowing Code
FaceOSc by Kyle McDonald
Delaunay Triangulation week 11 goldsmiths MA computational Art
Using FaceOSc week16 goldsmiths MA computational Art
http://openframeworks.cc/ofBook/chapters/animation.html by Z.Lieberman for the particles movement and noise classes
Shiffmann //shiffmann tuto https://www.youtube.com/watch?v=OkkjlHTdLGI for the flower animation
