The Awkward Eye
The Awkward Eye is a series of experiments happening at the juncture between computer vision, and the interactivity possibilities that this new technology opens, and literary production. The project explores various avenues and modalities of this intersection and offers ways in which these first steps can be taken further. The Awkward Eye thus attempts to elaborate an artistic connection between two otherwise relatively remote poles: the growing field of computer vision (and, more specifically, as will be seen below, smile recognition) and literary writing, in this case poetic writing.
Produced by: Guy Morgan & Jérémie Wenger
The textual experiments started with the smile detection in mind: if the reader smiles, something happens in or to the text; if the reader stops smiling, the text reverts to its initial state, or moves on to another one. At an abstract level the smile was used as switch, with two positions, on/off, which would trigger changes in the poem. A big question at this stage was whether the ‘smile’ element was in any way relevant, and could not be replaced by a click of the mouse, or an actual switch. The answer given was thematic: if the texts played with the semantic contents of the smile, with the (presumed) mood of the readership, a genuine connection could be secured. However, it quickly appeared that the simple on/off switch ‘functionality’ wasn’t enough, especially as a smile has duration (and possibly intensity, but that could not be studied at this stage). What emerged then is that the smiling could be used just as productively as a way of creating two different temporal spaces: one in which the smile is ‘active’, and one in which it is not. Each of the spaces then could be the theatre for various events, either continuous during the whole of the space, or discrete ones, etc. One of the major turning point happened at this moment, when we decided to integrate a random text production machine in the artifact: the two states would no longer be two (or more) different texts, but a state of production, where many ‘poems’ would be generated, and another, a static one, where the latest result would be presented.
This first phase of experiments, however, was brought to a halt for technical reasons (compatibility issues as well as Web implementability). As had happened before in the elaboration process, we had to refocus our energy onto another tool, one that could allow us to pursue similar alleyways of research (such as two textual spaces triggered by the reader), but without the smile involved. Having discounted an undocumented third-party library that used processing’s OpenCV library to specifically look for ‘smiling’ in the reader, we settled for identifying faces in a computer’s default camera, which brought us down to a more elementary level where it is the sheer presence or absence of a reader that triggers change in the text. Initially the interaction occurred when a face was recognised: the text stopped being generated, and the reader could access the result. In this way the tool’s ‘attention’ was focused on being helpful to us, humans. This was not as rewarding a experience as we planned, however – trial and error revealed a more playful, dynamic, a ‘shy’ poetry generator continually ‘spooling’ through word combinations when a face is recognised by the Open CV library – which is most of the time if you’re trying to read the poems. Only by obscuring your own face can you stop the hundred of poems from being produced, much faster than you can ever read – by peeking through your fingers, or from a very acute angle. In this way a push-pull, playful interaction is created. It also preserves the editing mechanism present in the former version – you can call ‘next’ on an unsatisfying poem by showing your face again.
Two final features would be added later in our experiments: a wild, virus-like functionality whereby every new static state would save itself onto the reader’s computer, generating an uncontrollable profusion of poems in the user’s folders; a ‘Penelope’ variation, which would substitute to the static state a slow and unpredictable erasure of the final result, meaning that any ‘final’ text would instantly start disappearing under the reader’s eyes, gnawed away by a machine that only reluctantly reveals its creation to the prying human eye.
All texts and discussions can be found here:
– Guy Morgan's blog;
– Jérémie Wenger's blog.
A clickable version of the poetry generator.
A GitHub repository with various versions and experiments.
References:
Ed Atkins, 2012. 'Depression'. [online]. Available from https://vimeo.com/61089836
Chung, B. 2011. 'Smile detection in Processing - Mac OSX'. [online] available from http://www.magicandlove.com/blog/2011/05/04/smile-detection-in-processing-mac-osx/
D-AW, 2013. 'Three OED Poems'. [online] Available from: https://poetry-contingency.uwaterloo.ca/three-oed-poems/
Flores, Leonardo. 'I ♥ E-Poetry'. 2011-present. Available from http://iloveepoetry.com/
Electronic Literature Organisation. Available from https://eliterature.org
Gottman, J, Murray J et al. 1995. 'Mathematics of Marital Conflict: Qualitative Dynamic Mathematical Modeling of Marital Interaction'. Journal of Family Psychology. http://joe.ramfeezled.com/wp-content/uploads/Cook-et-al-JFamilyPsyc-19951.pdf
Hayles, N. Katherine, 2007. 'Electronic Literature: What is it?'. Available from http://www.eliterature.org/pad/elp.html
Simanowski, R. 2004. Concrete Poetry in Analog and Digital Media. Centre for Art and Media. [online]. Available from http://www.dichtung-digital.org/2003/parisconnection/concretepoetry.htm
Thorp, J. 2014. Wordplay: A tool for freeform language exploration. [online]. Available from: http://blog.blprnt.com/blog/blprnt/wordplay