Connected Emotions
Connected Emotions is a study of how visual stimuli can influence a viewer to feel an emotion in response to what they are seeing. Our project reflected on how this perception of emotion can be studied and in turn represented, drawing on current computational arts based practices and theories of feminist technoscience.
produced by: Emma Dutton, Christina Karpodini, Hazel Ryan
Introduction
Emotions are an important aspect of our lives and ourselves. They define our actions and they are dispositions of them (Lang 1995). According to the theory of kinesthetic empathy, it is possible to experience emotion based only on observing the movements of another human (Reynolds and Reason 2012). It has been shown that when we see someone experiencing an emotion, we simulate it (Freedberg and Gallese 2007) which led us to question whether artists are aware of this emotional simulation when they are producing work, or whether it is an additional element that occurs after the creation of their art. To understand this, we engaged in discussion with several contemporary computational artists to understand how emotion influences their various practices: Memo Akten, Simon Katan, Andy Lomas and Atau Tanaka. A common thread we learned was that emotional connections were central to their practice, but none sought to specifically convey, or induce, a single emotion through their work. We wondered what it meant to attempt to render an emotion into an artwork or artefact.
Having been influenced by Tina Gonsalves who used computational practices to measure emotional responses and then use it in her art, we chose to use create an artefact using a similar means in our project. We used Galvanic Skin Response (GSR) to measure the physiological impacts of emotion through reading the arousal responses to an audiovisual stimulus. To enable a richer set of data, we also asked the participants to verbally explain to us how they felt after watching the material, speaking freely. This meant we had both quantitative and qualitative data to represent an individual's emotional experience.
We used this combination of data to create a 3D printed manifestation of one participant’s emotional response. One object displays the data from the GSR device over time. A second object is based on a sentiment analysis of the participant’s dialogue taken after the viewing. Both types of data can be seen as human experience filtered through machine knowledge.
The print process allowed us to reflect on the translations and, simplifications that must be made when attempting to visualise personal data. We found the art of Maurice Benayoun to be relevant when analysing our results. Drawing from affect theory (Wilson 2010), we would not conclude that the final objects are a realistic representation of the emotions of an individual. However there are patterns in the artefact supportive of the constructivist approach to emotion as not being hard-wired but being based around prediction (Barrett 2008): we speculate that the participant’s responses intensified over time as the audiovisual material’s context became clearer, strengthening understanding and anticipation. Our final product represented both the distance between the two types of data output, and the distance between machine and human understanding of emotion.
This project provided us with a unique opportunity to draw together primary research, theoretical sources and data visualisation techniques with connected threads of emotion and computational arts practice.
Future documentation of our project can be found on our website here.
References
Awan, N. (2016). Digital Narratives and Witnessing: The Ethics of Engaging with Places at a Distance. GeoHumanities, 2(2), pp.311-330.
Benayoun, M. (2018). Brain Factory Prototype 2. [online] Maurice Benayoun. Available at: http://benayoun.com/moben/fr/2016/12/26/brain-factory-prototype-2/ [Accessed 14 Dec. 2018].
Chafale, D. & Pimpalkar, A. (2014). Review on Developing Corpora for Sentiment Analysis Using Plutchik’s Wheel of Emotions with Fuzzy Logic. International Journal of Computer Sciences and Engineering, 2(10), pp.14-18.
Freedberg, D. and Gallese, V. (2007). Motion, Emotion and Empathy in Esthetic Experience. Trends Cogn Sci., 11(5), pp.197-203.
Gonsalves, T. (2010). Empathy and interactivity. Computers in Entertainment, 8(1), pp.1-14.
Lang, P. (1995). The emotion probe: Studies of motivation and attention. American Psychologist, 50(5), 372-385.
Reynolds, D., & Reason, M. (2012). Kinesthetic empathy in creative and cultural practices. Bristol: Intellect.
Wilson, E. A. (2010). Affect & Artificial Intelligence. Seattle: University of Washington Press.