Synesthetic Dreams
Synesthetic Dreams is a generative interpretation of the perceptual condition of synesthesia. Modalities are fused across the series of animations as a whimsical representation of the condition.
produced by: Safiyyah Nawaz
Concept and Background Research
Synesthesia is characterized by an unusual fusing of the senses; a stimulus from one modality, music for example, is associated with an unrelated modality, like sight, leading some synesthetes to be able to effectively “see music”. There are many variations of synesthesia, including:
Seeing colours when listening to music (Chromesthesia)
Associating letters and numbers with particular personality traits or colours (Ordinal Linguistic Personification, Grapheme-Colour Synesthesia)
Feeling sensations when you see somebody else experience them (Mirror-Touch Synesthesia)
Feeling shapes in your hand when tasting food (nameless, but from The Man Who Tasted Shapes)
To me, the existence of this phenomenon emphasizes the subjectivity of reality across individuals. Learning about ways others may see the world (figuratively and in the case of synesthesia, quite literally) creates opportunities to share our realities with each other.
Technical
With Synesthetic Dreams, I use the same four colours to represent four modalities of the face (Hearing, Vision, Taste, and Smell) and then use lerping and mapping to create fusion effects. Across scenes, these colours are included in animations as representations of modalities interacting with each other.
Synesthetic Dreams is composed of the following scenes, all coded in C++ using OpenFrameworks and the ofxPiMapper add-on, and projection mapped onto the polygonal face mask, Robert:
Face Outline. Each modality is highlighted by its corresponding colour. As all modalities are present, outlines of the face sporadically and randomly pile across the surface while subtly fusing between colours, adding layers of colour and dimension to the scene.
Neon Zoom. Each modality is once again individually highlighted, this time in a spinning and zooming sin-wave animation. Beginning as a slow animation, the spinning quickly picks up pace and eventually leads to a hypnotic strobe of fast-moving colour.
Sensory Fusion. This scene represents the movement towards fusion of two senses. Beginning with two colours, lines flash across the screen, becoming larger and more similar in colour until the moment of fusion is met. Circles in the forehead representing the neurotransmitter serotonin (which is believed to be implicated in synesthesia (Brogaard, 2013)), bounce around until they are released and envelop the whole surface.
Spiral Mind. In this scene, modalities are individually fused in different combinations, representing the range of experience even across synesthetes.
Future development
The current work could be expanded to elaborate on current theories of synesthesia, or even demonstrate actual perceptual experience. It could be interesting to use knowledge from neurobiology to convey a narrative of how synesthesia develops or functions on a molecular level. Additionally, as synesthesia is indeed a multi-sensory condition, another portrayal using sound, smell, touch, etc. could more realistically portray the experience of synesthesia.
Self evaluation
I am overall satisfied with the way Synesthetic Dreams turned out. I believe that my use of mapping to subtly change colour and shape achieved the desired effect. I could have further pushed myself to use text and image/video within the animations for a little more variety as well. I also would have liked to include more neuroscience theory so that the overall work may have more educational value.
I also believe that projecting on the polygonal face mask was a creative choice and worked well with the concept I chose. Although the surface was very complex, I was able to create a detailed XML file by making a separate app, then using that to map coordinates of the mask from a photo of it. I am pleased that we were able to pull off mapping on a surface that had so many contours, and for the end result to look so cohesive. I do believe that if the mask was larger we may have been able to have an even grander effect.
References
"bouncingBalls" code from IS71014B Workshops in Creative Coding 1, Week 4 - Repeat, repeat, repeat. Retrieved 2 February 2020, from https://learn.gold.ac.uk/course/view.php?id=12859§ion=4
Brogaard B. (2013). Serotonergic hyperactivity as a potential factor in developmental, acquired and drug-induced synesthesia. Frontiers in human neuroscience, 7, 657. doi:10.3389/fnhum.2013.00657\
Gaia, J. (2014). Urban Conspiracy. On 12th Floor Party. [Audio file]. Retrieved from https://www.epidemicsound.com/artists/jules-gaia
Kirkpatrick, N. (2018). The Many Types Of Synesthesia Explained | Betterhelp. Retrieved 2 February 2020, from https://www.betterhelp.com/advice/synesthesia/the-many-types-of-synesthesia-explained/