Fluid Senses
Exploring the Language of Smell Through Codes and Texture
by Rob Elford, Hongye Jiang, Marlena Kepa, Xinyu Zhang
Abstract
Our work focuses on studying the intersection of smell and touch and how smells can be interpreted through the feeling of touch. We analysed the importance of the olfactory and tactile senses and their potential for integration and communication through technology. Using visual and computer-generated experiences that can communicate both smell and touch in a digital environment, we try to answer the question: how can smells be reflected or integrated with the tactile experience through texture?
The realization that the olfactory sense plays an important role in one’s access to memory, emotions and materiality, yet its expressions are relatively limited, has led to the scientific research on the biological basis of odour coding, and how humans’ sensory nervous system functions differently in processing visual and olfactory signals. We investigated how the brain interprets tactile feedback through touch and more widely the relationship between smell and touch. We also looked at recent developments in computing to understand how computational systems might enable us to communicate our sensory experiences in new ways.
Inspired by olfactory artworks, we decided to adopt a speculative design practice for the artefact, in order to offer a novel framework that raises awareness for the sensorial ecology. Envisioning a future deprived of visual and auditory senses, where olfactory and haptic senses become the primary source for communication, we came up with a scenario that allows seamless translation between smell and texture using computational tools.
By utilizing braille-like textures applied to wearable objects and textiles, we present our shared vision of a sensual future and artistic communication through physical 3D printed objects, of which the embossed 3D texture can be reproduced on the surface of both textiles and products.
Based on geometric tactile languages such as braille, we created our own language to enable an individual, distinct, but also instantly recognisable representation of scent. Referencing the common descriptors of fragrance within the perfume industry, the tactile patterns are divided into three distinct lexicons representing the three classes of notes, allowing the reader to explore a three-dimensional space from both behind and in front of the object, enabling new and unique ways to engage with and experience the notes of a smell as a tangible sensation. We envision the language to be read by using both hands simultaneously moving from the edges of an object into the middle.
With further speculations in mind, such as augmented haptic feedback for virtual experiences, fictional and preserved historic scents, smell data transmission as a social practice, etc, we hope to advocate for a more fluid, accessible and imaginative sensory ecology.
Overview & Context
Our project explores the intersection of smell and touch and how smells can be interpreted through the feeling of touch.1 It is rooted in our shared interest in multi-sensory experiences in digital culture. We are all curious as to how senses can be interpreted and synthesized through computational practices. We have chosen to focus on smell (olfactory sensation) for two reasons: 1) it creates a subjective experience that can be difficult to decode or standardise, and 2) it is largely under-represented and poorly understood in contemporary digital culture which leans towards visual and audible experiences.
We wanted to explore whether the two senses (smell and touch) can be interpreted and translated from one to the other using digital tools such as algorithms, machine learning and digital fabrication; giving rise to new possibilities for non-visual sensory communication.
Haptics are a key focus of our research as we wanted to explore how smells can be reflected and experienced through a tactile experience using different textures, taking inspiration from Braille and other forms of textured language. We also wanted to explore the relationship of smell to memory, emotions, materiality as also their potential for integration and communication through technology.
Research
The starting point for the research component of our work was the exploration of olfactory perception. According to many neuroscientists, the biological basis for the interpretation of smell is fundamentally different from vision which is based on three criteria of perception. These are stimulus representation, perceptual constancy, and figure-ground segregation (Hutmacher).
Differently, when interpreting smell, the brain has to decide on the grouping (type), salience (importance), and value of the smell, matching that information with learnt understandings of those smells. In other words, contrary to how we understand visual cues, when it comes to smell our understanding (the olfactory system) is driven by context-sensitive decision-making.
One of the key characteristics of smell is its unique link to memory. Studies have found that memories evoked by smell are rooted in childhood. This is in contrast to visual or written cues and memory which are formed predominantly in adolescence and adulthood (Rubin et al.). Because of this, and because smell is such a powerful trigger for specific memories, clinical experts are increasingly looking at how smell can be used to help patients recover lost memories.
In the computational field, olfactory technology is both nascent and limited. Applications such as digital noses, breath detectors and nasal cannulas have been invented only and their use is speculative. It was only last October that Google AI announced that machine learning is able to detect and predict smells in a lab environment using graph neural networks (GNNs). (Wiltschko).
Indeed it is not yet clear whether it Is possible to program and transmit scents computationally. Some theorists have suggested that the most effective means of doing this may be through the design of automated smell tracking systems taking inspiration from the male moth (Vouloutsi el al.), but the best applications of such technology are still unclear.
Turning to art, we found some interesting examples of the use of smell as stimuli, the New York based artist, Anicka Yi, explores the idea “biologized machines”, through biochemical installations representing the smell of bacteria in decay and growth (“7,070,430K of Digital Spit”). Sissel Tolaas, who specialises in exploring smell in her practice, has compiled a smell-lexicon of 2,500 fictional words that describes the scents she keeps in a library of smells. Both practices challenge the traditional approaches to the exploration and interpretation of smell which helped inspire and shape the next stages of our project.
Overall, we were struck that there were many more studies on vision than on any olfactory experience. Some theorists ascribe this to the technical complexity of replicating smell, which has meant that olfactory technology lags when compared to other sensory technologies (Hutmacher).
Ideation & Artefact
Our intention was to create a physical 3d printed artefact that is both organic but also conveys smell in an interchangeable and novel way.
We created an extruded 3D texture that can be mapped across the surface of both textiles and products. The texture consists of embossed primitives that translate the scents of an odour from olfactory input into a tactile output. The texture can be read in a similar way to braille, where each symbol interprets a different note (see below) of a given smell, allowing the reader to interpret smell through their sense of touch.
This approach enables objects and textiles to contain hidden sensorial messages that can be interpreted by the wearer/user. Our intention was to convey the scent of coffee – given its universal, distinct, easily identifiable aroma. To explore if this could be realised as a real-world product, we have prototyped a 3d printed wearable mask embossed with textured patterns communicating a scent - coffee. Our textured smell is touchable, artificial, emotional, abstract, and wearable.
The mask allows the reader to explore a three-dimensional space from multiple angles (i.e. from both behind and in front of the object), creating new and unique ways engaging with and experiencing the notes of a smell as a tangible sensation. The mask can be read by using both hands simultaneously moving from the edges of an object into the middle.
To represent the different notes of our scent, we created three embossed patterns, each representing the three classes of notes used as a descriptor of fragrance within the perfume industry:
- The Top Notes: which are the initial, short-lived scents.
- The Middle or Heart Notes: After the top notes fade, the middle notes become more apparent. These are floral or full-bodied scents that form the core smell of an odour.
- The Base Notes: which are the lingering notes with a deep and dominant smell.
Ideally these would have been created through machine learning whereby each pattern could be algorithmically defined by recognising the distinct odorant molecules associated with each aroma. However, given the constraints of our current skill base, as well as timeframes, we decided to create our own unique, artistic, geometric interpretation.
Reflections and Learning
We asked people (our readers) to describe what smell our ask conveyed? We received several different answers, including bread, petrol, hospital and nature. Despite the fact that not one of our respondents identified the texture with the smell of coffee, we think there are good grounds to consider the answers satisfactory, as we could prove that people associate textures or memories with a sense of smell, but one that is strong distinctive with shades of bitter.
Given time constraints we also found speculative design to be a valuable approach for addressing the wider possible applications of olfactory technology and future uses. Alongside the prototyping process, we considered various scenarios for future olfaction: a product that could fabricate smell? A phone able to process and transfer smell data? A haptic screen for digital scent transmission? A virtual world of fictional smells?
Conclusion
Our sense of smell is an essential component of our experience. But consumer technology has embedded a hierarchy of the senses into everyday devices. Modern computing - which is overwhelmingly smooth, sterile, anti-smell - has focussed almost overwhelmingly on the eyes and ears. In a world that emphasizes hyper-sanitation and deodorization, the result is an "unbalanced sensorium" (Howes).
By contrast, smells have been historically considered as emotional, illogical, and inferior. Our relationship with smell needs to be re-examined, and re-contextualised. Our project has aimed to re-address this. Our view is that the, smell, the olfactory sense, plays a very important role in perception but it is often ignored in computation and technology.
Fragrances themselves can be difficult to reflect in verbal form, but we believe that in the future we might be able to translate the smell into textures similar to the Braille alphabet. Despite the difficulty of capturing smell as data, we believe that our project has shown that it may be possible to capture smell through textures and create a virtual smell.
Fluid Senses Podcast
Bibliography
Boyle, L; Solomon, C. “On Speculative Design: Benjamin H. Bratton.” DIS Magazine, dismagazine.com/discussion/81971/on-speculative-design-benjamin-h-bratton/.
Hutmacher, F. “Why Is There So Much More Research on Vision Than on Any Other Sensory Modality?” Frontiers, 19 Sept. 2019.
Ip, C. “On the Nose.” Engadget, www.engadget.com/2018-10-26-on-the-nose-sissel-tolaas-detroit-exhibition.html.
Persaud, KC. “Engineering Aspects of Olfaction.” National Center for Biotechnology Information, U.S. National Library of Medicine, pubmed.ncbi.nlm.nih.gov/26042329.
Rubin DC; Groth E; Goldsmith DJ. “Olfactory Cuing of Autobiographical Memory.” The American Journal of Psychology, U.S. National Library of Medicine.
Vouloutsi, V; Lopez-Serrano, LL; Mathews, Z; Escuredo Chimeno, A; Ziyatdinov, A; Perera i Lluna, A; Bermúdez i Badia, S; Verschure PFMJ. “The Synthetic Moth: A Neuromorphic Approach toward Artificial Olfaction in Robots.” National Center for Biotechnology Information, U.S. National Library of Medicine.
“Learning to Smell: Using Deep Learning to Predict the Olfactory Properties of Molecules.” Google AI Blog, 24 Oct. 2019, ai.googleblog.com/2019/10/learning-to-smell-using-deep-learning.html.
“Scientists Uncover New Connection between Smell and Memory.” ScienceDaily, 23 July 2018, www.sciencedaily.com/releases/2018/07/180723155726.htm.