WAYS OF TOUCHING
Our research is a case-study of a touch based interface, developed by the start-up Ultrahaptics. Their technology create three-dimensional fields that create a sensation of touch on the palm when passing the hand through. These ultrasounds is modulated by sophisticated algorithms and is produced by an array of transducers.
The video below shows a record of our experience at Ultrahaptics during a field trip to Bristol. Our experimentations led us to wonder what ‘it’ was we were touching; more than a digital representation, yet far from a solid form. We figured looking into the neuroscience and cognitive psychology behind it would help us get a better understanding of how touch works.
Ultrahaptics hardware uses acoustic radiation force, a force generated when the modulated ultrasound is reflected off the skin. When focused onto the surface of the skin, it induces a shear wave that will be sensed by the neurons in our hand. The displacement caused by the wave triggers mechanoreceptors within the skin, generating a haptic sensation. On an experiential level, it creates a vibrating membrane describing the shape. Its material nature is not solid and left us with the feeling that it lies in another state of existence. Its application are mostly imagined and depicted in the fields of VR, AR and as general purpose touch interfaces.
Ultrahaptics started in the academic world as a PhD research project. Now a company, it is funded by four large investment companies. While these companies provide the means to develop, it may also move its potential outcomes towards certain money driven ‘figurations’. It was also interesting to perceive some of the company’s publicity through Lucy Suchmann’s idea of “cultural imaginaries”. [fig. 1]
Cognitivism sees the human body as partly similar to a computer, in the way that it computes complex multiple signals into intelligible information. Studies in multisensory integration show that the modalities of the sense of touch interact at many levels. The neurons in our fingertips turn stimuli into information, either through the sensory nervous system or the cortical system. This gives us a sense of the complexity with which the sense of touch operates, and the the impossible task of simulating it with the current technology at-hand. We also begin to get a representation of what the physical properties of Ultrahaptics’ “half-tangible” simulations approximate as they only activate a small segment of the full set of modalities.
One of the major new elements we have seen from this innovation in interfaces is the idea that materiality could become temporal, displaying the true nature of modern digital mediums. For a long time digital work has been constrained into a static and permanent state when materialised in the physical world —a poor reflection of the flux nature of the digital world. This evolution allows us to feel entirely new dimensions at our fingertips.
These considerations led us to imagine where the future of touch and materiality may lie. In the animation below we speculate on the possibilities of a new temporal form. It shifts instantaneously in-between existing materials, and even envisages materials outside of our comprehension. We believe this could totally reinvent how we can communicate our with devices and each other, allowing new languages of touch to evolve and letting us translate what we cannot put into words, into touch.
RESEARCH BY: Julien Mercier, Jonathan Fuller-Rowell, George Simms.
Bibliography
Articles:
* J Andrew Pruszynski & Roland S Johansson. Edge-orientation processing in first-order tactile neurons. Nature Neuroscience. 2014/08/31/online. Nature Publishing Group. https://www.nature.com/articles/nn.3804 (last consulted 12/16/2018)
* Patrick McGurrin. The Sense of Touch. https://askabiologist.asu.edu/explore/how-do-we-feel-touch (last consulted 12/16/2018)
* Chegg Study. https://www.chegg.com/homework-help/definitions/cognitivism-13 (last consulted 12/16/2018)
* Victoria E. Abraira, David D. Ginty. The Sensory Neurons of Touch. https://www.cell.com/neuron/fulltext/S0896-6273(13)00710-1 (last consulted 12/16/2018)
* Mo Costandi. Brainy processing at your fingertips. The Guardian, online edition. 09/08/2018. https://www.theguardian.com/science/neurophilosophy/2014/sep/08/brainy-processing-at-your-fingertips (last consulted 12/16/2018)
* Giacomo Handjaras & mult. authors. How concepts are encoded in the human brain: A modality independent, category-based cortical organization of semantic knowledge. NeuroImage Volume 135, 07/15/2016. https://www.sciencedirect.com/science/article/pii/S1053811916301021 (last consulted 12/16/2018)
* Robert M. Friedman, Li Min Chen, and Anna Wang Roe. Modality maps within primate somatosensory cortex. https://www.pnas.org/content/101/34/12724 (last consulted 12/16/2018)
* Ritsuko Tsuboi. Sensory interaction refers to the interaction of the senses. https://prezi.com/zmeje8lwllyl/sensory-interaction-refers-to-the-interaction-of-the-senses/ (last consulted 12/16/2018)
* https://www.newscientist.com/article/mg21028185-800-haptic-soldiers-guided-by-buzzing-belt/ - New Scientist : Haptic soldiers guided by buzzing belt
* http://www.multi-sensory.info/images/files/CHI15_Obrist_TouchEmo.pdf - Emotions Mediated Through Mid-Air Haptics, Sussex University
* http://sro.sussex.ac.uk/71031/3/Accepted%20-%20Final%20version.pdf - The How and Why Behind a Multisensory Art Display, Sussex University
* https://www.kickstarter.com/projects/littleriot/pillow-talk-feel-the-presence-of-your-loved-one - Pillow Talk : Feel the presence of your loved one
* https://www.youtube.com/watch?v=4c1lqFXHvqI - Can we create new senses for humans? | David Eagleman | Ted Talk
Books:
* Neisser, U. Cognition and reality: Principles and implications of cognitive psychology. New York: Freeman. 1976.
* Hubert L. Dreyfus. What Computers Can't Do. New York; Harper & Row. 1972.
* Lucy Suchman , Human-machine reconfigurations: Plans and situated actions,Cambridge University Press, 2007.
* Donna Haraway, Introduction : A Kinship of Feminist Figurations, The Haraway Reader, Psychology Press, 2004
interviews:
* Interview conducted with Dr. Hannah Limerick, User Experience Researcher, Ultrahaptics, Bristol on 12/4/2018.
* Interview conducted with Dr. Noëlle Mercier, Neurologist CHUV, & Dr. Gilles Allenbach, psychiatrist CHUV, Lausanne, Switzerland on 12/8/2018.
Websites:
* www.ultrahapics.com
* https://woodfordfunds.com/
* https://www.feeliom.com/