Cloud AR
The Cloud is a virtual vessel of deconstructed automatic drawings, ripped diary pages filled with fear, anxiety and the dread of isolation. The portal to the Cloud is Takotsubo, a self-portrait in physical form that carries a QR code, the key to the Cloud.
produced by: Noa Geras
Introduction
The Cloud AR has its origins in a series of ink drawings, inspired by sumi calligraphy, created during lockdown using automatic drawing techniques.
The need to create this series came from the book Man’s Search for Meaning by Victor E. Frankl. As he states: “Everything can be taken from a man but one thing: the last of the human freedoms — to choose one’s attitude in any given set of circumstances, to choose one’s own way.”
Each drawing has a personal meaning behind it and serves as a public diary, recording a time of global anxiety, isolation and fear. Creating something of meaning makes us endure, and ultimately become resilient to hardships.
The drawings are sewn through with red string, which references book binding and the ritual of wearing a red string from the Kabbalah, as a protection from the “evil eye”. The drawings serve as a backdrop, and as a starting point to the physical object Takotsubo and the augmented reality Cloud.
Concept and background research - Takotsubo
During the lectures Physical Computing 1 and 2, taught by Professor Jessica Wolpert, I transformed the meanings of the drawings into a sculpture named Takotsubo. It is created out of a found cage around through which I weaved cut up tights I wore during lockdown. The inspiration for trapping the cage in a net of tights came from the practice of weaving cloth by Chiharu Shiota (State of Being), Tomás Saraceno’s augmented reality spiders (Webs of Life) and his real-life spider web installations (14 billions) and Cornelia Parker’s Cold Dark Matter.
Inside of the cage is a creature resembling an octopus that moves with the help of hidden servo motors. It is surrounded by miniature ink drawings that shake with the creature’s movements. This physical part of the project is named after a heart condition called takotsubo (Broken heart syndrome) which is also the name of a Japanese octopus trap.
On the creature Takotsubo - stuck in the centre of the cage, there is a QR code that serves as a key to access the augmented reality Cloud. The Takotsubo creature moves calmly when there are no people surrounding it. When a person puts their hand into the cage to scan the QR code with their phone, it gets alarmed, detecting the distance between the person and itself with its “eyes” (ultrasonic distance sensor), and begins to move rapidly. In order to access the Cloud, one must first stress the physical part of the artwork.
Technical aspects - Cloud AR
The Cloud AR is an immaterial and invisible artwork that the user can access from anywhere.
When the user scans the QR code on the Takotsubo, their phone directs them to a webpage - noageras.github.io, where they access the Cloud by following instructions. Their phone starts tracking the environment around them and it places the Cloud into the space. The augmented reality Cloud can be taken anywhere with their phones, used by anyone and at any time. The QR code becomes a sigil that the user can take home and possess. It is an artwork with a multitude of owners, free to use and access. The QR code is the key to the Cloud, to an augmented reality that is not bound by physical markers. In order to achieve that, I decided to use three.js to code the Cloud AR.
The Cloud is created using scanned elements from my lockdown drawings that were used as texture to a 3D sculpture made in Maya. The 3D sculpture was divided into three object groups (base lines, detail lines, small details). The objects were rigged, and five poses were created for the animation.
Into a WebAR project in three.js, the objects were imported and random timings for each pose were animated. This made every entrance to the Cloud AR a unique experience. By tapping the phone screen, the user can move the augmented reality Cloud where they please.
In the Cloud AR the individual memories and meanings of the automatic drawings are lost and transformed. The figurative aspects of them get destroyed and deconstructed to their bare line elements, thus forming the Cloud.
The anxieties that were once intimately connected to the automatic drawings lose their strength in the virtual realm. The Cloud is public, controlled by code, which creates mirror images of the drawings - trapped in a precise world, frozen in a never-ending movement. This transformation symbolises the loss of meaning that occurs when an object is placed in the virtual world, rather than in comparison to its physical form. If one tries to move images from the virtual world back into the physical realm, nothing becomes of them - only black and white code. To the human eye, this binary code becomes unreadable and nontangible; the drawings are destroyed, lost and bound without meaning in the stygian void of the Cloud.
The Cloud AR shows the inequality at the core of augmented reality technology. Without the right kind of device, the Cloud stays invisible. The only thing people see are the physical objects, the drawings on the wall and the Takotsubo sculpture in front of them. The virtual realm is the final frontier between those that do, and those that do not have the means through which to enter the Cloud.
Future development - Cloud AR
The Cloud AR is a project that evolves.
With augmented reality technologies advancing, the Cloud AR will feel more material, and its reality will feel more plausible. After posting a few photographs of the Cloud AR online, people wrote to me with questions on how I created this installation – they thought the Cloud AR was real. For me this is the base for future research.
Self-evaluation
The Cloud AR project started as the means of dealing with life in pandemic times. With virtual reality headsets not being safe for use because of Covid-19, I decided to work in a different technology rather than the one I originally started to explore.
AR was the perfect solution. I went back to my old sketch of the Cloud and realised that with augmented reality I would be able to achieve the things that once I could only imagine.
Instructions to access the Cloud AR:
Open a web browser in a phone and type: https://noageras.github.io/
- Android: In your web browser search bar, type https://noageras.github.io/, follow instructions and enter Cloud AR
- IOS: In your web browser search bar, type https://noageras.github.io/, follow instructions to download WEB XR Viewer, open and type in WEB XR Viewer search bar https://noageras.github.io/, click enter Cloud AR
References
- Viktor E. Frankl, Man’s Search for Meaning (Washington Square Press, 1985)
- Leonard Koren, Wabi-Sabi for Artists, Designers, Poets & Philosophers (Imperfect Publishing Point Reyes, California, 2008)
- Samuel Greengard, Virtual Reality (The Massachusetts Institute of Technology, 2019)
- The Twenty-First Century Art Book (Phaidon Press Limited, 2014), authors: Jonathan Griffin, Paul Harper, David Trigg, Eliza Williams
- Red String
- Chiharu Shiota, State of Being (Dress)
- Cornelia Parker, Cold Dark Matter
- Olafur Eliasson, WUNDERKAMMER
- Tomás Saraceno, Webs of Life and 14 billions
- Sound for video of Cloud AR created by Rutger Muller
Code used to create the Cloud AR:
- Three.js documentation:
- https://threejs-journey.xyz/
- https://threejs.org/examples/#webxr_ar_cones
- https://stackoverflow.com/questions/12626716/inconsistent-alpha-channel-in-three-js
- https://threejs.org/docs/#api/en/animation/AnimationMixer
- https://threejs.org/docs/#api/en/animation/AnimationAction
- https://threejs.org/docs/#api/en/constants/Materials
- https://threejs.org/docs/#api/en/renderers/webxr/WebXRManager
- https://threejs.org/docs/?q=gltf#examples/en/loaders/GLTFLoader
- https://stackoverflow.com/questions/63116505/three-js-webxrar-place-object-upright-on-a-wall-how-to-rotate-properly
- https://threejs.org/docs/#api/en/core/Clock
Software for converting fbx files to gltf files: https://github.com/facebookincubator/FBX2glTF - https://create.arduino.cc/projecthub/ryujenny3/servo-motor-ultrasonic-sensor-f951fe
- https://learn.adafruit.com/16-channel-pwm-servo-driver?view=all
I want to thank my friends Kevin Kuhn Agnes, Miia Remahl and Natsuki Hanazaki for their technical support with this project.
I am grateful to Dimitri Balojani, Samuel Turner, Sam Fisher and Nadia Geras, curators Fiona Irene Graf and Maja Pavlinić for the assistance with the theoretical part of my project.
Thank you to my professors at Goldsmiths for all your knowledge and advice.