1. Background
Plastic being is a body part replacement game with Head mounted display.
It began from the ideas of Transhumanism and Posthumanism. In the far future, as technology develops as humans become immortal beings. In the future I thought humans replace their old body parts to new one.
Gut is a ritual of Korean shamanism ‘Mu’. It is held in rite of passages to wish for good luck or comport. Gut is conducted by a shaman using talking, singing, dancing and some props. And one or more music players play traditional percussions such as Janggu and kkwaenggwari. There are various kinds of Gut. One of most common Gut is for a dead person. When a person is dead bitterly or for certain personal reasons, family members take part in Gut to console the soul. Although the process and probes of Gut are so different depending on regions, the universal purpose of the Gut for dead people is that make the soul comfort mentally and let the soul leave this world and go to that world(the other world). In here, the shaman does a role to connect this world with the other world. I inspired by this Gut for dead people because I thought that exchanging body parts and living eternally is similar to death paradoxically. Even if an individual can live forever, the time and the look will make a completely different person. Although we have lived as one entity for our entire life, our personality and values keep changing in some extent. The ritual to make comport for the dead person becomes the rite of the new life in this project.
2. Project description
Plastic being is a virtual reality performance game. It consists of one performer who is the Oculus rift user and the screen projection of one avatar who moves along with the Oculus rift user’s body. Since Oculus rift is a head-mounted display(HMD) for only one person, the screen projection is added for the audience. This project is expressed as a kind of rite of passage where the human earns a new body, free from the old or obsolete body, and the other spectators symbolise the audiences of the ritual like in an event such as a funeral or wedding ceremony, that usually family members and close friends attend.
On screen, the avatar represents the HMD user’s new body and both the HMD user and avatar perform together. For this effect, the screen will be translucent and made of netting fabric. This is done in the same way as the Diesel fashion show(2007) and the concerts of Hatsune Miku, using a net screen for the virtual character along with lighting for a real person.
The body replacement ritual is comprised of three stages. First, discard the old body. The old body could be the original body or an old artificial body. This is like a coming of age ritual or a wedding or another significant life-changing event, the changing body parts give the user a different set of abilities and a different look that has a direct effect on their personality. When the user wears the HMD, the person sees an avatar which represents their new body. Second, the highlight of the ritual, the HMD user replaces her or his body parts through a dynamic selection process. As they move, new body parts are chosen and the movements produce music as part of a ritualistic performance. On the screen, the ceremony is held as a performance where the new body and the old body are mingled and crossed with each other, and finally the old body is replaced by the new body. The person now has gained a new identity from their new body and will continue living their life. However, that life can continue changing. The key point of this work is not getting a set of new body parts. I want to stress on the continuous transition. Therefore, the ceremony ends calmly dimming the lights out instead of focusing on showing the new body in the last scene.
The reason why I choose Virtual reality is that VR is the technology to be able to embody impossible experiences such as body replacement.
3. Process
3-1. Character design
If you can exchange your body parts, what kind of design do you want? People can simply choose just a more beautiful human shaped body or a body part that has certain functions which remove the need to carry all sorts of devices. For this project, I designed five characters to adapt and survive in special situations that may occur far into the future.
1. The standard body which is identical with typical humans. It is almost the same figure as present human beings but it doesn’t have external/internal genital organs and breasts for feeding. The future human does not reproduce the natural way. Every human(or every being which has an intelligence as much or more than humans) is born in an artificial uterus.
2. Amphibious body to live in deep oceans and on the earth. Inspired by the air bladder of fish, inflatable/deflatable body can move into the water or high-density liquid. Arms are able to adjust the length and thin layers between long fingers and two legs perform as fins.
3. Balloon body to adapt to any gravity. Someday, humans can go to the other planets and may have to settle down on a totally different planet from Earth. In heavier planets, it can reduce its weight as the balloon-like head decreases. It can reduce its body size by rolling up the body in a ball inside it’s head. Besides, the flexible spherical head with a thin and long body can help it to live in gaseous planets such as Jupiter.
4. Skein body to survive any kind of accident. In a crash or collapse due to earthquake, the skein of thread body is composed of durable, sturdy and very tough thin threads which stay intact like threads of a rope under pressure. The threads embody arms, legs and hair ordinarily, and if needed, they can rapidly transform into other shapes.
5. Hard skinned body like beetles. The shell protects the body from external shock. The body centre is the energy store and is created from a soft core which digests the food and is visible externally. When in a protective mode, the body centre is compressed like a sandwich and enclosed by the upper and lower shell structure.
Based on the above concept, I modelled the characters with Maya and then make them have the same bone structure as the bones of the avatar in the Kinect SDK that I downloaded from the asset store.
3-2. VR environment
Since the virtual world’s environment is the special environment designed for the body replacement ritual, I picked 8 colours from Gut, and created a target shaped circle floor made of those colours. To create a ritualistic space, the structure of the room, a dome, was selected. The texture of the dome is made of a reflective material and there is a mirror ball of moving light to make the space feel splendid.
3-3. Music
Inspired by ‘Gut’, each body part has different sounds which are from the musicical instruments used for Gut. Music is an important part in Gut. It is a sort of composite art comprised of singing, dancing, playing instruments and visual arts. The most common musical instruments in Gut are percussions such as Janggu(double-sided drum), Jing and kkwaenggwari. Instead of using those instrument sounds directly, I made the sound using Ableton. Downloaded and tested many different percussion sounds from freesound.org and modified the sounds to be similar in mood to the instrument sounds of Gut. I intended the sound in my work to be cheerful and exciting to express the joy of earning a new, longer life but not too similar to the actual traditional Korean sound.
3-4. Body replacement
From the beginning, I tried to avoid a UI system for picking and replacing such as touch or pointing in VR so that people could enjoy it and play it very easily without any learning or practice. Technology is becoming part of us like our own organs. I believe easy technology is the better technology. Therefore, to be able to learn how to play or control immediately, I made that process of learning like playing.
Once the user moves their body, the avatar’s look immediately reacts and changes. And the change is going on for 90 seconds. I assumed that 90seconds is the minimum time people would play before they felt bored.
3-5. Exhibition space design/Mask
The space design is an important part in this project. I want people to experience my work as a whole from the moment they come inside the room to when they leave. To do that, I’m going to put a mirror sheet on the door window. Since this project is about the human’s look, I want people to see their pre-posthuman look in the mirror before they come inside. To connect the VR experience with the real world, I decided to make the same floor as in the VR environment. I found some plywood and drew concentric circles which I then masked and sprayed with vibrant spray paint durable enough to be stood and danced on by the public. I made the ending of the VR experience fade to black to match the real world darkened space. To secure and gain more room, PC and Oculus rift will be installed on the wall.
Lastly, the HMD is the device to connect the real world and virtual world but it breaks the illusion as well. To maintain the illusion, I’m going to make a mask to hide the device with 14mm EVA white foam. It is foam used by costume play fans. The mask will be a helmet shape to cover the whole head and inside the helmet, the Oculus rift covered with 2mm foam sheet and attached with masking tape.
For the translucent screen, I chose the fly mesh. I tested organza fabric and a few other mesh fabrics. Organza was almost perfect for this type of projection, but it looked too shiny. So I have bought insect mesh and am waiting for delivery. Unless the projection images look good. I am going to use the less shiny fly mesh even though it seems to have a lower resolution.
4. Challenges
The biggest challenge was that there were many jobs to do by myself. This project contains many parts. Creating a character like Patricia Piccinini’s artworks or making the future world can be one complete artwork in itself. Also, coding in Unity3D and 3D modelling and animating in Maya were hard jobs for me. Fortunately, the Unity asset store was very helpful. I used assets that were already made by other developers and creators.
I made a mirror and a reflective surface dome and floor. and a few lights. Those things made the program heavy. Particularly, reflection probe for shiny and reflective wall on the dome and the mirror. It was not much different even if I used ‘bake’ mode instead of ‘real-time’ mode. So I adjusted the reflection conditions to their minimums. The worst part was that I had to give up using reflection probe for a mirror, which made almost perfect reflections. Although I found a shader and script originally made for water reflection and then modified for a mirror, there were no shadows in the mirror. I changed the mirror to a camera display rendered into a texture because I searched a lot but I couldn’t find a realistic mirror reflection.
Coding an avatar movement with the Kinect camera sensor was also not as easy as I thought. I used Kinect v2 SDK. It has its limits. For example, when I made the character models, there were limitations to their designs to fit the SDK’s humanoid bone structure and all characters have to have the same humanoid structure.
5. Technical implementation, algorithms & processes
The character models where built from the concept art designs using polygon modelling techniques. After the models where built, they were then rigged with a humanoid skeleton ready to be used in Unity with Unity’s mecanim animation system. The rigging was done using the advanced skeleton system 5, a free plugin for Maya.
After rigging, some basic animations where created to give some life to the characters using blendshapes. The blendshapes allowed me to do vertex deformation of the model and blendshapes are the only way Unity can read that kind of animation data.
Once the model file was done, I imported into Unity as FBX and connected it to the Kinect SDK asset from the asset store which controls the avatar movement and therefore the user's movement.
The models' arms, legs, head and body were connected to a script which reads the speed of those body part movements and then creates either a wobble/distortion effect or a complete transition to a different body part. The movements also control which sound effect is played.
At the beginning, I had used an mesh-modifier-asset for a wobble/distortion effect.(https://www.assetstore.unity3d.com/en/#!/content/644)
It worked very well with only two characters, but when I added more characters, it got so slow because this asset works in CPU. So I decided to use Shader which works in GPU. I used Shader Forge to make a wobble/distortion effect.
6. Future development
Since this project began from a broad topic, future human beings having immortality, I thought about so many things regarding future humans and society. One of them is if we can replace our body parts, what is ‘I’, what makes identity as an entity?
I would like to explore the effect alteration has on identity and it’s influence through alteration of visible characteristics such as height, race, gender and so on.
In terms of the technical side of the project, I felt the possibilities and limitations of virtual reality and a head mounted display(HMD) used at the same time. There are many things to research in this field, and I want to study ways to overcome the limitations of current HMDs and extend the HMD experience.
Reference
Hatsune Miku Live 2011 https://www.youtube.com/watch?v=YSyWtESoeOc
Cosmos, Carl Sagan
http://scienceon.hani.co.kr/406294
Kinect v2 SDK https://www.assetstore.unity3d.com/en/#!/content/18708
Kinect v2 Tips, Tricks and Examples
Code
https://github.com/Eungyeong/plasticbeing