Rumors from Cyberworld
This is a project called Rumors from cyberworld. It is a real-time face captured with the data visualization project wrote by p5js and also involved with machine learning, face tracking and sound.
produced by: Mengdi Li
Introduction
This project is inspired by Internet rumors and the pressure from the social network, talking about the impact of virtual rumor especial cyber violence towards physical human beings. In continuation to the middle WIP show, where I used a website game called Cyber Forest to demonstrate surveillance and soft violence, this time I focus on the harmful rumor from websites and social media based on the monitored cyber world.
The Background
The original intention of this project starts from a piece of news. China suffered extreme weather in July this year. After the natural flood disaster in Hunan province, a photo was circulated on the Internet. In the photo, a man is sitting in front of a subway station, wearing a dark blue raincoat, black sunglasses, and a mouth mask. A piece of cardboard stood on the bicycle parked next to him, and ten words are written in black ink, "Niuniu: Dad still wants to take you home."
Many people sympathized with the father’s experience, but his wired behaviour was questioned and hyped by netizens, they thought the photo was difficult to judge the degree of truth. Some netizens also pointed out that this photo was “hyped”. The reason was that in the official list of Zhengzhou subway victims, the youngest was also 20 years old and had no minors, and “Niu Niu” sounds like calling for a little girl. However, in the Henan dialect, elders or people generally call their daughters “Niuniu”. “Did you really lose your daughter?" "Do you want to use weird behaviour to get attention?" Do people need to speculate everything to reach the “truth”? It is a heartbreaking thing for a father to prove the death to other people in order to face the overwhelming questions from the cyberworld.
The Concept
I got inspiration from an artist called Jon Rafman. As he mentioned in his project Nine Eyes of Google Street View, everything from the world could be recorded by a moving car. Google street view provides an effort to document everything in the world. Street view cameras, which record potentially violent and illegal activities but have no moral standards or judgmental objectivity. The camera on this car is supposedly neutral gaze. It provides a world as “You are stared at by everyone but at the same time, no one is really watching you.” Surveillance is an ongoing systematic collection, analysis, interpretation and dissemination of data regarding a health event for use in public health action to reduce morbidity and mortality and to improve health. (CDC) It is a passive process while collecting data. The function of surveillance is like human eyes. Therefore I am inspired to use the camera as the eye of the computer that is supposed to be neutral but was given prejudice. The viewer is prompted to show their face in front of the camera, the data of images on the screen is extended from only video buffer to diverse demonstration.
The Technology and Design
This project starts based on this event, talking about what is rumor and why people spread rumor. There are two main layers shown on the screen: images and texts.
What is Rumor? It is an unofficial interesting story or piece of news that might be true or invented and quickly spreads from person to person. Nowadays, It is disseminated through social media. Judging from the content of current Internet rumors, there are mainly slanders directed at individual citizens and fabricated against public events. They are concealed, speculative, offensive, retaliatory, seductive, and compulsive. Anxiety and uncertainty mood is a breeding ground for rumor. Based on that, I made some unpleasant sounds and shaking images to increase the feeling of anxiety. Rumor is a currently circulating story or report of uncertain or doubtful truth. Therefore the left screen monitor shows the interweave of error pixels with the real-time images from the web camera. On the right half of this screen, I associate the images with time information and split them into different fragments.
About the truth. Like the Japanese film Rashomon mentioned, due to the complexities of human nature, the truth is always multidimensional and diverse. This film is talking about four people recount different versions of the story of a man’s murder and the rape of his wife. As the facts are difficult to understand quickly, rumor offers a clear point of view for people to understand the world during the process of information exchange, which means rumor could address a lingering uncertainty from the chaos. Firstly people cannot see the story behind it, due to the spread of rumor, people could gradually see it. Like the myth or folktale, rumor often becomes the first stage of people knowing a story. Here I used Mosaic shader to extract the pixel information to demonstrate the simplified process.
Another possible reason for people to spread rumor is to help their social status and boost their social power. These mainly happened during influencers. Here I am using the Instagram surface as a symbol. Under the uncertainty, no one could judge if it is a rumor or not. Rumors will blind people who don’t know enough stories about the truth. Therefore I made an Instagram filter based on face tracking and calculate the centre point of the face.
The project here is trying to evolve participants as part of the network information, see the data in different ways and what is their reaction once they find themselves in the centre of rumor in the cyber world. I got the comments from the social network about the ‘raincoat daddy’. On the right computer, the scrolling texts are the original and translated version of comments about ‘raincoat daddy’. In the right part of the screen, the serial of shifting float number is an array of vectors, which was generated by the Cross-lingual (XLING) Universal Sentence Encoder model. I fed the comments into this model and these vectors are the output. It represents how computers and the virtual world understand sentences. On the left side with the dark background, these two looping videos are the visual way of computer or artificial intelligence’s view of understanding, which are generated by Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks (AttnGAN) model. It could turn text descriptions of scenes into synthesized images.
The decoration of this project aims to imitate a corner of daily life with uncanny feelings. Normal desk with iMac, quite familiar but kind of different feeling. The light from the table hits the person’s face directly, just like the spotlight of a stage. Participants’ data are used by the computer and shown in different ways from readable text to images and vectors.
I want my participants to have a general feeling if they become the centre of the rumor. How does the internet twist the information about the shape of your body and the trail of the behaviour, what kind of information will participant remain to the cyberworld and what kind of reaction people may have, and what might be the understanding or view of artificial intelligence? Except for the face filter on the phone, what’s the feeling if people see their face on the show.
Self evaluation
I quite enjoy the process of exploring a word. Participants can be attracted by the images and themselves on the screen. I was planning to publish these two websites online, however, due to the limitation of school WIFI, I gave up this plan. For the future plan, I’d like to make this project to an online stage with this webcam capturing function, which could involve more participants and gather their feedback easily.
References
Clmtrackr: https://github.com/auduno/clmtrackr
Cyber Forest (Mid WIP project): https://www.lillianlimd.com/post/cyber-forest
GAN Learning: https://dev.to/serokell/top-18-low-code-and-no-code-ml-platforms-5981
Namespacing Tutorial: https://www.youtube.com/watch?v=Su792jEauZg
Nine Eyes of Google Street View (2008 - ongoing): https://anthology.rhizome.org/9-eyes
Machine Learning Testing: https://mikeheavers.com/tutorials/free_cloud_gpu_model_training/
P5JS Library: https://itp-xstory.github.io/p5js-shaders/#/./docs/continue-learning
Rumor Defination from Cambridge Dictionary: https://dictionary.cambridge.org/dictionary/english/rumor
Rumor Surveillance: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3298271/
Runwayml learning: https://dev.to/jochemstoel/runwayml-next-generation-machine-learning-for-creators-1b4p