Who is at the Door?
Who watches you? Who are you watching? The Metropolitan Police in London are currently testing facial recognition without our consent. Corporations monetise our behaviours in order to predict and influence our choices. The ever-tightening knot of surveillance is panspectric, fluid and endless.
produced by: James Tregaskis
Introduction
My interest in surveillance has been a consistent theme during my MFA. In the second year, more and more examples came to light.
Approximately a year ago, our group of houses were replacing the video entry phone system and I seized the opportunity, re-purposing them for further experimentation. I planned on developing them for a project on surveillance. The phone-intercoms contain a ‘flattened’ cathode ray tube (CRT), a small loudspeaker and a handset with switch on the handset cradle and three buttons on the front panel.
Concept and background research
For the project, repurposed, obsolete home intercoms were modified in orderto watch ourselves and each other. I wanted to explore the possibilities of combining old hardware, modified to represent aspects of surveillance and re-imagine how they might be used.
The final assembled installation is constructed from rough timber, treated for outdoor use. The timber is warped and as a result, fits together badly. Each intercom is mounted each of the six vertical posts. Bolts hold the hexagonal structure together; given the unfinished appearance combined with the repurposed phones, is this how our world will look after total economic collapse? Perhaps the ‘people in charge’ (not necessarily a government) will have to recycle old equipment to keep ties on us. Bentham’s watchtower is reversed; does it undertake panspectric surveillance now, rather than panoptic? Machines watch other machines for no particular reason. Fluid surveillance carries on indefinitely and pointlessly. Could this have been made by a lonely ‘prepper’, protecting his property scaring off intruders like a farmer’s scarecrow?
Massive changes to society are taking place at a bewildering rate; as Gabriel Tarde warned us in the 19th century, he foresaw a time where feelings, emotions and affect are spread frictionlessly in our ‘digital sea’. Surveillance, for good or ill is part of this – society is vulnerable to asymmetric control, like never before, all due to mankind’s progress. Our Atlantis is upon us now.
Technical
The intercoms originally ran on 33V AC; this was to maintain a workable current over a long distance. I obtained a 33V transformer, upon connecting it up - I saw blue smoke coming out of the circuit board. I was left to think, what have I done!
In the end, I salvaged the cradle switch and discarded the PCB inside the unit altogether. The Cathode Ray Tube (CRT) runs on 12V DC and I included a small amplifier in conjubction with a Raspberry Pi to control the unit(s). In all, I had 6 working intercoms having tested the CRTs with a 12V power supply. I used the video output from each Raspberry Pi.
The original loudspeaker inside each unit were bulky, and I replaced them with a tiny flat 8 Ohm version. In order to make the sound heard from the audio jack on the Raspberry Pi, I used a small 5V amplifier which was mounted inside the back of the intercom cover.
The first version was exhibited at the popup in December in New Cross Road. I included a vintage 1983 Minitel which displayed Tweets based on surveillance-related search terms. The intercom screen showed a loop of CCTV sequences I obtained from www.insecam.org, recording them on my laptop. Live CCTV from Insecam.org was possible but most of it was relatively dull so I opted for recorded footage. MPlayer was used to display the videos via an externally mounted Raspberry Pi running Raspbian.
Following on from this early inception of the project, I found a way to mount a Rapberry Pi 3 Model A+ (a smaller form factor version of a Raspberry Pi 3 B) – I soldered both audio channels, video and power directly to the back of the Pi’s board to save space.
The next showing of the intercoms was again in our next Popup exhibition, I had all six intercoms running, each with its own implementation on the theme of surveillance. This included:
1. Software defined radio (SDR) digital radio tuned into FR24.com and configured to display aircraft call sign and altitude located over Heathrow airport. The data was mediated via radio frequency wireless and decoded via an SDR analogue to digital decoder attached via the USB port on the phone.
2. Using a camera attached to the phone, a program using OpenCV library written in Python captured people’s faces and eyes in real time.
3. (and 4.) Videos showing DDR archived footage of East German police and excerpts from Xu Bing’s CCTV feature length film ‘Dragonfly Eyes’. The video would switch to another video (e.g. CCTV from www.insecam.org) when the phone cradle was lifted. I used Python with Will Price’s module OMXplayer wrapper (https://github.com/willprice/python-omxplayer-wrapper) to control OMXplayer which uses the Pi’s GPU and is more stable than MPlayer.
5. Twitter – surveillance search terms in real time using Python and Twitter search API.
6. Stop and Search data from Metropolitan Police website. I used data from three months previously, obtained outside near Southgate Tube station. Python was used to request the data using the Metropolitan Police API. The text was slowed down considerably in the code to make it easier to read.
In the first iteration ealier in the year, the intercoms were not connected in any way with each other. The phones were shown in the small shop-gallery in New Cross and placed throughout the space.
I went on to integrate them and explore further possibilities by adding proximity detectors, audio and more cameras.
Final Installation
The third iteration of the intercom project for the final September 5th 2019 exhibition continued. I opted to assemble interface boards for each intercom in order to connect a HC-SR04 sonic proximity sensor. The HC-SR04 is extremely inexpensive and will reliably detect a human from up to 2 metres away. I opted to mount one on each intercom. Secondly, I tested and added the amplifiers in each phone.
The installation used a locally running Wi-Fi router (HooToo Tripmate Titan) with each computer having a fixed IP address. The Tripmate connected to external internet provided by the university (G11 ssid) Wi-Fi.
Each Raspberry Pi had a standardised configuration of GPIO connections, wired up via a pair of female headers. I wanted to refer to the GPIO pins in the same way for all machines in the programs I was running.
In addition to the Pi-attached camera, I deployed 3 Raspberry Pi Zero’s attached to a small camera. Each Pi Zero ran MotionEyeOS operating system, written by Calin Crisan. The streamed video from each of the MotionEyeOS units could be accessed via a fixed IP address running on my local Wi-Fi network. Each Pi Zero/Camera assembly was mounted on a 3D printed enclosure and mounted inside the CRT enclosure, a tight fit, but it worked.
I printed a shroud in order to retain a Perspex window at the front of the CRT electronics. I wanted to expose the inner workings of the machine to emphasise the modifications I was making on the intercoms. It was also important to protect anyone from the high voltage components inside the CRT enclosure. The power board for all the intercoms was left exposed as well for the same reason, this was mounted on the gallery wall, high and out of reach.
Another component was incorporated into the installation. As in the previous year I came across an interesting aspect of surveillance; harvesting Mac addresses from any Wi-fi in the area.
I wanted to incorporate another aspect of surveillance I had come across in the previous year but had not implemented: Ray Burnette had written (and others have contributed) to a project where an ESP8266 board is used to scan any Wi-Fi on any channel, in unmanaged, promiscuous mode. I used a Node MCU ESP8266 to capture the data and then ran a Python program to strip out and parse the Mac addresses. The Mac addresses were stored in memory and any new ones sent on to the other Raspberry Pi’s using RabbitMQ. Listener program on other Pi’s speak out the mac address (using espeak).
Another listener Python program runs on two other intercoms to alternately switch camera views from the Pi Zero cameras. The listener program use virtual alt-tab keypresses to switch tabs in the browser displayed to change camera view, found at each fixed IP address.
On another intercom, I combined the Twitter API (tweeting one of many random sureveillance themed words) with Metropolitan Police stop and search data API and also the ESP8266 newly appearing Mac addresses. The RabbitMQ server ran on this machine as well, to handle the messaging among the intercoms.
OpenCV facial recognition was running on another intercom, each time a face was recognised, it was appropriately labelled with a surveillance-related text next to the face it found.
On the other two intercoms, I retained the surveillance-themed and CCTV videos reacting when the telephone handset was lifted; a plane crashes, the phone is lifted and we see a peaceful Japanese beach.
The 12V and 5V power was supplied by a collection of power supplies, mounted on a piece of MDF. Care was taken to install a fuse (12 total) for each power line. I had already experienced a short-circuit while building the installation in my studio and I wanted to prevent any problems, given the number of cables being used. Originally I used telephone bell wire but later replaced it with 3A mains flex which proved more durable. The large number of power supplies derives from the fact that they were designed for 12 screens originally. I decided to leave the power set up as it looked more imposing.
Below is a high-level data flow diagram of the devices installed for the exhibition. The proximity detectors, cradle switches and buttons initiate messages that in turn trigger sound files derived from listener programs (written in Python) from speakers in the phones. I found one speaker to be quite loud enough in the gallery where the installation was shown, although earpieces in the phone handsets output audio as well.
Code for project can be found at: https://github.com/jtreg/who_is_at_the_door
Influences
Trevor Paglen
Paglen in his photography reveals GCHQ, NSA buildings, satellite dishes and undersea cables. He articulates the unseen or non-obvious physical nature of our panspectric (Tardean) world. His work is not computational but has shone a light on how far surveillance has driven into the built environment.
Hanun Farocki
The late Hanun Farocki stands out as a supreme essayist and filmmaker. Is essay-films and later gallery-based video work reveal an academic and political investigation of watching at a distance. I was fortunate to be able to watch many of his films, including Images of the World and the Inscription of War (Bilder der Welt und Inschrift des Krieges) (1989), Prison Images (2000), Eye Machine Series (2003). He uses ‘found’ CCTV footage, industrial training films, archived film clips and home movies to build his essay-films. ‘Ich glaubte Gefangene zu sehen (I thought I Was Seeing Convicts’), Farocki shows how customers are tracked in supermarkets, just as how prisoners’ movements are automatically monitored in the prison yard. Farocki anticipated many of the concerns with privacy and its political dimensions, being explored by contemporary artists.
Laura Poitras
Laura Poitras in addition her documentary on Edward Snowden, Citizen Four, has also worked in the gallery space on issues of surveillance. Her work, Astro Noise first shown at the Whitney Museum, presented a sequence of projections, real-time video and surveillance related artefacts that remind us that we are the ones watching and at the same time, the ones being watched. Laura Poitras work expresses the ideas of who watches who, an aspect of my installation I have worked towards as well.
Xu Bing
Internationally acclaimed installation artist Xu Bing and his assistants culled through approximately 10,000 hours of surveillance videos sitting in the cloud and assembling them to make ‘Qing ting zhi yan’ AKA ‘Dragonfly Eyes’ (2017). I extracted tiny clips from the film as part of the installation. The fictional narrative flow of the film is somewhat contrived; however, the imagery is dramatic and posits questions of privacy framed in a Chinese context.
Jill Magid
Magid before working as an artist already had worked as a spy, police officer and war journalist. She surveilled the workings of the Dutch Secret Service (Algemene Inlichtingen en Veiligheidsdienst) to create the project ‘Authority to Remove’. Magid’s work tests the boundaries of what aspects of surveillance material is permitted to be shown. She also reveals the inner working of professional ‘surveillance workers’ in the intelligence service arena. Her commitment to drive deep into the subject is impressive to me.
Julian Oliver
Before I started my MFA, I have came across Julian Oliver’s work in the Framer Framed gallery in Liverpool. This exhibition featured many pieces based on the artist as trickster. The work I saw there was Newstweak. This ambitious work demonstrated how with the use of elegantly thought through networking and hardware he was able to subvert information flows by intercepting data traffic to nearby Internet users and alter their news flow, creating fake news.
During the computational art course, I was alerted to his other work, including Men in Grey, Transparency Grenade and Border Bumping to name but a few. I love how he modifies existing physical objects into thought-provoking comments on surveillance. I attempted to do this myself by modifying the intercoms. Without question, in my opinion, is the most important artist alive today working to combine political critique with ingenious artefacts and he Is fluent use of information technology/ networking.
In the research for my extended essay, I spent much time reading and examining work by other artists too numerous to mention here. It provided me with a deeper understanding of the subject, also revealing what a huge subject surveillance is. The subject material is so wide that degree courses could be solely focused on this area of study. My challenge has been to distil down my understanding of the subject into a working art project.
Future development
As I explain below in my self-evaluation, I would make the RabbitMQ configuration much simpler and reduced to perform the overall choreography of the interaction easier to understand. Also, the use of video projection might help to augment the tiny screens and make the material more readable/coherent.
There are ample ways of expressing how we are being surveilled. One direction the piece could go is to use sensory devices. Infrared heat sensing cameras, voice recognition, environmental data API’s to name but three.
Regrettably, I had another six screens which, due to their old age failed to work. I had made bespoke Perspex enclosures for them to be held in. I was saddened, because they looked lovely, but I could not justify setting them up in the gallery. I would love to have more of these old recycled units. They are still being made in China, but the minimum order seems to be 100 units. Imagine the gallery with many many more mounted on one wall!
The scope of the surveillance theme continues to expand and pervade our world experience and does not see any sign of abating. The challenge for me would be how to express the massive and ephemeral aspects of its effects inspired by the Internet of things and developments in social media and the synergy of big data and machine learning.
Self evaluation
Earlier trials using Open Sound Control (OSC) became harder to implement with my planned data flows, therefore I changed the Python code I had written with OSC to use RabbitMQ instead. I was able to write subscriber and publish programs on each Raspberry Pi to react to button presses and proximity messages from the HC-SR04’s.
The OpenCV facial recognition piece worked consistently well, I feel the audience appreciated its irony. The coding for this was fairly simple but very effective. The use of video controlled by interactivity with the handsets also was well received. These component parts were not dependent upon messaging and could be independent upon throughout the exhibition.
I also used an esp8266 to catch newly arrived wi-Fi devices that pinged out Mac addresses. This is manifested in announced audio generated by Espeak. This added some intrigue to the surveillance team however I doubt if any visitor understood the implications or meaning of the spoken Mac address.
However, much of the piece required RabbitMQ messaging to link multiple event connections from the buttons, cradles and HC-SR04 sensors. This let me down during the Viva. I restarted the RabbitMQ server and it worked again afterwards. If I had more time, I would have refined the messaging system to send one to one messaging links rather than the 'fanout' option where Messages are sent from one to all of the attached intercoms. This this would have reduced messaging traffic considerably and not overload the overworked raspberry pi that the RabbitMQ server was running on.
The Twitter and Metropolitan Police stop and search data worked reliably, and I felt it gave was a useful rich contribution to the surveillance theme.
The self-imposed handicap of using small monochrome screens inside the telephone intercoms limited the clarity of what I was able to show, however it was intended to remind the viewer of the past and how we have come in a short time. Unfortunately, one of the screens had reached the end of its natural life, the picture was distorted and we could not see the videos I had prepared on this unit, although the (East) German soundtrack of the film could still be heard.
Included in the installation were three tiny cameras continually monitoring the surrounding area, the watching browser(s) were controlled by interactive messaging with RabbitMQ. Perhaps improved choreographing and thought put into the camera interactivity would have been another area to work on.
Overall, I was proud to present the ontology of surveillance, expressed not only via obsolete apparatus but also inject into the piece contemporary aspects of more recent developments in techno-surveillance.
The installation also contrasts A rough DIY construction combined with a delicately built set of electronics. I was happy with this paradoxical presentation of the work.
Another aspect of the presentation is how an artist's own psyche can be decoded in the way the installation/creation manifests. How does this solitary, detatched figure as the tower, the sentimentality of the old apparatus, the obsessive attention to the smaller components and the cavalier, perhaps makeshift supporting structure correspond to my inner conversions? Can we decipher the intentions of the artist when viewing the piece as a whole? I can only ask these questions to the viewer, as I have asked them myself.
My blog: Tregaskis.net
References
MotioneyeOS:
https://github.com/ccrisan/motioneyeos/releases/tag/nightly-dev
ESP8266 Sniffer:
https://www.hackster.io/kosme/esp8266-sniffer-9e4770
https://www.hackster.io/rayburne/esp8266-mini-sniff-f6b93a
OpenCV:
https://www.learnopencv.com/install-opencv-4-on-raspberry-pi/
https://pythonprogramming.net/garage-stoplight-raspberry-pi-tutorials/
OMXPlayer Wrapper:
https://python-omxplayer-wrapper.readthedocs.io/en/latest/
http://www.insecam.org/
https://www.Rabbitmq.com
Paglen:
https://www.theguardian.com/uk/2013/jun/21/gchq-cables-secret-world-communications-nsa
Farocki:
Harun Farocki’s Bilder der Welt und Inschrift des Krieges (Images of the World and the Inscription of War), 1989.
Poitras:
Astro Noise a Survival Guide Laura Poitras Pub Whitney Museum of American Art
https://en.wikipedia.org/wiki/Citizenfour
Bing:
http://www.xubing.com/
http://www.imdb.com/title/tt6576482
Magid:
https://www.tate.org.uk/whats-on/tate-modern/exhibition/level-2-gallery-jill-magid
https://www.stroom.nl/activiteiten/tentoonstelling.php?t_id=6430733
Oliver:
https://criticalengineering.org
https://julianoliver.com/output/category/projects
https://transparencygrenade.com/
Other:
Electronic Frontier Foundation http://www.eff.org
Picard, Rosalind W. https://www.pwc.com/us/en/technology/publications/assets/pwc-wearable-tech-design-oct-8th.pdf. Affective Computing Cambridge, Mass.: MIT, 2000.
Privacy, Big Data and the Public Good Frameworks for Engagement, Ed. Julia Lane, Victoria Stodden, Stefan Bender, Helen Nissenbaum Cambridge University Press 2014
Total Surveillance John Parker Investigating the Big Brother world of e-spies, eavesdroppers and CCTV Pub. Piaktus 2000
Identifying Citizens ID Cards as Surveillance David Lyon Pub. Polity Press 2010
Database Nation the Death of Privacy in the 21st Century Simson Garfinkel Pub O’Reilly 2001
Supervision and Introduction to the Surveillance Society John Gilliom and Torin Monahan Pub University of Chicago Press 2013
Astro Noise a Survival Guide Laura Poitras Pub Whitney Museum of American Art
Feminist Surveillance Studies Ed. Rachel E Dubrofsky, Shoshana Amielle Magnet Pub. Duke University Press 2015
Discipline and Punish the Birth of a Prison Michal Foucault Pub Penguin 1977
Surveillance as Social Sorting Privacy, Risk and Digital Discrimination Ed. David Lyon Pub Routledge 2003
In the Age of the Smart Machine the Future of Work and Power Shoshana Zuboff Pub Basic Books 1984
The Age of Surveillance Capitalism the Fight for A Human Future At The New Frontier of Power Shoshana Zuboff Pub Hachette Book Group 2019
Taryn Simon An American Index of the Hidden and Unfamiliar (2007) Whitney Museum