Avatar
Avatar is an interactive projection, inspired by the idea of brain connectivity.
produced by: Gaytri Geeta Roopnarine
Introduction
Avatar uses geometrical elements of ellipses, lines and rectangles to create an interactive work that responds to the user’s fingers and hand movements. It could be used to enhance brain connectivity in users, particularly in the aging population.
This work is called Avatar because it refers to the soul and its interactions, sometimes like a pool of liquid, sometimes changing shapes, always connecting between its shapes but remaining the fundamental element of itself.
Concept and background research
Brain connectivity refers to a pattern of anatomical links, functional connectivity or effective connectivity between distinct units within the nervous system. The connectivity pattern is formed by structural links such as synapses or it represents statistical or causal relationships measured as cross-correlations, coherence or information flow. Neural activity, and by extension neural codes are constrained by connectivity and thus brain connectivity is crucial to how neutrons and neural networks process information(1).
Studies have found that video gaming can stimulate neurogenesis (growth of new neurons) and connectivity in the brain regions responsible for spatial orientation, memory formation and strategic planning, as well as, fine motor skills. Increasingly, the level of connectivity between brain areas are being linked to higher intelligence consciousness. (2)
However, most action video games are fast, intense, and unpredictable, emphasise peripheral processing, require selection between different action plans and might sometimes be violent. These characteristics make this type of games unsuitable for older adults. In addition, action games were rated as less enjoyable by older adults than non-action games(3).
Non-action video games have been used in older populations to increase cognitive performance and while those are based on specific tasks, I propose that an art activity which allows the user to be creative and not be obliged to perform in order to get a particular result might have stronger and more pleasurable benefits. It could also stimulate the alpha brain waves, inducing relaxation and possibly warding off depression.
I feel that this algorithm could be developed into an app that might be useful for older people who are suffering from memory or brain ageing issues such as early stage Alzheimer’s dementia or mental health issues. This algorithm has simple geometric shapes that creates a range of interactions from simple to complex which a user can focus on while at the same time increasing brain connectivity as he responds the colours, forms, movements and transformations that his fingers and hands initiate.
Casey Reas’s writing on software structures and his reference to Sol Le Witt’s method of wall drawings was an influence on my thinking process. Reas states that he often spends a few days writing a piece of technical code and then works with it intuitively as if a drawing, erasing, reshaping and holding the surface through instinctual actions(4). He cites Sol deWitt’s wall drawings as a defining factor in his process. Sol deWitt created a series of instructions for wall drawings which allows a user to construct the drawings independently of time or place.
The algorithm for this project is similar in that is a series of instructions that allows the user to create a drawing interaction but with the difference that each interaction is different depending on the user’s intuitive or conscious movement of his fingers and hands while reacting to the shapes on the screen.
Technical
This work used the Microsoft kinect sensor which is a peripheral device (designed for XBox and windows PCs) that functions much like a webcam. However, in addition to providing an RGB image, it also provides a depth map. Meaning for every pixel seen by the sensor, the Kinect measures the distance from the sensor and with the depth information, a threshold can be set. The algorithm can calculate the xy location in a given depth location. (5)
The near threshold used is 240 and the far is 220. When the kinect detects movement within this area it uses the motion to interact with the algorithm on the screen, creating new shapes and forms depending on the positions of the fingers and the hand. The user sits at a table and interacts with the kinect which can be located to the side of the screen or in front at a height.
The interaction is based on the Fiery Comet exercise given in class by Theo Papatheodorou. The sea sounds used in the interaction was recorded early one morning at the seaside in Athens.
Future development
In the future, a series of interactions could be created with different forms, colours and interactions in virtual space. It could scale in terms of difficulty and complexity allowing a range of implementations. Also the user could also have the option of getting a screen shot as a print as a memento and a reminder of the ‘artwork’ he was working with.
It would be useful to link to researchers in neuroscience and test whether this program might be of benefit as in the UK we have an increasingly large older population which requires a great deal of economic support. Art-based computer interaction which helps people to become less cognitively challenged and at the same time give satisfaction could be something quite worthwhile.
Self evaluation
While interacting with the work, I felt myself responding to several things at the same time. My hand movements sometimes created unexpected funnel shapes that morphed into barrel forms and then disappeared while linking to a series of coloured lines of varying line widths that crisscrossed along the surface of the screen. The side to side movements of the lines, the front to back reduction and disappearances of the ellipse shapes and the tiny movements of the green and yellow circles with the sometimes shadowy shapes of my projected body as it moved in and out of the camera range created connections that demanded attention. I felt that I was moving between areas of my brain as I paid attention to hand movements, the interactions they created and also at the same time, judging whether they were interesting or not. I also paid attention to areas of the mapped space that were interesting and the movements that created unusual effects and tried at the same time to memorise these interactions. These interactions created a focus which was quite intense and enjoyable at the same time. This feeling of enjoyment lasted for some time after interacting with the work. I am quite happy with the outcome, while at the same time realising that there is a lot of ideas to explore in the future.
This project uses a simple algorithm to create the shapes but I believe that this is one of the reasons that it can be used as tool in cognitive enhancement of older populations or even people who are mentally challenged as it gives results that are not complicated but easy and fun to interact with. Also the geometrical shapes are clearcut and induces a sense of order which works well with the other interactions so the brain has something to hold on it while it explores.
Initially I wanted to develop something based on Casey Reas’s Software Structure algorithm which I implemented as a stand-alone project to create ‘Butterfly’ :
https://youtu.be/dobZzKcIFlI
This is one of the programs I can link to the kinect in the future and have it as another alternative to Avatar.
References
1. http://www.scholarpedia.org/article/Brain_connectivity
2. https://www.psychologytoday.com/us/blog/the-athletes-way/201310/video-gaming-can-increase-brain-size-and-connectivity
3. https://www.frontiersin.org/articles/10.3389/fnagi.2014.00277/full
4. http://artport.whitney.org/commissions/softwarestructures/text.html
5.http://shiffman.net/p5/kinect/
http://openframeworks.cc/ofBook/chapters/image_processing_computer_vision.html
https://github.com/kylemcdonald/ofxCv
https://github.com/ofTheo/ofxKinect