score a blob
A bodily interactive installation that invites participants to leave behind a trail of their figure moving in space. The trail is captured and added to a growing archive of messy choreographic scores.
produced by: Eirini Kalaitzidi
Introduction
The participants walk across an assigned line on the floor and notice a blob of their figure appearing additively on the screen and distorting the background image. A Kinect is detecting the moving body at the specific distance that the line indicates and a corresponding algorithm shapes a dynamic fluffy blob on screen, as the smooth outline of the moving body. Along with the continuously moving blob, a new static one is traced onto the screen every 700 milliseconds until there are 8 past body postures left on screen. A blue trail of the body’s route is also left behind while following the centroid of the moving body. A screenshot of the visual output is saved automatically, every time the participant reaches the end of the line/pathway.
These outputs are librettos of the bodies, hence potential choreographic scores which compose an archive that could be explored and interpreted creatively by a choreographer and/or a performer.
Apart from serving this archiving purpose, the participant is welcome to play around irregularly in space, while scoring blobs and adding a discreet movement to the gray-scaled static sea background, by distorting it.
Concept
“score a blob” is created in response to an insisting personal interest in generating, manipulating, inventing and archiving choreographic scores.
Scores transfuse a new, elongated time hypostasis into dance since dance’s temporality is being captured by something time-resistant: writings, graphic designs and data. Each of the captured scores could be seen as a plan, as a libretto, as an instruction, as a hint, as a tool or as a whole play. The creative purpose is not to limit its potential essence but to explore it as a visual information oscillating between a choreographer and a performer, who both are free to interpret it as much poetically or descriptively as they wish. What will emerge between them, cannot be controlled or foregone. The scores are in the disposal of the recipient minds and bodies.
“score a blob” was created as a a playful interactive installation but its main goal was to record and collect bodies’ creative pathways. This way, participants’ dancing becomes the yeast for choreographic generation in opposition to the conventional routing: from choreography to dance.
Technical
The project is built using solely openFrameworks/C++ with the ofxKinect, ofxCv and ofxOpenCv add-ons.
There are four main technical elements that compose the visual output: the real-time moving blob, the past static blobs, the fading trail and the distortion of the background image.
The blobs’ visualisation relied on tracking the moving body through a depth camera and then using the contourFinder to design a polyline from it. I experimented with different designs of the blob, from simple rectangles enclosing the body, to polygonal outlines, but the smooth curving blobs seemed like the ideal choice for the scores. The moving blob is mirroring the moving body in real-time while the past static blobs are stored in a deque of 8 blobs, constantly refreshing as long as a blob is tracked in space.
The trail is a continuously expanding and continuously fading line following the centroid of the blob. The sharpness of some angles created when changing the centroid drastically is something I exceptionally like in relation to the smoothness of the contour. The screenshot also relies on the centroid.x: whenever the moving blob exceeds the (kinect.width - kinect.width/6), a score is captured.
The final touch involves the distortion of the background image with pixels’ manipulation. I planned to use the ofxFlowTools add-on but I finally came up with a way to achieve the desired effect without its contribution. I created a mesh of the kinect image and dispositioned slightly the vertexes “touched” by the moving body or correspondingly, encapsulated in the blob. The chosen background image (photo from personal archive) is a top view of the sea, in gray-scale colours. I wanted a naturally fluid material to be the static background that only the moving body could shake.
Once I added this pixels’ manipulation, the frame rate dropped below 10fps and it was not consistent. Until that moment, I depended the deque of blobs' refreshing on frameCount( ), so I changed this dependency on ofGetElapsedTimeMillis( ) which was much more reliable.
Self evaluation
Since the beginning of this post-graduate programme I have had the goal to computationally generate, manipulate and deconstruct choreographic scores and I am pleased to witness a new way of achieving that in this interactive project of mine.
I am very satisfied with the design of the blob and its moving behaviour. I acknowledge the fact that I sacrificed my vision of passing through different levels and versions of image distortion due to lack of time for extra experimentation. Despite of me eternally supporting a minimal aesthetic, I admit that most of the pop-up visitors did not notice the current discreet distortion effect unless they were very observant or I pointed it out to them. Nevertheless, I noticed that they truly enjoyed interacting with the “score a blob”, although it was set-up in an isolated part of the exhibition and therefore, it was not easily trackable. This is something I need to consider when exhibiting my future work.
Overall, I am very pleased with the result. I believe it is quite playful and engaging as an installation while as an archive of allusive choreographic scores it is purposefully simple, poetic and challenging for the notion of dance and its temporality.
from amateur improvisational dancing to future-proof scoring (to disciplined dancing)
Future development
This lastly mentioned part is the next stage of this project. I would be really interested in and challenged by adding an extra interactive element to this existing relationship between human amateur dancing and digital scoring. This element would be the reversion from the static score to dancing, but this time it would not be improvisational, rather - it would be disciplined and dependent on the captured choreographic score. The dancer interpreting the score could be virtual or physical computed. This way the visitor would initialise a sequence of dynamic (human moving) - static (captured score) - dynamic (digital dancer) events.
Finally, while building this project, I realised that the effects I design could easily act like a partner for a solo dancer in a live performance, similar to what “Glow” performance suggests. This approach would be totally different from what I intended to achieve in this specific occasion (timeless archive of scores vs real-time performance) but it would be equally interesting for future development.
References
Flow distortion example: https://github.com/kylemcdonald/ofxCv/tree/master/example-flow-distort
CREA for computer graphics’ generation, using the motion data from a Kinect camera: https://github.com/fabiaserra/crea
Interaction with contourFinder: https://openframeworks.cc/documentation/ofxOpenCv/ofxCvContourFinder/
Lab Assignments from Workshops in Creative Coding II:
week 13 / optical flow controller
week 16 / kinect grid
week 16 / kinect fiery comet
Installation that stores people’s moving behaviour, made by Undef: https://www.undef.ch/project/user-632
GLOW dance performance with interactive video technologies: https://www.youtube.com/watch?v=2AautwIOON8
Millard, O. 2016. What’s the score? Using scores in dance improvisation, published in Brolga 40. https://ausdance.org.au/articles/details/whats-the-score-using-scores-in-dance-improvisation