Digitalising Movement Analysis
‘What is human movement in the absence of the body?’ (Paul Kaiser)
The field of movement analysis is a fairly subjective one, especially in relation to dance. This paper explores Laban movement analysis and the ‘digitalisation’ of this. It looks into the information we can get from the digital data of dancers’ movements, analysing electromyogram signals, motion captured data and computer vision techniques. The focus of the project is to implement practice based research with a post phenomenological approach to human computer interaction, whilst placing the practice within figuration of the body[1], movement analysis and dance theory whilst also touching on kinesiology. More precisely, it looks at how we can analyse efforts of movement digitally in relation to space and how we can dictate direct and indirect space through a digital platform. By translating movement to a machine, does it give a different quality to the way in which we may consider movement and what it is to be human or machine?
produced by: Abi Price
Machines are best recognised for providing scientific analysis of data that is analysed, especially in relation to movement that is very geometrically based. Take space as an example; it can tell us where the body is in a particular space more precisely than the human eye. But can we venture into this deeper with the machine understanding the weight and time of movement also? It is this approach which interested me in looking at a dancer’s movement through a screen. By digitalising movement analysis, it can be repeated/replayed, amended, viewed in different ways and the data can be stored. Notation and analysis in dance are verbal and written traditions, but how do we go about preserving digital techniques?
Background
Choreographer Rudolf Laban, coined the term ‘efforts’, embodying 8 actions or gestures whose forms are made up of 3 key sections of performative quality. The combination of space, time and weight form the 8 efforts and each is split into two opposing qualities. Space; direct/indirect, time; sudden/sustained and weight; strong/light. An example being the slash effort, comprised of indirect, sudden and strong movement.
Though it is important to think about Laban’s efforts in dance, its wider implementation gives scope to a range of fields. Historically, Laban established choreology, the discipline of dance analysis. His technique has since been developed by Lisa Ullmann, Irmgard Bartenieff and Warren Lamb, proving its success. LMA does not just apply to dance, but actors, musicians and physical therapists, as well as in psychology and human computer interaction. Its purpose originally stemming from factory workers in World War II and production lines known as ‘Profiling Systems’, used for the optimisation of workers, this efficiency being key to an ever expanding technological industry. Another technically important use of LMA is in user experience. UX Matters has a great article, outlining how Laban’s 8 efforts can be translated to the experience of using a smartphone (Lepore, 2010).
The project recognises one particular paper that has been influential in the process. I discuss later how my research is similar as well as how it differs and attempts to add to the field of computational art as well as dance. Seeing, Sensing and Recognizing Laban Movement Qualities (Fdili et al, 2017) provides a reliable and scientific approach whilst still remaining creative. With the samples collected, new movements can be analysed by looking at the patterns and trends of the EMG data. The punch and slash effort seem to give a similar reading, but if there was a tiny bit more power to the slash gestures, we would be unable to tell them apart which would prove problematic. The same with wring and press, which is why I am keen to break down my data into separate muscle groups as I feel these graphs could be interpreted differently.
From researching, one area that stood out to me was that of space and how it is communicated through movement as there is a lack of knowledge with no definite way of expressing space. When we think of a dancer in space, perhaps instinctively we think of the body simply being a coordinate of a point in space or where the body or a certain body part is at a certain point in time, whereas I want to move away from this mathematical approach. It isn’t necessarily about how the body uses external space or geometry but what we can draw from this by looking at the pathway of movement. Direct movements take a straight pathway and indirect, the opposite, curved or zigzagged. However, there is more to space when we consider the kinesphere[2]. It is internal to the dancer, being more about their thoughts and attention rather than what we may recognise was about where the body is located in space, which is why there is increased subjectivity around the definition. Direct movements may focus on one particular direction and be specific whereas indirect movements encompass all around awareness of the space, both externally and internally.
In dance, the performer uses their face to communicate emotion and the quality of movement, so does looking at digital data through visualisation techniques, give a different quality to how we view movement? Can we also recognise qualities from digital traces and how we may gather information about the emotional state? Choreographer and media artist, Kate Sicchio states that she can tell someones emotion based on the way they interact with their smartphone (2018) by considering how movements can be related to the Laban efforts, linking to how awareness is brought to how we are mindlessness when it comes to technology. We just do it, we don’t think. This cartesian detachment of the mind occurs with the disembodiment from the technology. When interacting with a projection for example, a natural response is to wave the arms around, not paying attention to the mind and what this connection to the technology represents.
Results/reflection/critique
Stage 1 of the project involved computer vision techniques using the Kinect as an easy to access technology. The first weeks research consisted of the first of many physical experiments creating and experimenting with different OpenFrameworks programs.
I explored this in relation to different levels of digital bodies, shown in the path below and looked at how they figure the body.
human (normal video stream)—> digital body (shadow)—> outline (contour finder)—> points (important points)—> traces (visualised movement pathway)
By breaking the body further away from a human figure each time through the experiment has given a more in depth level of understanding about depth of quality and the efforts of movement. Out of all these experiments, I was keen on following the trace path representation of the body. One of the next steps I wanted to explore was giving the system information so it can make a decision about which effort a performer is carrying out, however, as computers are not yet accomplished enough to predict or recognise movement qualities, especially with computer vision alone, a change of track was in order.
For stage 2 of the practice based research, I recorded movement using a marker less motion capture system developed by Anton Koch with Motion Bank to collect data as well as looking at captured data from Effect [3]. Analysing in this way combined the reliability of accurate data with a more creative approach, both in analysation of the data and in the way we visualise it.
I used the system to record 3 dancers in space, choosing one dancer’s right hand as the point to track as it is probably the most dominant body part that gives the best range of movement. I created a few simple digital Choreographic Objects style visualisations such as William Forsythe created for Synchronous Objects. By creating these visualisations, it ties into the forms of archiving and contributes to a form of notation similar to Motion Bank’s online dance scores.
I created a Processing program and looked at the live data side by side. I visualised the captured data as a trace to analyse how the lines were drawn, finding this the most reliable way of determining the space element of movement and whether it was direct or indirect. For test 1, I made a program with trail opacity, showing the time element. Opaque trails represented a sustained quality, whereby translucent, a more spread out, a sudden quality. Test 2 explored the space element, looking at the angles to give classification to direct or indirect movement. Most of the time sharper angles meant indirect movement, more specifically shorter lines with sharp angles. However, this does not necessarily mean straight lines are the only direct movements.
This worked quite well as a more creative approach of looking at scientific data. This method proved difficult to capture the weight element as strong and light come from an internal force that I have found only visible when looked at live or through recorded video, as facial and body language contribute to this.
This was actually more of a geometrical way of looking at movement, which could be classified as ‘“kinetic isometries”. Forsythe has described this as learning to develop a facility for imagining the geometric relationships of different parts of the body as they move, or transferring the shape of one part of the body to another part’ (Forsythe, 2018).
It is necessary to digitalise this data as the invisible trace shown underneath the choreography gives a better insight and deeper understanding, especially to categorising the space element of movement. There are so many ways to visualise the captured data but like with other approaches, this part of experimentation had its limitations. There was only limited time I could spend with the system but I would have preferred to collect a better data set with more travelling around the space and increased definition of the efforts.
‘Nearly all models in the literature focus on capturing one body part and use either data taken from motion capture systems or accelerometers. There is hardly any model combining visual data with bio-signals such as muscle contraction data that could be crucial to model Efforts’(Fdili et al, 2017)
This led onto stage 3. It involved working with the Myo armband to collect a more accurate data set. The EMG data was visualised in MaxMsp using signal graphs. I decided I wanted to use the EMG data rather than the accelerometer, as the 8 efforts encapsulate weight or force well and this was a great way to capture this using individual arm muscles. I went through each Myo channel with each effort and made a table with the range of values that the effort triggered. Then decided which channel and values best represented that particular effort which worked fairly well for the actions I had created.
This approach could be difficult as there are endless movement variations for each of the efforts. So instead I chose one simple movement for each effort and analysed which muscles these movements trigger using a visual graph output in the Max patch. My end result was a patch that printed the particular effort to the console when that action was made. Although this was the first stage, it worked fairly well for some of the efforts, not so well for wring, glide and float as the sustained quality was hard to work with. Although this particular instance of capturing 8 movements would have given more reliable results with machine learning, a machine cannot possibly learn every movement possibility the body can make, so I hope by using muscle data, this can be the first step of creating a relationship between muscle activity and qualities of movement in order to recognise qualities. Whilst machine learning is great at accurately classifying a movement and gesture recognition, relating the geometry of where a particular body part is in space, the EMG data relates to Laban’s efforts more precisely, giving a deeper understanding of the internal movement or intention of the body and is a step towards behaviour recognition. For example if I hold my hand in a fist, machine learning can recognise this and tell me the pose is a fist but EMG signals can tell me if I am holding my hand in a loose fist or squeezing it tightly, giving the recognition of the weight element of an action.
I briefly looked into the biomechanics or kinesiology to look for a relation between particular muscles and what ‘effort’ they contribute to. We can also use this to internalise the efforts with small micro-movements informing choreography, this linking to internalised direction of indirect space.
As with any experiment, there are limitations. There was the challenge with the raw data being very jumpy rather than seemingly precise lines like the Seeing, Sensing, Recognising paper acknowledges. Also, efforts such as wring, triggered actions like punch because it is activating the same muscle, which is not necessarily a bad thing however. The same muscles make different actions, which I have likened to choreography, the same body parts making different movements.
This last stage of experimentation has proved the most useful as the connection is closer to the body as well as making the invisible visible with the visualisation. I would tie this method (Myo) to Laban’s term eukinetics[4] whereas, the motion capture would be closer tied to choreutics[5].
Regular video, I believe figures the body a lot worse off that the live movement as it is hard to distinguish the weight effort through this flatness, one sided view of an action. The Myo definitely gave a sense of embodiment with the project, maybe this was because movement comes from the inside of the body. Through this approach, I have found weight the hardest overall category rather than space like many researchers have. The motion capture represented space and time well and the Myo good for weight so combining these techniques would prove useful.
The project covered a range of experimentations, although still in the early stages, I can see a use of archiving these techniques and how they can come to benefit the wider fields of health, gaming, entertainment, but specifically for health as a record can be kept of how the body performs in a given situation. Although it won’t erase the traditional way of looking at LMA, it has proved a useful tool for refiguring the body as the human still has the say in the digital which is important as movement is a very human thing, close to the body. [6]
Conclusion
Although a relatively speculative project, experiments with an array of data visualisation techniques gathered from movement data has perhaps given something new to the field of dance tech. Findings, especially from the EMG data started to carve some pathways picking out ‘space’ and showing that indirect and direct space can be visualised with this technique. This was the most scientific technique used so perhaps this physical extension and the data coming from close to the body is the way forward. We have more of a direct connection to the screen with a wearable rather than the indirectness of 2D video recordings. I would love to take this further by narrowing the study to working more with the Myo armband. I would redefine some of the parameters and I could see a performance being crafted as an outcome, whereby one dancer performs with the Myo, influencing another depending on which effort quilters dancer one shows and we can then assign these back to visuals or can send to another dancer to interpret from text.
[1] Figuration - ‘Kembar (2003) identifies figuration, in its mobilisation as a means of intervention, as “visual or verbal images which embody transformations in knowledge, power and subjectivity”(Suchman, 2007)
[2] Kinesphere - the area around the body that can be reached.
[3] Effect - choreographed by Taneli Törmä.
[4] Eukinetics - Laban’s effort studies
[5] Choreutics - Spatial Harmony theory (geometrical)
[6] ‘The loom, like many other forms of industrial machinery, established a new hybrid combining the perfectly accurate machine with its still necessary, but more “limited”, human operator’. (Suchman, 2007) - We cannot have a machine without the human.
Bibliography
Blades, H. (2015) Affective Traces in Virtual Spaces: Annotation and Emerging Dance Scores. Performance Research - A Journal of Performing Arts. 20(6) pp 26–34.
Chotrani, C. (2018) The importance of documenting & archiving the performing arts. Culture 360. Available from https://culture360.asef.org/magazine/importance-documenting-archiving-performing-arts/
Davies, E. (2006) Beyond Dance Laban’s Legacy of Movement Analysis. New York ; London: Routledge.
Dubner, S. Freakonomics Radio Live: ‘Jesus Could Have Been a Pigeon.’ Freakonomics [podcast] Available from http://freakonomics.com/podcast/tmsidk-duckworth-2018/.
Fdili Alaoui, Sarah, Jules Françoise, Thecla Schiphorst, Karen Studd, and Frederic Bevilacqua. (2017) Seeing, Sensing and Recognizing Laban Movement Qualities. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI ’17. pp4009-4020.
Forsythe, W. and Libeskind, D. (2010) Organisation of Body and Space in Drawings. [online] Anastasia Galicheva. Available from http://anastasiagalicheva.com/Organisation-of-Body-and-Space-in-Drawings-by-William-Forsythe-and.
Hachimura, K. (2006) Digital Archive of Dancing. Available from https://pdfs.semanticscholar.org/60f8/3fd836bc8bc20107328a8bb9fa680cd07516.pdf
Hansen, M. (2012) Bodies in Code: Interfaces with Digital Media. Routledge Ltd.
Hodgson, J. (2007) Mastering movement : The Life and Work of Rudolf Laban. Methuen Drama.
Koch, A. and Delahunta, S. (2017) Dance Becoming Data Part Two: Conversation Between Anton Koch and Scott DelaHunta. [online] Computational Culture. Available from http://computationalculture.net/dance-becoming-data-part-two-conversation-between-anton-koch-and-scott-delahunta/.
Leach, J., and Delahunta, S. (2017) Dance Becoming Knowledge: Designing a Digital Body. Leonardo. 50 (5) pp461–467.
Lepore, T. (2010) Laban Movement Analysis for User Experience Design. [online] UXmatters. Available from https://www.uxmatters.com/mt/archives/2010/02/laban-movement-analysis-for-user-experience-design.php.
Motion Bank (2019) Online Scores. Available from http://scores.motionbank.org/
Picard, R. (2000) Affective Computing. Cambridge, Mass.: MIT.
Portanova, S. (2013) Moving without a Body: Digital Philosophy and Choreographic Thought. Cambridge, Massachusetts: The MIT Press.
Respini, E., Roslyn S., Louise N., (2018) William Forsythe: Choreographic Objects. Institute of Contemporary Art Boston. Boston, Munich , New York.
Rocha, J., and Snelting, F. (2017) The Possible Bodies Inventory: dis-orientation and its aftermath. Inmaterial. vol 2 (3). Available from https://www.raco.cat/index.php/Inmaterial/article/view/343359.
Santos L., Dias J. (2010) Laban Movement Analysis towards Behavior Patterns. In: Camarinha-Matos L.M., Pereira P., Ribeiro L. (eds) Emerging Trends in Technological Innovation. IFIP Advances in Information and Communication Technology. 314 pp187–194.
Sicchio, K. (2015) Laban Principles for UX Design. Available from https://www.slideshare.net/ksicchio/laban-principles-for-ux-design.
Suchman, Lucy. (2007) Human-machine reconfigurations: Plans and situated actions. Cambridge University Press.
Vellose, E., Bulling, A., Gellerson, H. (n.d) Inertial Body Tracking for Affective Computing Using AutoBAP [online] Xsens 3D Motion Tracking. Available from https://www.xsens.com/customer-cases/inertial-body-tracking-for-affective-computing-using-autobap/.
Other references
Balandino Di Donato - Myo Max patch
Motion Bank - Effect Player
Anton Koch- markerless motion capture system