Black in White - Magnetic in Paper
An interactive installation piece that converts traditional Chinese landscape ink painting to computational arts, at the same time, it is a re-narrative of the materials.
produced by: Shuai Xu
Concept and background research
Chinese landscape painting
Many critics consider the landscape to be the highest form of Chinese painting. Early landscape first appeared as part of figure painting, providing the natural setting for story narration.
Images of nature have remained a potent source of inspiration for artists down to the present day. While the Chinese landscape has been transformed by millenary of human title, Chinese artistic expression has also been deeply imprinted with forms of the natural world. Viewing Chinese landscape paintings, it is clear that Chinese depictions of nature are occasionally mere representations of the external world. Instead, they are expressions of the mind and heart of the individual artists—cultivated landscapes that embody the culture and cultivation of their masters.
Computational materials science
The new method, called computational materials science, is a powerful way of discovering new materials with desired properties — such as improved charge and discharge speeds for battery materials — and of understanding and fine-tuning the properties of well-known, long-used materials such as steel alloys, ceramics, and cement composites, whose fundamental properties are still surprisingly little understood.
Introduction
The charm and life of Chinese landscape paintings is not only the external appearance of the paper but also the invisible sensible rhyme beyond the pen and ink. However, the traditional form of Chinese landscape painting is to express their emotions through the painter's sway on paper. Most of them are shown in a freehand way, and the penetration or spread of ink on rice paper sometimes brings people a surprise style. My work was replaced by magnetic powder by changing the traditional material ink. During the movement of the magnetic powder, traces are left on the rice paper; this is a re-narrative expression of the material. At the same time, leap motion control movement of the magnetic powder according to the gesture of visitors. I wish my work can redefining people's feelings about Chinese landscape painting, and feeling the charm of Chinese landscape painting in black and white.
Technical
Physical computing
One of the biggest challenges in this work is how to incorporate the narration of physical computing into the visual arts. From the very beginning, I wanted to use some sensors that could skeleton body tracking. I have considered using KinectV1 (Depth camera), Realsensor(Depth camera), and Leapmotion(Hand gesture sensor). I have to choose one that I am most satisfied with it, and I don't need too much data error. In the meantime, I did a lot of experiments which include what kinds of sensor easy to use, and what kinds of the sensor can give us a stable data also not generate errors. In order to ensure the data output is stably convert the data and do not generate extra data and errors. At the same time, when people want to draw a painting, the hand is their most essential tools. As well as, "freehand" in Chinese has a meaning is not used their hand, and therefore I choose Leapmotion as my sensor.
The structure I use six stepper motor control the electromagnetic position, at first, I tried to control the movement of the position in other ways, but the effect was not ideal, so I switched to stepper motors, which can move efficiently and can be easily controlled by Arduino.
Machine learning algorithm
In order to make sure my installation can get the correct action instructions and make it easier to use participant gesture data. I use some machine learning algorithms in my work, which can make sure the instruction of the participants can be executed more accurately. Because everybody has a different from using their hand, as well as if I only use the pure original data, everything will become harder. In this way, when my algorithm was analyzing the data and giving instructions, the identification and direction of the work will not change accuracy. So that it is essential to get new data to use machine learning algorithms rather than based on the beating data, make the installation crazy.
Because I only need to move the installation directly, according to the movement of the participants' gestures and be able to change direction flexibly. I send 15 inputs data to Wekinator and get two outputs data, which I choose classification method. Per output with five classes, at the same time, I trained a large amount of gesture data and got very stable data. It does not cause irregular movements of the installation due to the mechanical problems of gestures and the hands of different age groups.
Totally the whole machine learning algorithm has ten classes corresponding respectively, forward, backward, left forward, right forward, left backward, right rear, eight directions, and two blank commands.
Processing to Arduino
The communication between Processing and Arduino, I use the serial function. Firstly I use Leapmotion get original participant gesture data and use OSC send to Wekinator. Therefore, I get new data from Wekinator and use OSC send back to Processing, and next Processing gets the right instruction to Arduino through serial.
Process record video：
User Painting achievement：
Others
Materials representation
How to choose the right materials to complete our project has always been a challenging problem. In the field of computational arts, artists have a new re-narrative way to express art forms, they increasingly pay attention to participants' experience. Therefore, we need to consider what kinds of materials can bring the best effect to participants, let them participate in our works. As a result, we must create a new relationship between visitors and materials, so that sensors become our bridge between visitors and materials, of course, we need to show the characteristics of materials. In my work, I hope to represent the relationship between black and white in traditional Chinese landscape painting. Finally, I try to use many materials for my project, but when I came into contact with magnetic powder, I felt that this material could create a relationship between visitors.
Physical structure
In order to let the electromagnet follow the movement of the gesture and give quick feedback. I choose to use stepper motors and lead screw for my motion frame. Simultaneous I use machine learning to get the right result, which can let the structure move to right position. As well as, The steppers' bracket use 3D Printed and laser cutting to build the structure.
Future development
For the traditional narrative use computational way to representational, which always is a goal what I am trying to do. Furthermore, trying new materials and combining the two is what I need to do. So in the next step, I want to use machine learning and material re-narrative more deeply in computational art. Maybe I will expand my project, I will try to use the Magnetic fluid, use a more complex algorithm, combine embodiment make live performance, also combine the generative art(visual or music) in the algorithm which makes the work look richer and have meaning.
Self-evaluation
This project is a new attempt for me, because in the previous learning process, I didn't dare to do something that no one had done before, but this time, although it was just the beginning, I got enough The experience gives me the ability to move on to the next stage and try new materials and new technologies.
References
Processing library：
Serial
OscP5
Leapmotion
Programme reference:
Woodall, W., 2019. wjwwood/serial.
schlegel, andreas, 2019. sojamo/oscp5.
Schwittlick, M., 2019. schwittlick/LeapMotionP5.
PROTEUS_an analog-digital interactive display at Ars Electronica [WWW Document], n.d. . Vimeo. URL https://vimeo.com/269581453 (accessed 9.12.19).
Computational Compositions: Aesthetics, Materials, and Interaction Design [WWW Document], n.d. . International Journal of Dsign. URL http://www.ijdesign.org/index.php/IJDesign/article/view/757/301 (accessed 9.12.19).
Art, A.D. of A., n.d. Landscape Painting in Chinese Art | Essay | Heilbrunn Timeline of Art History | The Metropolitan Museum of Art [WWW Document]. The Met’s Heilbrunn Timeline of Art History. URL https://www.metmuseum.org/toah/hd/clpg/hd_clpg.htm (accessed 9.12.19).
Landscape Painting | Chinese Painting, n.d. . China Online Museum. URL https://www.comuseum.com/painting/landscape-painting/ (accessed 9.12.19).
Magnet + Iron filings = Fun Magic & Satisfying Experiment - YouTube [WWW Document], n.d. URL https://www.youtube.com/watch?v=LMFAOLKaYd0&list=LLx9CXS7uU7IcPA_IQSXYl9Q&index=7 (accessed 9.12.19).
Smelik, A. (2010). The scientific imaginary in visual culture. Goettingen: V & R Unipress, pp.69-88.
Salter, C. (2015). Alien agency. Cambridge, Massachusetts: The MIT Press.