Lively
A machine performance which combines Chinese traditional art with new media art. This project aims to rebuild traditonal semiotics by using technologies and explores the relationships between cultrues and technologies.
produced by: Dongyuan Liu
Introduction
Lively is a machine performance which combines Chinese traditional art with new media art. Chinese traditional drums are always used in traditional celebrations, such as Spring festival, traditional wedding and so on. Using rapid rhythm and flashing firecrackers can arouse public’s attentions. More than the traditional celebration form, it bears more of the traditional Chinese social characteristics.
Concept and background research
I was mainly inspired by 'The surrender of culture to technology', by Neil Postman. In this book, Postman pointed out several relationships between culture and technology.While he explores the dark side of technology, I am thinking about the fact that if the significance of the traditional cultural symbols is weakened and exhausted due to the excessive consumption of technology, then can technology be used as a way to reconstruct and interpret traditional symbols and give it new energy. How do we balance the relationship between technology and traditional culture?
Fidgety (in between up and down), by GayBird, was awarded Lumen Prize in 2018. They use the sound of physical collisions and technology to reconstruct the expression of '忐忑'. Whether in the early stages of the work and the final expression of the work, they break and rebuild the semiotics. Express a word that describes the mood in a way that combines sound with visual movement.
Furthermore, This project was continued the basic thinking of my previous projects, which focus on the transformation and expression of semiotics in the installation art. By using technology, such as Projectionmapping, Arduino, machines, I want to achieve the purpose of reconstructing traditional semiotics. In other words, technologies as a tool can be used to rebuild traditional culture semiotics.
Technical
The collaboration between softwares
Because the whole performance does not include human actors, I used OSC and serial communication to achieve cooperation between the three software. The core master of the whole performance is MaxMsp. The signal is sent to the Openframework and Arduino via OSC and serial communication, and then MaxMsp can control the motion of the visual part and the arms. The Openframework in turn controls the start and end of timeline in MaxMsp so that the synchronization of picture and motion will be achieved.
Maxmsp
I used the counter as a timeline in maxmsp to manually compose the entire three-minute performance. It is like what kind of rhythm and how fast speed are used to command the robot arm at specific time point. Moreover, I wrote a few basic rhythms and states, like stopping, hitting together, 1212, 11221 and so on.
Openframework
I arranged the visual part in the Openframework, because the visual part is mainly to imitate the scene of the firecracker flashing, so there is no use of too many colors, the whole process is basically white to make the performance simple, and some different elements such as circles and points are added in the middle. The hollow circle is designed to allow the viewer to understand the hitting position of the drum by sound and image in dark conditions. The creation of the visual part is mainly based on the structure of the drum. For example, the dot pattern is created based on the screws around the drums. As for the flickering effect, it is based on the received numerical signal from MaxMsp, and using map function to convert 0-127 into 0-1, thereby judging whether the drum stick hits the drum(1 is a hit, 0 is lift up). If it received 1, the image shows. If it received 0, the image disappears. The effect of flicker is produced at a frame rate of 60.
Arduino
I mainly use servo driver shield, which can drive up to 16 servos at same time. PWM is used to control the rotational position of the servos. The numerical signal received from MaxMsp is converted to a 150-550 rotation range by the map function to control the movement of the drum sticks.
Others
Performance arrangement
I controlled the entire performance for about three minutes. First, I informed the audiences the begin and end of the performance by the warm lighting and the rotation of the robotic arm at the begin and end of the setup. This format is imitated the human performers' performances. After the performance started, I set up a simple rhythm and a progress of slower to faster rhythm to tell the audiences how the visual part is connected with the movement of robotic arm. After that, I combined different rhythm drums beats with different speeds, and combined the richer visual elements to push the performance to the climax. Such an arrangement is helpful for mobilizing the mood of the audiences.
Physical structure
In order to hit the center and edge of the drum, I designed the arm to be a structure that can move in two dimensions, that is, the lower big servos can control the drum sticks to move left and right, and the upper small servo can control the drum sticks to move up and down. In addition, in order to make it distinguishable from front to back, I designed it to be inclined at a 15 degrees angle. And rotate at the begin and end of the show to inform the viewer the begin and end. The servo brackets (3D Printed) and laser cutting were also used to build the robot arm. After the three times tests, I confirm the firmness and structural rationality, and start copying.
Technical problems encountered
The biggest problem in the process is the delay caused by the way the serial communication accepts information. Even if the servo can be rotated at a faster speed, serial communication is sent one after another as the information is transmitted. This means that each time every servo read data, they need to wait for 8 messages to be sent. This causes the problem of data reading confusion when setting the rotation speed of the servo too high. For example, the No. 1 servo reads the value which should be read by the No. 3 servo. I can only try to get the servo rotation speed that matches the reading time.
In terms of physical structure, I need to consider the robustness of the material to prevent the entire structure from escaping due to the strong vibration of the servo during the performance. After three times tests, the size of each component and whether more holes are needed are determined.
Future development
During the exhibition, many people suggested me to make a whole band of traditional musical instruments. This is a very interesting direction. It can make the reconstruction of traditional cultural symbols more completely. For the performance of the arrangement, I can add machine learning into my band, for example, they can perform according to each other’s’ beats. For a general direction of the second year, I want to learn about sound, like generated music, or some physical sounds. I think this will be a great help for the integrity of my work. It will be also helpful to research on immersive experiences, space, and so on, Which can complement my imperfections in the creative process.
Self evaluation
For the expected effect of this work, such as the audience's reaction and the completeness of semiotics’ conversion and expression, I think it is closer to my expected effect. I am very happy with the reaction of the audiences during and after the performance. They are happy, interesting or excited, which makes my work more complete. I want to create a lively atmosphere, and the reaction of people around in an environment is also a factor to consider. However, the audiences that asked me questions after watching the show seemed to be more interested in my robotic arm which is not the core of my project. On a technical level, I have not optimized it for some details. For example, the problem of the shaking of the servo, and the coordination of the visual part and the drum hitting time point. Furthermore, I can continue to add content to the visual section to enrich it. Similarly, for the change of the drum rhythm can be more enriched. I can ask some help from or collaborate with professional drumers.
References
- Postman, Neil. Technopoly: The Surrender of Culture to Technology. 1st ed, Knopf, 1992.
- (SKU:ROB0074)Hexapod spider robot aluminum bracket - DFRobot Product Wiki. http://wiki.dfrobot.com.cn/index.php?title=(SKU:ROB0074)HEX%E5%85%AD%E8%B6%B3%E8%9C%98%E8%9B%9B%E6%9C%BA%E5%99%A8%E4%BA%BA%E9%93%9D%E5%90%88%E9%87%91%E6%94%AF%E6%9E%B6.
- Adafruit Customer Service Forums • View Topic - PWM: Servos vs. Electronic Speed Controlers: Can I Use the s. https://forums.adafruit.com/viewtopic.php?f=8&t=125963&p=628580&hilit=deadband#p628580.
- ‘Fidgety (in between up and down) • Lumen Prize’. Lumen Prize, https://lumenprize.com/artwork/fidgety-in-between-up-and-down/.
- OpenFrameworks Tutorial - 047 Fullscreen on Application Launch. YouTube, https://www.youtube.com/watch?v=U5sdTbFApmo.
- Serial Input Basics. https://forum.arduino.cc/index.php?topic=288234.0.
Openframework Addons
- ofxGui
- ofxJSON
- ofxOsc
- ofxPiMapper
- ofxXmlSettings
Adapted from
- projectionSceneManager from WCC week 10
- OscSender from WCC week 17
- Osc Reveiver from WCC week 17
- Using Servo Motors with the Arduino | DroneBot Workshop. https://dronebotworkshop.com/servo-motors-with-arduino/.
Arduino Library
- Adafruit PWM Servo Driver Library, Written by Limor Fried/Ladyada for Adafruit Industries.