Re - thinking Re•wind

"What you want to hear, you hear not. For, what finds its way out from the underground

and the out there is spoken in rhythms and tones, In a language that solicits a different hearing."

Trinh T. Minh-Ha, elsewhere, within here 

produced by: Petros Velousis

OVERVIEW

Rewind is author's work that approaches the processes of encoding and decoding of multiply-encrypted messages. Music is used for the articulation of the message, and is considered through its potential to engage and activate varyingly the decoding capabilities of the viewer / listener. The author is questioning how the decoding of the discourse may be constructed through music / audio broadcasting, when almost every visual representation is absent. The use of diverse encoding systems for the reproduction of the same message is investigated and incorporated to the artefact, that contextualize different audience positions integrated to the triptych Human - Blinds / Music Box - Machine. This discussion attempts to define connections between the parts of the triptych, to render the Machine visible to Human, to approach rising similarities and define possible connections between them. There is also increased significance on the use of the music box for the author, through his own self-analytical journey, as the means to activate recollections of childhood events and thus increase the emotional openness of the audience to the message.

Artefact - Immersive experience

The processing program contacts the Twitter API for tweets with the hashtag  #amalgamation. Those tweets are subsequently displayed on screen in three different coding systems; a special-font type of the Latin alphabet, a braille font type and in a generated QR code, all drawn with dots. The three instances are turned from dots into holes, and punched on the music box's papers with the use of a laser cutter. The paper is fed to three different music boxes that transform the dots in sounds, playing in synchrony. A kinect camera is looking for human bodies / blobs in the room. Their distance to the music boxes is altering the speed of their motors, similarly to an intimate encounter; the closer the person, the faster the motors' turning pace. This in turns, affects the tempo of the tweets' musical reproduction, amplifying the immersive experience and closing the communication circuit with continuously produced iterations of the same process.

The paper approaches Rewind through the communication process methodology introduced by Stuart Hall, focusing on the processed of production, circulation and consumption of media messages.

INTRO

From Encoding to Decoding - Hypothesis

Stuart Hall argued that the meaning is not statically determined by the encoder/producer, and that there is no 'perfectly transparent communication', as there is no 'perfectly equivalent circuit'. 'In actual discourse most signs will combine both the denotative and the connotative aspects'. This renders the audience active recipient of meaning, that naturally perceives the message and its fluidity of meaning in association to its deep semantic cultural codes, at the individual scale. Hartley holds the same view, suggesting that language, culture and beliefs can influence the understanding or not understanding of the transmitted messages. Barthes effectively stated that the connotative levels of signifiers 'have a close communication with culture, knowledge, history, and it is through them, so to speak, that the environmental world invades the linguistic and semantic system. They are, if you like, the fragments of ideology' (Hall, 1973). 

Consequently, "lack of fit" is identified between the phase of the message production (encoding) and the moment of reception (decoding), due to not only the innate, relative and topographic differences of encoder/receiver; but also the asymmetry of their codes at the moment that the meaning structures are transformed. Umberto Eco coined the term 'aberrant decoding', to state messages interpreted differently to the intention or the original meaning of the encoder. On the same topic, Hall notes "What are called 'distortions' or 'misunderstandings' arise precisely from the lack of equivalence between the two sides in the communicative exchange. Once again, this defines the 'relative autonomy', but 'determinateness', of the entry and exit of the message in its discursive moments". Nevertheless it is worth mentioning that there are limits and parameters into which decoding operates and into the extend that misinterpretations of the meaning can be allowed, to entail effective communicative exchange which in turns requires some degree of reciprocity between encoding and decoding. It is therefore undoubtedly important for the encoder and the receiver to share the coding system of the communication means (Hall, 1973). 

Focusing on decoding, Hall believed it is the most important part of the process. Decoding is constructed by the reception, the absorbance and the understanding of the transmitted information. On this axis he formulates three potential positions of the receiver which are relevant to his ideology and cultural imperatives and from which the decoding can be constructed; the dominant, the negotiated and the oppositional. 

The dominant - hegemonic position, where the viewer interprets the actual, connoted meaning of the broadcast. The producer succeeds in conveying his message to the receiver, fully and straight and the receiver decodes it in accordance to the reference code in which it has been encoded. Hall notes that in this case, the receiver "is operating inside the dominant code". In this view, no misunderstandings occur, and presumably receiver and sender share similar ideologies and beliefs. Symbolically, this case is ideal-typical of "perfectly transparent communication", where the message is clear and successfully sent and received. 

The negotiated code or position, where there is mixture of acceptance and rejection from the audience point of view, but nevertheless the majority understand adequately the dominant definition and the professional signification of the message. It is also possible that while the audience understands the meaning of the message, its opinions differ to a certain extend. In essence, there are adaptive and oppositional elements; the audience allows the hegemonic definitions for grand significations (abstract), while it retains its own interpretations and negotiated versions in local, situational conditions.

The oppositional code or position, where the viewer decodes the message in a globally contrary way. It is not audience's incapability to understand the meaning of the message; on contrary, it understands both literal and connotative aspects of the conveyed discourse but chooses to completely reject it, detotalize it and reproduce it within its own alternative referential framework. This oppositional reading can be due to audience's disassociation to the view, or because it does not reflect upon commonly accepted views of society, or simply due to not holding the same view.  

Hall based his communicational model referencing the television broadcasts, that inevitably are accounting for messages with audiovisual characteristics. On first level of approach, how can we identify this communicational model of encoding and decoding in the sphere of audio exclusively? Is it possible to create interaction with the audience, with the lack of images? And how is a message perceived when there is no visualization to support it? Another question that rises, relates to the decoding positions that Hall proposed. How can different encoding systems be interpreted by the same receiver/individual and what is perceived when the same message is encoded by different encoded systems? Is the result different in what ways? Is there a dominant position for each viewer, or his position is also dynamically changing, following the message's alterations? 

Using Hall's communication process model, the author will approach Rewind through the four stages that Hall introduced. It is important to note that this structured conception of the different communication stages is not linear; instead, the stages or moments below incorporate linked and infused complex relations. The process therefore is the 'complex structure in dominance', as Hall coined, retaining its distinctiveness and in the same time allowing for connotations. 

STAGE 01 - PRODUCTION

The algorithm is scanning Twitter to identify and store all the tweets that are using the hashtag #amalgamation. In this stage, reflecting upon the society and the intended assumptions of the audience, the author is selecting three different encoding systems that apply to different types of audience. The message correspondingly is fed to three different encoding systems: the Latin Script, the Braille Code and the QR Code, to be encoded.

Latin Script - Humans: Hegemonic position

Latin is an alphabetic type of script, currently the most widely used in the world. Often referred to as the global language or lingua franca, Latin is used to express the predominant, hegemonic position. There is a political coloring on the hegemonic position according to Hall, as the hegemonic viewpoint defines within its terms the mental horizon, the universe of possible meanings, of a whole sector of relations in a society or culture; and that it carries with its terms the stamp of legitimacy - it appears coterminous with what is 'natural', 'inevitable', 'taken for granted' about the social order. The international popularity of this encoding system leaves small room for misinterpretations and therefore is symbolically used, with the intention for the audience to acknowledge its legitimacy to encode a message that has already been signified.

Braille Code - Blind & Music Box: Negotiated position

Braille code is developed as the medium for blind people to interpret the visual. Lacking from their vision, they depend on the other senses and in specific their touch to receive the visual aspect of the broadcasted message. This creates another layer to the message that renders the communication mediated. In turns, the fact that the message is reformed and produced through this mediated form of communication, is possible to create connotations, in terms of less fixed, associative meanings. The nature of the braille code also amplifies this aspect. The interpretation of the meaning depends on the braille coded language that is used (English), as the various braille languages slightly differ, incorporating diverse symbols and signs. In addition, the encoding system itself corresponds to specific group of the audience, able to understand the initial message denotation. These aspects are able to stimulate certain levels of partially controlled misinterpretations, that aim to create noise in the decoding; more changeable and conventionalized meanings to be interpreted through diverse receiving, that clearly vary from instance to instance. This renders the Braille code as the negotiated position, provoking a mixture of adaptive and oppositional elements.

In this work, the complete lack of visual stimuli that aims to direct the experience of the viewer in the audio, empowers the use of the braille code, as blind people generally have increased perception of sounds. The lack of vision, enables larger brain capacity for processing the information from other senses. Blind people are also trained to orient by echolocation, that enables them to interpret the sound waves reflected by nearby objects and, thus, accurately identify their characteristics, such as location and size. Lastly, it is worth mentioning the visual connection of the Braille code with the 'language' of the medium in use, the music box, which gives room for further referential stimuli to the viewers.

QR Code - Machine: Opposing position

QR code is a type of matrix barcode, essentially a machine-readable optical label. In practice, QR codes often contain data for a locator, identifier, or tracker that points to a website or application. In this regard, OR codes can be characterize as visual representations of the 'language of the machine', relating both the process of storing the information into a digital space, and the symmetrical image of black and white dots to the 0 and 1 of the binary code. The use of the QR code is therefore connected to the oppositional position, as human being are incapable to interpret the message without the interference of technology. This notion attempts to raise a point; the machine, is brought in the foreground, as symbolic reference to the symbiotic relationship between man and machines, of which technology is the mediator. There are not only messages to be received and interpreted by human beings but also messages for the machines, that can be interpreted only through the reception, rationalization and decoding systems of machines. This formulates a mediated interaction between human and machine, as the language of the latter is presented in plain view, in conjunction and parallel occurrence with the language of the first. The thing is given the same importance as the human, and this renders it visible to the human eye. This provides opportunities for human and nonhuman solidarities; between artificial and biological, natural and machinic. It is referred to the oppositional position, as only through opposition machines manage to establish their existence to humans, towards an object-oriented ontology. 

STAGE 02 - CIRCULATION

This stage incorporates the broadcast of the message to the audience. Circulation is partially responsible for receiver's perception of the message. The success of this stage, thus, entails the reception of the message by the audience. In regards to Rewind, the encoded tweets are fed into three music boxes in the form of nodes on paper. As music boxes rotate, the message is decrypted into music and diffused in the space in synchrony. 

The Music Box

The medium is 'a' message

The music box is capable of playing the notes that are punched within the grid’s limits of the paper. Each square of the grid, corresponds to a specific musical note of the music box. 

In respects to its sonic affordances, the music box is commonly identified as nostalgic, esoteric, dreamy and toy-like. In the view of the author, nevertheless, it also carries the affordances of its clockwork mechanism, in terms of time qualities. The overall winding, that is evident and easily perceivable both audibly and visually, creates the time reference. The listener / viewer, captivated by its childlike sound, is implored to experience introspective processes, through a recollection journey to his childhood. This reference that is currently in progress renders him emotionally responsive. From this state, being in close connection to past memories, the listener / viewer is contingent on absorbing the message that this vehicle-medium attempts to convey.

There is added symbolism in the encoding-in-music of the written language. During the circulation of the message, the second level of encoding is materialized. In this stage, where the encoded messages are fed to the medium of circulation - the music boxes. The spoken languages are transformed into music notes of the music box, that forms the image of a speaking object. The music boxes are signing tweets. This renders the music box not only as the means to articulate a message but also as a thing with its own ontology. The use of a thing that plays music, a music box that is the background of a scene, is placed in the centre of the exhibition event, reproducing messages. This notion aims to enhance the audience's responsiveness towards the impersonal life and raise its awareness of the complicated web of entwined connections and discontinuities between bodies.  This also reverses the relationship of the audience with the artefact, as the viewer / listener is essential for the completion of the moment; to receive the broadcast message and to interact with the medium, and finally reproduce the new message.  

STAGE 03 - DISTRIBUTION / CONSUMPTION

Three different music pieces are being composed in real time, originated by the same source - the tweet, encrypted through different encoding systems and played simultaneously by three music boxes. 

The sounds are expressing the same information in three different languages and the audience is called to understand the message, and experience the unification of meaning. Is this provoking the confusion of the tower of Babel, or indeed the sounds interwind into the music that enables audience to receive and perceive the intended message?

Experiencing material affordances

The diverse “paternal geometries” of each of the three languages - coding systems, create different material affordances which affect and define the sound, rhythmically and tonally. 

In specific, the QR codes, with their strictly square geometry and equal sizes, provoke extensive repetition regarding both the rhythmic and tonal aspects of the sound. The melodies are giving the sense of an almost absolute symmetry, that was indeed predicted. The only sense of musical ornaments or else, sound randomness, is produced by the pixels inside the square of the QR shape. In this sense, the produced noise is limited and coupled with its symmetry, strengthens the connection of this sound production to the communication of the machine: 0 and 1. 

On the other hand, in the Latin-originated sound, this absolute square - geometrization was absent. Being a proper alphabet, with distinctive and finite letters, is forming various words and phrases (tweets). This creates multiple levels of complexity in the perceived sounds, and gives out to the listener the sense of freedom. Furthermore, the lack of geometrical shapes (apart from the circle [“Oo”]) and the presence of short lines and curves instead, produces shorter rhythmic and melodic patterns, each of which corresponds to the alphabet’s letters. This, coupled with their dense assembly and varied combinations, greater polyphony and embellishments are observed, that decorate the melodic line. The result is rather abstract, and atonal, rendering its sound the 'noisiest' of all three languages! Nevertheless, this increased variation is indeed stimulating and, to some extent, resembles birdsongs.

Finally, the (English-based) braille alphabet has the geometric qualities of a dice; there are no actual curves which in turns results into less melodic ornaments. The music sound is less descriptive, with mechanical hue, and therefore, combines the material affordances of the previous two, to a great extent. In specific, similarities with the Latin alphabet can be addressed regarding the short length of its letter shapes, though shorter in size. This produced shorter in time and easier to distinguish repetitive, rhythmic and melodic, patterns. Its polyphony and music pattern density are ranging between the two previous languages.

The actual experience in all, is creating a sequential level of music / sounds hierarchy, that directs the listener from strict geometry to pure freedom. Or, in other words, from the inorganic, human-made, artificial (machine) to the organic, biological, natural (human); proposing the creation of a communication channel, a virtual connection and, through the understanding of the broadcasted meaning - the tweet, a unification; the amalgamation.

STAGE 04 - REPRODUCTION

Interaction of the artefact with the viewer

The multiple broadcasts of the same message in different encoded systems, with varied decoding capabilities from the viewer is giving room to varied interpretations of the message. The experience of the artefact is immersive in the capability of interaction with the viewer / listener. As he moves, his body alters the contouring of the space. The camera scans the alteration, and reacts algorithmically, provoking noise to the audio production. This noise is not catholic to all music boxes, which enables the preservation of their identities as autonomous ontologies. In essence, there is heterogeneous assemblage through affective bodies. The body of the viewer is affecting the audio message that is produced by its vicinity to the music box; and each music box interprets this approximation in diverse way, retaining its individuality. The body of the viewer in essence is extended by its interaction with the music boxes and in turns music boxes are rendered visible to the viewer by their integrated intelligence to enable this interaction. Intelligence of living creatures, that is described as one's ability to perceive information -room alterations, and store it as knowledge that can later on stimulate and form adaptive behaviors addressed to a certain environment or context - create noise in the audio message.

CONCLUSION

The investigation of the artefact through Halls communication model and the structure of the experience in a step-by-step process, responded to the raised questions with useful insight and more references, but mainly with increased interest for further studies. 

Limitations & Future Steps

In conventional approaches, the note mapping on the paper of the music box have the purpose of triggering a specific note, from the given melody. In the case of this project, due to the qualities of the different alphabets, it was impossible to map every hole of every single letter to explicitly match the note square on the paper’s grid, so as to trigger a specific note. So, the mapping was addressed in a way that each tweet (on every language) had to be within the limits of the grid’s height in order to be reproduced musically. This, for the author, created an interesting arbitrary randomness. In future applications, it would be intriguing to experiment with different mapping techniques of the encoded message on the paper that is fed to the music box, that would potentially provoke the formation of repetitive music patterns in the installation due to the recurrence of specific tweets. 

In terms of computation, the author's initial intention was the creation of this communication circuit in real-time, with the automation of the process of encrypting the encoded messages on the paper and feed it directly to the music boxes. If the process was realized in real-time as proposed, each installation would be one moment of its own, in the sense that the same artefact would reproduce namely one iteration of the process, different from the one before and never the same with the next one. In that sense, the concept of time would be rendered an interesting topic for future explorations. 

Furture studies would also incorporate the evaluation of the response of the viewer / listener to the audio messages, their interaction with the medium and their interpretation of the composed music. The message is the same in three languages, originated by the same hashtag. How is this affecting the viewer/ listener? An interesting investigation in further studies would be, therefore, the documentation of people reactions to the produce messages. Do they perceive patterns in the behaviour of the artefact? Do they realize the way that they interact, through their movement? What feelings do the various crypted languages expressed in sound patterns, provoke to the listeners? And, is the resulting sound a unified music piece, as it was intended to be? Do the sounds of music create repetitive patterns that can be perceived by the listener within his spectrum of hearing or music capabilities? Or the result is totally atonal and random?

Further insights to the perception of the artefact could also rise through the mapping of the viewers/listeners movements in the space. Their interaction with the artefact due to their position within the space and the corresponding reaction of the artefact in terms of varied patterns of produced noise. Would it be completely random, or it would enable the identification of spatial patterns? What would that mean for the perception of the sound? 

Sound is a force that constantly rustles the surroundings, recreating vibrations and directing reverberations, formulating relations between interiorities and exteriorities, accumulating singularities into a collective body or separate them; 

This research in its essence is aiming the attention at the freedom of listening, which incorporates the capacity and imaginary of physical resistances. Through listening to oneself, one is enabled to deepen his conscience and consciousness. And in turns to listen to others; through this filter they will resound with particular hope or anger. Only through this interaction one initiates to truly experience the interdependencies of which he is a part. And this has the potential to encourage the experience of a collective making of this life. As Brandon LaBelle beautifully remarks "if sound is a force that continually stirs the surroundings, driving forwards an array of vibrations and reverberations, ripping relations between interiorities and exteriorities, the inner depths of surfaces, and beyond, collecting singularities into a collective body, or forcing them apart, it does so in such a way as to potentialize the inherent flux of things - intensify the animate conditions of life in the making."

DEFINITIONS

1 Twitter : Twitter is a social media platform, selected due to its raising popularity and for currently being presumably the timeliest medium for news. It is a microblogging system that permits users to send and receive short posts called tweets, based on the following - followed process where users can see tweets from the ones following on their 'timeline', similarly to Instagram. The user creates his own tweets or re-tweet existed information. It is widely used within organizations, groups, academics, politicians, students and general public.

2 Latin Script : Latin is the official or co-official alphabet used by the western civilization and in 3 continents. Consisted of an average of 26 letters, it is widely adopted; statistically around 4.9 billion people or 70% of the global population relies on this alphabet. There is also the concept of 'International English' as a global means of communication in numerous dialects, and the movement towards an international standard for the language.

3 Braille Code : Braille is tactile writing system used by visually impaired people. It is traditionally written with embossed paper. It is derived from the Latin alphabet, albeit indirectly. In Braille's original system, the dot patterns were assigned to letters according to their position within the alphabetic order of the French alphabet, with accented letters and w sorted at the end. The basic braille alphabet, braille numbers, braille punctuation and special symbols characters are constructed from six dots. These braille dots are positioned like the figure six on a die, in a grid of two parallel vertical lines of three dots each. From the six dots that make up the basic grid, 64 different configurations can be created.

4 QR Code : Quick Response Code consists of black squares arranged in square grid with white background. It is readable by any imaging device such as a camera, and processed using Reed–Solomon error correction until the image can be appropriately interpreted. The required data is then extracted from patterns that are present in both horizontal and vertical components of the image. In practice, QR codes often contain data for a locator, identifier, or tracker that points to a website or application. A QR code uses four standardized encoding modes (numeric, alphanumeric, byte/binary, and kanji) to store data efficiently;

BIBLIOGRAPHY

1. Acar, A. (2006): Models, Norms and Goals for English as an International Language Pedagogy and Task Based Language Teaching and Learning, The Asian EFL Journal, Volume 8. Issue 3, Article 9.

2. Davis, H. (2004): Understanding Stuart Hall. SAGE.

3. Durham, M. G. & Kellner D. M. (2001): Media and Cultural Studies, Key works. Part II: Social Life and Cultural Studies, 13 Encoding / Decoding, Stuart Hall. Revised Edition, Blackwell Publishing Ltd. Available online: https://we.riseup.net/assets/102142/appadurai.pdf

4. LaBelle B. (2018): Sonic Agency, Sounds and Emergent Form of Resistance. Goldsmiths Press, London

5. Teel A. (2017): The Application of Stuart Hall’s Audience Reception Theory to Help Us Understand #WhichLivesMatter? Online article:  https://medium.com/@ateel/the-application-of-stuart-halls-audience-reception-theory-to-help-us-understand-whichlivesmatter-3d4e9e10dae5

REFERENCE

• Denso ADC (2011): QR Code Essentials. DENSO Wave Incorporated. PDF, Available online: http://www.nacs.org/LinkClick.aspx?fileticket=D1FpVAvvJuo%3D&tabid=1426&mid=4802​
• https://www.worldatlas.com/articles/the-world-s-most-popular-writing-scripts.html
• https://inhabitat.com/ecouterre/soundweaving-turns-folk-embroidery-into-playable-music/
• http://kathyhinde.co.uk/music-box-migration/
• http://www.helenpritchard.info/helen-pritchard/jsut-code-2010
• https://www.miuc.org/new-media-dead-media/
• https://link-springer-com.gold.idm.oclc.org/article/10.1007/s00146-011-0335-5
• https://www.edge.org/conversation/brian_eno-composers-as-gardeners