Adriana Sa complete work


AUDIO-VISUAL RESEARCH 2011-2015

Research summary

This site presents research developed in the context of a practice-based PhD in Arts and Computational Technologies (2011-15), with the EAVI group at Goldsmiths, University of London. The
research was supervised by Atau Tanaka and Mick Grierson and
funded by the FCT - Foundation for Sciences and Technology.

 

A Perceptual Approach to Audio-Visual Instrument
Design, Composition and Performance

This research develops a perceptual approach to audio-visual instrument design, composition and performance. The approach informs practical work as well as a parametric visualisation model, which can be used to analyse sonic expression, sensory dominance and spatial presence in any audio-visual performance language. The intent of the practical work is for the image to function as a stage scene that reacts to the sound, without distracting attention from the relation between the sounds themselves. This is challenging, because usually vision dominates over audition. 

To clarify the problem, I bridged audio-visual theory, psychology, neuroscience, interaction design and musicology. The investigation led to three creative principles: a) to threshold control and unpredictability in order to convey sonic expression; b) to dispense with disruptive visual changes and apply Gestaltist principles to visual dynamics, so as to facilitate perceptual simplification; and c) to create an audio-visual relationship that produces a sense of causation, and simultaneously confounds the cause and effect relationships. The latter principle was also demonstrated with a study on audio-visual mapping and perception, whose conclusions are equally applicable to the audio-visual relationship in space.

My three creative principles informed the development of an instrument that combines a zither and audio-visual 3D software, exploring disparities between the acoustic and digital outputs. The parametric visualisation model can represent how any audio-visual performance work might converge or diverge from these principles. It combines parameters for interaction, sonic & visual dynamics, audio-visual relationship, physical performance setup and semantics.


 

Research outputs
Theoretical work:
Creative principles
Parametric model
Parametric representation
Practical work:
Practice summary
AG#1
Arpeggio-Detuning
AG#2