WINDOWMATTER SERIES:

Adriana Sa solo AG#1 demo
 
duo with John Klima AG#1 screenshots
 
duo with Fala Mariam audio recordings
 
orchestral version funding & supports  

The WINDOWMATTER project series was developed along with an audio-visual 3D software called AG#1. In this series I played with the first version of my audio-visual instrument, which started being developed in 2008. The instrument combines acoustic sound, digital sound, and digital image. The software operates based on amplitude and pitch detection from a zither with custom tuning and strings. This is also the case with the subsequent software versions, Arpeggio-Detuning and AG#2.

AG#1 was a modification of software originally written by John Klima (2007), ironically titled Portuguese Guitar Hero after the popular Guitar Hero video game series. It uses a 3D engine called Torque and an audio library for video game programing, FMOD. 3D software can create a sense of depth, and extend one's sense of spatial presence into a digital world beyond the screen. I wanted to use a video game platform, and simultaneously make it evident that my creative concerns and aesthetics are very distant from gaming. I modified the software without changing the code structure, yet these modifications circumvented design theories embedded at low level in the digital architecture of the game engine.

When AG#1 detects audio, it changes the 3D camera view and places virtual sound emitting objects in the 3D world. Their position in the 3D world affects the spatialisation of related sounds. I parameterised these digital functions, created a new 3D world, new sounds and new mapping strategies. A description of my creative strategies can be found HERE. A related article can be found HERE. Bellow is documentation about the different WINDOWMATTER performance works.


SOLO VERSION
 



In the WINDOMATTER performances where I played solo, AG#1 operated based on the input from my custom zither, which also inputted a sampler / delay modulator. In this way, the sonic constructions intertwined acoustic sound, digital sound, and electronic sound. I wanted the audience's perceptual motion to be driven by the music, and the image to create a sense of total environment . The image should extend their sense of presence to a digital world beyond the screen. It should become an organic stage scene that reacts upon the sound, modulating perceptual experience with an undulating view over an unchangeable landscape.

Performances:

  • St. James Church/ Goldsmiths College, London, UK 2011
  • Share, NYC, USA, 2010
  • Off-Limits, In-Sonora Festival, Madrid, Spain, 201
  • ZDB, Lisboa, Portugal, 2010
  • Moagem, Festival Y, Fundão, Portugal, 2009
 
AUDIO RECORDING
   
GRAPHIC SCORE
for a solo performance:
 

 

DUOS WITH JOHN KLIMA
 




These WINDOWMATTER duos were performed with two different versions of the AG#1 software, which operated based on my zither input and the input of John's Portuguese guitar, or his 'noise-makers'. In each version, the interaction design, audio architecture, digital sounds and audio-visual mappings were very different. Yet the image was similar: the two versions were networked, so that each provided a different visual perspective over the same digital 3D world. I made the 3D world look like a moving painting. In performance, two moving paintings were projected side-to-side, upon us.

Performances:



DUO WITH FALA MARIAM

Performance: Teatro S. Luiz, Lisboa, Portugal, 2010

 

In this WINDOWMATTER duo I played my zither with aged strings and ambiguous tuning, inputting the AG#1 software. Fala Mariam played her trombone, dry. Each of us made a graphic score, and the sonic construction resulted from their combination. The image was projected upon us both.

AUDIO RECORDING


ORCHESTRAL VERSION
 

studies

toshio's instruments




In this WINDOWMATTER project several musicians extended their instrumentation with AG#1. Each created a set of digital sounds, to be activated upon the sound of their instruments. John wrote different versions of the C++ code. Each version of the software had its particular sound architecture and configuration, and all versions were networked. I made a graphic score for the collaborative sonic construction. We performed the piece in different site-specific situations. In each occasion, several perspectives of a same 3D world were projected over semi-transparent surfaces, creating an architectural space of light.
      

Adriana Sa, zither, AG#1
John Klima, portuguese guitar, AG1
O.blaat, hacked wii controller
Sei Miguel,  pocket trompet
Manuel Mota,  electric guitar
David Maranha,  violin & dobro
Fala Mariam
,  trombone
Cesar Burago
,  percussion
Andre Gonçalves
,  analogue modular synth
Toshio Kajiwara, turntables, modular synth, toys

Performances see text in PT

  • WINDOWMATTER _ inside the time (photos above)
    esidency at Regueirão dos Anjos, Lisboa, with a final, site-specific performance-installation (July 1st - 21th 2008)
  • WINDOWMATTER _ outside the shpere (photos bellow)
    Site-specific performance-installation at Largo do Seculo/ outdoors and Século Bar/ indoors (July 25st 2008)
     The in-door performance:
john klima and david maranha
manuel mota
adriana sa, fala mariam and and andre gonçalves
toshio kajiwara
0
toshio kajiwara and sei miguel
         The outdoor rehearsal:
john klima
adriana sa
margarida mendes
cesar buraga
sei miguel, john klima and toshio kajiwara
adriana sa and fala mariam
toshio kajiwara and margarida mendes
fala mariam, cesar buraga, manuel mota and david maranha
john klima, david maranha, toshi kajiwara and sei miguel
fala mariam and manuel mota
photos by O-blaat (that's why she's not in the pictures!...)


DEMONSTRATION OF AG#1 OPERATING BASED ON MY ZITHER INPUT:

 
PERFORMANCE AUDIO RECORDINGS:
Adriana Sa  
 
Adriana Sa & Fala Mariam

 

AG#1 SCREENSHOTS

 

 

FUNDING AND SUPPORTS:


Direcção Geral das Artes / Cultural Ministry Portugal

Quarta Parede

Festival Numero

MusicBox

In-Sonora - Muestra de Arte Sonoro e Interactivo
& Portuguese Embassy in Spain

FCT - Foundation for Science and Technology & European Union (PhD grant)