Perception is a process of multi-sensorial
synthesis [Calvert et al. 2004], in which the visual discounts the aural and
the aural discounts the visual. In many aspects
vision dominates over audition, but attention can be manipulated so this does not occur [Sinnet et al. 2007]. I have drawn from perception science to argue that attention dynamics depend greatly on the interplay of continuities and discontinuities. To compare the strength of sound and image I proposed a particular notion of intensity: intensity is the neural imapact of any change in
stimuli causing an increase of neural activity. In this way, it derives from the combined effects of automatic
(exogenous) and deliberate (endogenous) attention. It depends on the event
itself, the stimuli panorama, and perceptual resolution.
Whether musical motion is sonic or audio-visual,
it is the interplay of continuities and discontinuities which fundamentally
directs attention. The intensity of each event depends greatly upon its relation
with other events, as well as upon the current state of the person's attention.
I created a taxonomy to distinguish between a sense of continuity that is
primarily driven from stimuli (exogenous), and a sense of continuity that
depends more on the individual (endogenous). It also distinguishes between
radical discontinuities, which impose disruption, and ambivalent
discontinuities, whose acknowledgement depends on perceptual resolution.
Taxonomy of Continuities and Discontinuities Related with Intensity and Attention
||binds all types of
continuities and discontinuities /
occurs at high hierarchical level in
perceptual organisation /
requires long-term memory, attention and conscious
|no intrinsic motion /
lowest intensity, dispenses with attention
display a similar interval of motion /
gestalt of good continuation
stimulus or sudden interruption of cohesive motion /
|continuous at low
perceptual resolution and discontinuous at high resolution /
on deliberate (endogenous) attention.
How the image can modulate and NOT obfuscate the sonic experience:
|To keep the music on the foreground, one must dispense with radical visual discontinuities, which automatically attract attention. But there can be a wealth of visual discontinuities at detail level, inviting for deliberate attention.
Gestaltist principles can be applied to create
a sense of overall visual continuity, so that attention is invited to focus on ambivalent discontinuities.
The same principles manifest in the visual and the aural domain: