How Cinema Sounds Affect the Perception of a Motion Picture

The object of the present study was to investigate the effects of sound on the perception of motion pictures by film viewers. For this purpose, we selected a sequence from the film “Elephant” (G. Van Sant, 2003) and composed two alternative soundtracks. The three different versions of the sequence (one with the original soundtrack and two with the alternative ones) were screened to 51 adult students (mean age 21.3 y.o.), who answered a series of questions after each screening. We concluded that different soundtracks change the perception of the audience mainly concerning the recognition of the film gender and the emotions that the viewers develop while watching the sequence. Moreover, the sound-design can possibly create visual illusions when the source of sound is shown on screen. Finally, factors like the sex and the field of studies seemed to differentiate our results, thus revealing their potential effect on the viewer’s perception.


Introduction
The ability of our organism to receive, translate and respond to environmental stimuli, mainly visual and auditory, has been crucial during the evolution of the human species and always maintains its great importance for its survival. In our days, a "civilization of the image" is established and motion pictures consist one of its major aspects. Filmmakers have developed methods and techniques to engage their audience, driving their attention and emotions. The effects of movies experience on our society and culture have many times been questioned by philosophers, film-makers, film-theorists, and psychologists [1].
Since the process of environmental stimuli perception has a neurological basis, brain scientists have also recently considered the nature of movies experience. An important number of studies during the last 50 years have investigated the mechanisms of the human nervous system that coordinate the reception of various environmental stimuli in order to create what the individual understands as "reality". In general, older studies supported the idea that vision is dominant between the five senses in this "reality-building" brain process [2,3]. However, recent papers conclude that there is a balance between vision and audition [4][5][6][7][8][9] and probably a cross talk between these two sensory pathways [10]. The development of methods for screening brain activity has permitted the identification of brain areas that are implicated in the process of interacting between visual and auditory stimuli [11] contributing to linking brain with mental processes.
Despite the scientific interest for the film viewer's brain activity, the movie experience hadn't, till recently, been approached by empirical interdisciplinary research. Shimamura [1] introduced the term "psychocinematics" to describe the scientific analysis of the aesthetic nature of movies by scientists, philosophers, filmmakers, sociologists, psychologists and other related endeavors, as a "holistic" approach to the phenomenon of cinematic perception. Concerning specifically the sound, one of the two major qualities of the cinematic reality, psychoacoustics tries to investigate the mechanisms of sound perception from the human brain [12]. In contrast to acoustics that study the physical and measurable parameters of sound waves (wavelength, frequency etc), psychoacoustics refer to subjective sound features including loudness, pitch, tempo, and melody and to all the different kinds of their effects, thus contributing in revealing the different aspects of sound as a stimulus [13][14][15][16][17][18].
Taking into account the great range of effects of the various sound features on the listener, as well as the importance of both audition and vision in constructing a common audiovisual reality [19][20], it is of great importance to identify the relation between sound and image in the case of film synthesis, since cinema is exactly what we call an "audio-visual art". More specifically, in this study we aim to focus on the effects of different sound designing on the perception of a motion picture. Our main purpose is to try to answer the following two questions: (1) can we change the viewers' perception of the film's gender or dramatic elements (crucial moments, protagonist feelings), by performing changes in the sound band? (2) are there any differences in the perception of the above elements between different groups of viewers? As far as we know, this is the first time that a study of sound effects on the image is performed in terms of the cinematic art.

Using a Film Sequence as a Research Tool
A three minutes sequence from the film "Elephant" directed by G. Van Sant [21] was selected as a methodological tool for the conduction of our study. The sequence is composed by two steadicam shots, during which we follow a young man walking in a college / high school campus. The sound designer of this original version (from now on called "version 1") was Leslie Shatz, who developed a dramatic / poetic synthesis for the sequence, in which non diegetic, sad piano-music was the dominant element. Besides version 1, two different soundtracks were composed for the same sequence. For "version 2" we composed a dramaticrealistic soundtrack, using natural sounds of the environment (eg. voices, rain, vehicles etc) and excluding any sound sources out of the sequence's reality. For "version 3" we composed a comic soundtrack including natural "gang-sounds", eg the funny voice of somebody singing, or animal sounds, again excluding non-diegetic sound sources.

Sample and Experimental Procedure
The above products of audiovisual synthesis were used as a methodological tool for the following research: The three versions of the sequence were screened to 51 adult students (mean age 21.3 y.o., Table 1) of the Departments of Film Studies (32 students) and Biology (19 students) of Aristotle University of Thessaloniki, Greece. Two separate screenings took place, each one for the students of each department. During screening, the sequences' order was different in the two groups of spectators.
Before the screening, all the individuals were asked for possible disabilities in hearing and seeing. The viewers were asked to write their answers on a series of questions they were given after each screening within five minutes (see Appendix for further details on the questionnaire's format). The same set of questions, was given after each screening. No hearing or seeing disabilities were mentioned by the subjects, so all the 51 questionnaires were taken into account for the rest of the analyses. The most important of the findings after collecting and processing the data, follow in the next session. Have seen the movie 4 Haven't seen the movie 47

Conclusions
Concerning the identification of the film gender, versions 1 and 2 were characterized as "drama films" by more than three quarters of the subjects in both experiment groups. However, version 3 (comedy soundtrack) was characterized as "drama film" or "comedy" by almost equal parts (near 40%) of the sample in both groups. This could mean that elements of drama and realism are easier identified by the film viewers, compared to comedy. This is also supported by the finding that the majority of subjects in both groups (more than 80% for both cases) selected version 2 as the original soundtrack of the sequence.
The time (season) in the film-world was found to be determined mainly by visual than auditory elements of the sequence for the majority of the subjects. Visual elements also determined the viewers' perception of the sequence crucial moment in all groups.
The emotions of the protagonist, according to the viewers' approach, were different in the three versions of the sequence and in most cases they were similar to the emotions of the viewers.
Sound design was able to create visual illusions. In both versions 2 and 3, there was a part of the sample that "saw" elements that were not on the screen, but existed only as sounds. It seemed that women, students of biology and individuals without knowledge of an instrument were mainly subjected to this kind of illusion.
We conclude that, although different parameters of the sequence (film gender, crucial moment, emotions etc) are not equally affected when the sound band is changed, the sound remains a dynamic quality of the cinematic aesthetics, which can be manipulated in order to cause a different perception by the viewers.