Narrative music : towards an understanding of musical narrative functions in multimedia

Abstract: As the computer screen is replacing the book as the dominant medium for communication (Kress, 2003), questions about how meaning is constituted by the multimodal interaction of different media (including music) is becoming increasingly important in contemporary research of pedagogy, sociology and media studies. The overall aim with this licentiate thesis is to explore musical narrative functions as they appear in multimedia such as film and computer games. The thesis is based on three publications. Publication 1 proposes a classification of musical narrative functions, with 6 narrative classes(the Emotive, Informative, Descriptive, Guiding, Temporal and Rhetorical classes) and 11 categories. The relational interplay of music with contextual factors is emphasized. Publication 2 describes the design of a software tool, REMUPP (Relations Between Musical Parameters and Perceived Properties), to be used for experimental studies of musical expression. REMUPP is used for real time alteration of musical expression, by the manipulation of musical parameters such as tempo, harmony, rhythm, articulation, etc. Publication 3 describes a quasi-experiment using REMUPP, where a group of young participants (12-13 years old) were given the task of adapting musical expression - by manipulating 7 parameters - to make it fit 3 visual scenes shown on a computer screen. They also answered a questionnaire asking about their musical backgrounds and habits of listening to music, watching movies and playing computer games. Numerical data from the manipulations were analyzed statistically with regards to the preferred values of the musical parameters in relation to the different visual scenes. The results indicated awareness and knowledge about codes and conventions of musical narrative functions, and were to some degree affected by the participants' gender, musical backgrounds and media habits.