The ear that dreams: eye tracking sound in the moving image

Drawing on cinematic theories of sound, and neuroscientific understandings of attention, comprehension, and the gaze, this video essay employs eye tracking technology in a sound on/off comparative analysis of the first five minutes of the Omaha Beach landing scene from Saving Private Ryan (Spielberg, 1998). The film was chosen as a case study because it involves complex sound design, moments of perceptual shock, internal diegetic sound, spatial and temporal shifts in sound, and heightened sonic agency.

Six viewers were eye tracked at the Eye Tracking Lab at La Trobe University, Melbourne, and the data analyzed through a combination of close textual analysis and the statistical interpretation of aggregate gaze patterns. The viewers were shown the sequence twice: once with its normal audio field playing, and once with the sound taken out.

In this video essay I interpret this data to answer the following questions:

To what extent do viewers’ eyes follow narrative-based sound cues?

How does the soundtrack affect viewer engagement and attention to detail?

Is there an element of prediction and predictability in the way a viewer sees and hears?

Do viewers’ eyes ‘wander’ when there is no sound to guide them where to look?

Ultimately, I ask how important is sound to the cinematic experience of vision: Does the ear dream?