Why what you see affects what you hear
When you hold a conversation with someone you’re combining auditory cues with visual cues to make sense of what is being said. Typically one of your senses (hearing or vision) is dominant, which influences your perception of sound or what you hear. This is explained by the McGurk effect — when you hear one syllable, but you see the mouth movement of another syllable, and this leads you to perceive a third syllable. According to this effect if you are experiencing poor quality auditory information but are receiving good quality visual information, then you are more likely to experience the McGurk effect. Sensory dominance or sensory weighting between your senses will result in different perceptual effects. But is the McGurk effect more prominent in children or in adults?
Adults and 10–12-year-olds are more influenced by visual stimuli, while children rely more on auditory information.
A new study from the University of Nottingham, in collaboration with Nottingham Trent University, showed there is a developmental shift in sensory dominance as children grow older. The researchers assessed the effects of manipulating the clarity of the heard and seen signal upon the McGurk effect in 29 children aged 3–6 years, 32 children aged 7–9 and in 29 children aged 10–12. They also assessed the effect on 32 adults aged 20–35.
The researchers designed a “spy” game called Spot the Sound for the study. Adult participants completed a task in a quiet testing lab while child participants completed a task in a quiet room at the university alongside other studies taking place. Each group was presented with visual stimuli that consisted of videos of a single female speaker vocalising one of three syllables; “Ba”, “Ga” or “Da”. In 75 per cent of the trials, congruent auditory stimuli were presented. In 25 per cent of the incongruent trials, a visual “Ga” and an auditory “Ba” were presented. Syllables were also presented either without noise or alongside white noise. Within each trial, the video presented was followed by an on-screen message asking, “What did you hear?”, after which participants could respond using three counterbalanced response keys —“Ba”, “Ga” or “Da”/“Tha”.
The researchers found that the McGurk effect was higher in adults compared to children in the 3–6 years age group and the 7–9 years age group, but not in the 10–12 years age group. This showed that adults and 10–12-year-olds are more influenced by visual stimuli, while children rely more on auditory information. Children are less susceptible to visual illusions, which might affect their perception of sound.
The researchers explain that part of the brain responsible for auditory information develops earlier than the part of the brain responsible for processing visual information, which accounts for the dominance of the auditory sense in childhood. However, children gain more visual experience across childhood through activities like reading, and as developmental changes take place and reach adult-like levels by 10 years of age, there is a sensory shift in the dominance of visual information over auditory information. That’s why adults are likely to be influenced by what they see, which will affect what they hear.
Source: Scientific Reports
