Hearing foreign voices: does knowing what is said affect visual-masked-speech detection?

Perception. 2003;32(1):111-20. doi: 10.1068/p3466.

Abstract

We investigated audio-visual (AV) perceptual integration by examining the effect of seeing the speaker's synchronised moving face on masked-speech detection ability. Signal amplification and higher-level cognitive accounts of an AV advantage were contrasted, the latter by varying whether participants knew the language of the speaker. An AV advantage was shown for sentences whose mid-to-high-frequency acoustic envelope was highly correlated with articulator movement, regardless of knowledge of the language. For low-correlation sentences, knowledge of the language had a large impact; for participants with no knowledge of the language an AV inhibitory effect was found (providing support for reports of a compelling AV illusion). The results indicate a role for both sensory enhancement and higher-level cognitive factors in AV speech detection.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Adult
  • Auditory Perception*
  • Comprehension
  • Female
  • Humans
  • Language*
  • Lip
  • Male
  • Psychological Tests
  • Sensory Thresholds
  • Signal Detection, Psychological
  • Visual Perception*