Prosody off the top of the head: Prosodic contrasts can be discriminated by head motion

Erin Cvejic, Jeesun Kim, Chris Wayne Davis

Research output: Contribution to journalArticle

29 Citations (Scopus)

Abstract

The current study investigated people's ability to discriminate prosody related head and face motion from videos showing only the upper face of the speaker saying the same sentence with different prosody. The first two experiments used a visual-visual matching task. These videos were either fully textured (Experiment 1) or showed only the outline of the speaker's head (Experiment 2). Participants were presented with two stimulus pairs of silent videos, with their task to select the pair that had the same prosody. The overall results of the visual-visual matching experiments showed that people could discriminate same- from different-prosody sentences with a high degree of accuracy. Similar levels of discrimination performance were obtained for the fully textured (containing rigid and non-rigid motions) and the outline only (rigid motion only) videos. Good visual-visual matching performance shows that people are sensitive to the underlying factor that determined whether the movements were the same or not, i.e., the production of prosody. However, testing auditory-visual matching provides a more direct test concerning people's sensitivity to how head motion/face motion relates to spoken prosody. Experiments 3 (with fully textured videos) and 4 (with outline only videos) employed a cross-modal matching task that required participants to match auditory with visual tokens that had the same prosody. As with the previous experiments, participants performed this discrimination very well. Similarly, no decline in performance was observed for the outline only videos. This result supports the proposal that rigid head motion provides an important visual cue to prosody.

Original languageEnglish
Pages (from-to)555-564
Number of pages10
JournalSPEECH COMMUNICATION
Volume52
Issue number6
DOIs
Publication statusPublished - 2010

Fingerprint Dive into the research topics of 'Prosody off the top of the head: Prosodic contrasts can be discriminated by head motion'. Together they form a unique fingerprint.

  • Cite this