View in EDS PDF Full Text

The Role of Auditory and Visual Modality in Perception of Focus in Mandarin Chinese.

Saved in:
Bibliographic Details
Title: The Role of Auditory and Visual Modality in Perception of Focus in Mandarin Chinese.
Authors: Shanpeng Li1, Yihan Wu1, Sasha Calhoun2, Mengzhu Yan3,4 mengzhu_yan@hust.edu.cn
Source: Journal of Speech, Language & Hearing Research. Aug2025, Vol. 68 Issue 8, p3843-3860. 18p.
Subject Terms: *Language & languages, *Audiovisual materials, *Learning, *Attention, *Speech perception, *Auditory perception, *Visual perception, Statistical power analysis, Noise, Task performance, Research funding, Prompts (Psychology), Descriptive statistics, Physiological aspects of speech, Linguistics, Data analysis software, Confidence intervals, Phonetics
Geographic Terms: China
Abstract: Purpose: Speech perception is a complex process that involves multiple sensory modalities. Despite our intuitions of speech as something we hear, accumulating evidence has shown that speech perception is not solely dependent on the auditory modality. While it is well established that auditory and visual cues can both help listeners perceive focus, the latter is not established in Mandarin, and the relative contribution of these cues is not established at all. The current study investigated Mandarin listeners' integration of auditory and visual cues in the interpretation of focus in noise-degraded speech, through a question-answer appropriateness rating experiment. Method: To explore the effectiveness and relative contribution of auditory and visual modality in the interpretation of Mandarin focus, participants did a question-answer appropriateness rating task involving subject focus, object focus, and broad focus. All the question-answer pairs were constructed in three modalities: audio only, visual only, and audiovisual. They were instructed to rate the appropriateness of the question-answer pairs. A babble noise was superimposed on the audio track for the audio only and audiovisual conditions. Results and Conclusions: Although auditory cues via prosodic prominence were an effective cue to interpreting focus, visual cues were proven more effective, at least with degraded audio. Overall, this research contributes to our understanding of the interaction between linguistic cues and sensory information during language comprehension, widens the range of languages included in this body of research, and provides important implications for future studies on focus processing in various linguistic contexts and communication settings. This, in turn, will deepen our understanding of the multimodal nature of language comprehension. [ABSTRACT FROM AUTHOR]
Copyright of Journal of Speech, Language & Hearing Research is the property of American Speech-Language-Hearing Association and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Education Research Complete
FullText Links:
  – Type: pdflink
Text:
  Availability: 0
Header DbId: ehh
DbLabel: Education Research Complete
An: 187402266
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: The Role of Auditory and Visual Modality in Perception of Focus in Mandarin Chinese.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Shanpeng+Li%22">Shanpeng Li</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Yihan+Wu%22">Yihan Wu</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Sasha+Calhoun%22">Sasha Calhoun</searchLink><relatesTo>2</relatesTo><br /><searchLink fieldCode="AR" term="%22Mengzhu+Yan%22">Mengzhu Yan</searchLink><relatesTo>3,4</relatesTo><i> mengzhu_yan@hust.edu.cn</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Journal+of+Speech%2C+Language+%26+Hearing+Research%22">Journal of Speech, Language & Hearing Research</searchLink>. Aug2025, Vol. 68 Issue 8, p3843-3860. 18p.
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: *<searchLink fieldCode="DE" term="%22Language+%26+languages%22">Language & languages</searchLink><br />*<searchLink fieldCode="DE" term="%22Audiovisual+materials%22">Audiovisual materials</searchLink><br />*<searchLink fieldCode="DE" term="%22Learning%22">Learning</searchLink><br />*<searchLink fieldCode="DE" term="%22Attention%22">Attention</searchLink><br />*<searchLink fieldCode="DE" term="%22Speech+perception%22">Speech perception</searchLink><br />*<searchLink fieldCode="DE" term="%22Auditory+perception%22">Auditory perception</searchLink><br />*<searchLink fieldCode="DE" term="%22Visual+perception%22">Visual perception</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+power+analysis%22">Statistical power analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Noise%22">Noise</searchLink><br /><searchLink fieldCode="DE" term="%22Task+performance%22">Task performance</searchLink><br /><searchLink fieldCode="DE" term="%22Research+funding%22">Research funding</searchLink><br /><searchLink fieldCode="DE" term="%22Prompts+%28Psychology%29%22">Prompts (Psychology)</searchLink><br /><searchLink fieldCode="DE" term="%22Descriptive+statistics%22">Descriptive statistics</searchLink><br /><searchLink fieldCode="DE" term="%22Physiological+aspects+of+speech%22">Physiological aspects of speech</searchLink><br /><searchLink fieldCode="DE" term="%22Linguistics%22">Linguistics</searchLink><br /><searchLink fieldCode="DE" term="%22Data+analysis+software%22">Data analysis software</searchLink><br /><searchLink fieldCode="DE" term="%22Confidence+intervals%22">Confidence intervals</searchLink><br /><searchLink fieldCode="DE" term="%22Phonetics%22">Phonetics</searchLink>
– Name: SubjectGeographic
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22China%22">China</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Purpose: Speech perception is a complex process that involves multiple sensory modalities. Despite our intuitions of speech as something we hear, accumulating evidence has shown that speech perception is not solely dependent on the auditory modality. While it is well established that auditory and visual cues can both help listeners perceive focus, the latter is not established in Mandarin, and the relative contribution of these cues is not established at all. The current study investigated Mandarin listeners' integration of auditory and visual cues in the interpretation of focus in noise-degraded speech, through a question-answer appropriateness rating experiment. Method: To explore the effectiveness and relative contribution of auditory and visual modality in the interpretation of Mandarin focus, participants did a question-answer appropriateness rating task involving subject focus, object focus, and broad focus. All the question-answer pairs were constructed in three modalities: audio only, visual only, and audiovisual. They were instructed to rate the appropriateness of the question-answer pairs. A babble noise was superimposed on the audio track for the audio only and audiovisual conditions. Results and Conclusions: Although auditory cues via prosodic prominence were an effective cue to interpreting focus, visual cues were proven more effective, at least with degraded audio. Overall, this research contributes to our understanding of the interaction between linguistic cues and sensory information during language comprehension, widens the range of languages included in this body of research, and provides important implications for future studies on focus processing in various linguistic contexts and communication settings. This, in turn, will deepen our understanding of the multimodal nature of language comprehension. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Journal of Speech, Language & Hearing Research is the property of American Speech-Language-Hearing Association and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=187402266
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1044/2025_JSLHR-24-00664
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 18
        StartPage: 3843
    Subjects:
      – SubjectFull: Language & languages
        Type: general
      – SubjectFull: Audiovisual materials
        Type: general
      – SubjectFull: Learning
        Type: general
      – SubjectFull: Attention
        Type: general
      – SubjectFull: Speech perception
        Type: general
      – SubjectFull: Auditory perception
        Type: general
      – SubjectFull: Visual perception
        Type: general
      – SubjectFull: Statistical power analysis
        Type: general
      – SubjectFull: Noise
        Type: general
      – SubjectFull: Task performance
        Type: general
      – SubjectFull: Research funding
        Type: general
      – SubjectFull: Prompts (Psychology)
        Type: general
      – SubjectFull: Descriptive statistics
        Type: general
      – SubjectFull: Physiological aspects of speech
        Type: general
      – SubjectFull: Linguistics
        Type: general
      – SubjectFull: Data analysis software
        Type: general
      – SubjectFull: Confidence intervals
        Type: general
      – SubjectFull: Phonetics
        Type: general
      – SubjectFull: China
        Type: general
    Titles:
      – TitleFull: The Role of Auditory and Visual Modality in Perception of Focus in Mandarin Chinese.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Shanpeng Li
      – PersonEntity:
          Name:
            NameFull: Yihan Wu
      – PersonEntity:
          Name:
            NameFull: Sasha Calhoun
      – PersonEntity:
          Name:
            NameFull: Mengzhu Yan
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 08
              Text: Aug2025
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 10924388
          Numbering:
            – Type: volume
              Value: 68
            – Type: issue
              Value: 8
          Titles:
            – TitleFull: Journal of Speech, Language & Hearing Research
              Type: main
ResultId 1