A symmetrical attention-assisted multi-modal fusion network under uncertain absent modalities.
Saved in:
| Title: | A symmetrical attention-assisted multi-modal fusion network under uncertain absent modalities. |
|---|---|
| Authors: | Li, Jiayao1,2 (AUTHOR) lijiayao@sxau.edu.cn, Cai, Saihua1,3 (AUTHOR) caisaih@ujs.edu.cn, Zhao, Kaiyi4 (AUTHOR) zhaokaiyi@nuc.edu.cn, Sun, Ruizhi1,2 (AUTHOR) sunruizhi@cau.edu.cn, Yuan, Gang2 (AUTHOR) yuangang@cau.edu.cn, Chen, Zeqiu2 (AUTHOR) chenzq@cau.edu.cn |
| Source: | Expert Systems with Applications. May2026, Vol. 309, pN.PAG-N.PAG. 1p. |
| Subjects: | Attention, Multisensor data fusion, Missing data (Statistics), Transformer models, Machine learning |
| Abstract: | The research of absent modalities learning is an important challenge in the field of multi-modal learning. A series of methods have been proposed to address the degraded prediction performance due to missing modalities. However, existing studies ignore the deep symmetric information of different modalities and lack considering the uncertainty of missing modalities. To address these two challenges, we propose a S ymmetrical attention-assisted M ulti-modal F usion N etwork called SMFN under uncertain absent modalities. First, the modality representation network is utilized to represent different modalities in order to obtain the semantic features of different modalities; Then, the symmetrical attention-assisted module is designed for symmetrically exploring both intra- and inter-modal feature information for tackling the first challenge; Next, a multi-modal fusion module is introduced to map the feature information of own modality and different modalities into the same common space, thereby enhancing the robustness of uncertain missing modalities; Finally, the dependency between modalities is learned using the transformer to accomplish prediction is accomplished. In addition, a pre-training model is also designed to train the complete modalities for improving the prediction accuracy. Extensive experimental results on benchmark datasets demonstrate that compared to the state-of-the-art models, the SMFN improves the accuracy for 6.52% and the F1-score for 11.59% on average, it also exhibits good robustness and convergence. [ABSTRACT FROM AUTHOR] |
| Copyright of Expert Systems with Applications is the property of Pergamon Press - An Imprint of Elsevier Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 192303993 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: A symmetrical attention-assisted multi-modal fusion network under uncertain absent modalities. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Li%2C+Jiayao%22">Li, Jiayao</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<i> lijiayao@sxau.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Cai%2C+Saihua%22">Cai, Saihua</searchLink><relatesTo>1,3</relatesTo> (AUTHOR)<i> caisaih@ujs.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Zhao%2C+Kaiyi%22">Zhao, Kaiyi</searchLink><relatesTo>4</relatesTo> (AUTHOR)<i> zhaokaiyi@nuc.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Sun%2C+Ruizhi%22">Sun, Ruizhi</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<i> sunruizhi@cau.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Yuan%2C+Gang%22">Yuan, Gang</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> yuangang@cau.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Chen%2C+Zeqiu%22">Chen, Zeqiu</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> chenzq@cau.edu.cn</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Expert+Systems+with+Applications%22">Expert Systems with Applications</searchLink>. May2026, Vol. 309, pN.PAG-N.PAG. 1p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Attention%22">Attention</searchLink><br /><searchLink fieldCode="DE" term="%22Multisensor+data+fusion%22">Multisensor data fusion</searchLink><br /><searchLink fieldCode="DE" term="%22Missing+data+%28Statistics%29%22">Missing data (Statistics)</searchLink><br /><searchLink fieldCode="DE" term="%22Transformer+models%22">Transformer models</searchLink><br /><searchLink fieldCode="DE" term="%22Machine+learning%22">Machine learning</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: The research of absent modalities learning is an important challenge in the field of multi-modal learning. A series of methods have been proposed to address the degraded prediction performance due to missing modalities. However, existing studies ignore the deep symmetric information of different modalities and lack considering the uncertainty of missing modalities. To address these two challenges, we propose a S ymmetrical attention-assisted M ulti-modal F usion N etwork called SMFN under uncertain absent modalities. First, the modality representation network is utilized to represent different modalities in order to obtain the semantic features of different modalities; Then, the symmetrical attention-assisted module is designed for symmetrically exploring both intra- and inter-modal feature information for tackling the first challenge; Next, a multi-modal fusion module is introduced to map the feature information of own modality and different modalities into the same common space, thereby enhancing the robustness of uncertain missing modalities; Finally, the dependency between modalities is learned using the transformer to accomplish prediction is accomplished. In addition, a pre-training model is also designed to train the complete modalities for improving the prediction accuracy. Extensive experimental results on benchmark datasets demonstrate that compared to the state-of-the-art models, the SMFN improves the accuracy for 6.52% and the F1-score for 11.59% on average, it also exhibits good robustness and convergence. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Expert Systems with Applications is the property of Pergamon Press - An Imprint of Elsevier Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=192303993 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1016/j.eswa.2026.131179 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 1 StartPage: N.PAG Subjects: – SubjectFull: Attention Type: general – SubjectFull: Multisensor data fusion Type: general – SubjectFull: Missing data (Statistics) Type: general – SubjectFull: Transformer models Type: general – SubjectFull: Machine learning Type: general Titles: – TitleFull: A symmetrical attention-assisted multi-modal fusion network under uncertain absent modalities. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Li, Jiayao – PersonEntity: Name: NameFull: Cai, Saihua – PersonEntity: Name: NameFull: Zhao, Kaiyi – PersonEntity: Name: NameFull: Sun, Ruizhi – PersonEntity: Name: NameFull: Yuan, Gang – PersonEntity: Name: NameFull: Chen, Zeqiu IsPartOfRelationships: – BibEntity: Dates: – D: 05 M: 05 Text: May2026 Type: published Y: 2026 Identifiers: – Type: issn-print Value: 09574174 Numbering: – Type: volume Value: 309 Titles: – TitleFull: Expert Systems with Applications Type: main |
| ResultId | 1 |
