View in EDS

Semi-supervised multi-label dimensionality reduction learning based on minimizing redundant correlation of specific and common features.

Saved in:
Bibliographic Details
Title: Semi-supervised multi-label dimensionality reduction learning based on minimizing redundant correlation of specific and common features.
Authors: Li, Runxin1,2 (AUTHOR) rxli@kust.edu.cn, Zhou, Gaozhi2 (AUTHOR) gz_zhou@stu.kust.edu.cn, Li, Xiaowu2 (AUTHOR) lxwlxw66@kust.edu.cn, Jia, Lianyin2 (AUTHOR) lianyinjia@kust.edu.cn, Shang, Zhenhong1,2 (AUTHOR) szh@kust.edu.cn
Source: Knowledge-Based Systems. Jun2024, Vol. 294, pN.PAG-N.PAG. 1p.
Subjects: Feature selection, Matrix decomposition, Machine learning, Feature extraction, Least squares, Dimension reduction (Statistics)
Abstract: Multi-label learning, like other machine learning methods, suffers from dimensionality disaster. However, due to the limitations of multi-label dimensionality reduction frameworks, multi-label dimensionality reduction techniques are difficult to effectively implement semi-supervised models, instance correlation constraints, and feature selection and extraction strategies. In this paper, we propose a novel semi-supervised multi-label dimensionality reduction learning approach based on minimizing redundant correlation of specific and common features (SMDR-MRC, in short). Firstly, we employ matrix factorization technique to transform the HSIC-based dimensionality reduction model MDDM (Y. Zhang and Z.-H. Zhou, 2010) into a least squares problem, so that the label propagation mechanism can be naturally integrated into the model. Secondly, cosine similarity and the k -nearest neighbor technique are used to generate constraint terms for low-dimensional manifold correlations and instance correlations, respectively. Finally, to identify the specific and common features of the low-dimensional manifold structures, we introduce two projection weight matrices constrained by the l 1 -norm and l 2 , 1 -norm. In particular, we define a non-zero correlation constraint that effectively minimizes the redundant correlation between the two matrices. To solve the optimization model with nonlinear binary regular terms, we employ a novel solution approach called S-FISTA. Extensive comparison experiments on 17 multi-label benchmark datasets with the 15 top-performing multi-label dimensionality reduction techniques (including a neural network technique and three feature selection methods) show that SMDR-MRC outperforms all of them. • HSIC estimator is reconstructed as the least squares to integrate label propagation. • Learning specific and common features of low-dimensional manifold by sparsification. • Introducing non-zero correlation constrain to minimize redundancy interference. • Integrating instance and low-dimensional manifold correlations into model learning. • S-FISTA optimization technique is proposed to solve the binary quadratic regular model. [ABSTRACT FROM AUTHOR]
Copyright of Knowledge-Based Systems is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Description
Abstract:Multi-label learning, like other machine learning methods, suffers from dimensionality disaster. However, due to the limitations of multi-label dimensionality reduction frameworks, multi-label dimensionality reduction techniques are difficult to effectively implement semi-supervised models, instance correlation constraints, and feature selection and extraction strategies. In this paper, we propose a novel semi-supervised multi-label dimensionality reduction learning approach based on minimizing redundant correlation of specific and common features (SMDR-MRC, in short). Firstly, we employ matrix factorization technique to transform the HSIC-based dimensionality reduction model MDDM (Y. Zhang and Z.-H. Zhou, 2010) into a least squares problem, so that the label propagation mechanism can be naturally integrated into the model. Secondly, cosine similarity and the k -nearest neighbor technique are used to generate constraint terms for low-dimensional manifold correlations and instance correlations, respectively. Finally, to identify the specific and common features of the low-dimensional manifold structures, we introduce two projection weight matrices constrained by the l 1 -norm and l 2 , 1 -norm. In particular, we define a non-zero correlation constraint that effectively minimizes the redundant correlation between the two matrices. To solve the optimization model with nonlinear binary regular terms, we employ a novel solution approach called S-FISTA. Extensive comparison experiments on 17 multi-label benchmark datasets with the 15 top-performing multi-label dimensionality reduction techniques (including a neural network technique and three feature selection methods) show that SMDR-MRC outperforms all of them. • HSIC estimator is reconstructed as the least squares to integrate label propagation. • Learning specific and common features of low-dimensional manifold by sparsification. • Introducing non-zero correlation constrain to minimize redundancy interference. • Integrating instance and low-dimensional manifold correlations into model learning. • S-FISTA optimization technique is proposed to solve the binary quadratic regular model. [ABSTRACT FROM AUTHOR]
ISSN:09507051
DOI:10.1016/j.knosys.2024.111789