View in EDS

Unconventional application of k-means for distributed approximate similarity search.

Saved in:
Bibliographic Details
Title: Unconventional application of k-means for distributed approximate similarity search.
Authors: Ortega, Felipe1 (AUTHOR) felipe.ortega@urjc.es, Algar, Maria Jesus1 (AUTHOR) mariajesus.algar@urjc.es, de Diego, Isaac Martín1 (AUTHOR) isaac.martin@urjc.es, Moguerza, Javier M.1 (AUTHOR) javier.moguerza@urjc.es
Source: Information Sciences. Jan2023, Vol. 620, p208-234. 27p.
Subjects: Indexing, Metric spaces, Computing platforms, Distributed computing, Function spaces, Machine learning
Abstract: • This paper presents MASK, a multilevel algorithm for approximate similarity search • MASK can distribute the index over as many computing nodes as we can afford. • Experimental results show the applicability of this novel indexing method • MASK achieves superior performance with high-dimensional and high-sparsity datasets. Similarity search based on a distance function in metric spaces is a fundamental problem for many applications. Queries for similar objects lead to the well-known machine learning task of nearest-neighbours identification. Many data indexing strategies, collectively known as Metric Access Methods (MAM), have been proposed to speed up these queries. Moreover, since exact approaches to solving similarity queries can be complex and time-consuming, alternative options have emerged to reduce query execution time, such as returning approximate results or resorting to distributed computing platforms. In this paper, we introduce MASK (Multilevel Approximate Similarity search with k -means), an unconventional application of the k -means algorithm as the foundation of a multilevel index structure for approximate similarity search suitable for metric spaces. We show that this method leverages inherent properties of k -means for this purpose, like representing high-density data areas with fewer prototypes. An implementation of this new indexing procedure is evaluated using a synthetic dataset and two real-world datasets in high-dimensional and high-sparsity spaces. Experimental tests show that MASK performs better than alternative algorithms for approximate similarity search. Results are promising and underpin the applicability of this novel indexing method in multiple domains. [ABSTRACT FROM AUTHOR]
Copyright of Information Sciences is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Description
Abstract:• This paper presents MASK, a multilevel algorithm for approximate similarity search • MASK can distribute the index over as many computing nodes as we can afford. • Experimental results show the applicability of this novel indexing method • MASK achieves superior performance with high-dimensional and high-sparsity datasets. Similarity search based on a distance function in metric spaces is a fundamental problem for many applications. Queries for similar objects lead to the well-known machine learning task of nearest-neighbours identification. Many data indexing strategies, collectively known as Metric Access Methods (MAM), have been proposed to speed up these queries. Moreover, since exact approaches to solving similarity queries can be complex and time-consuming, alternative options have emerged to reduce query execution time, such as returning approximate results or resorting to distributed computing platforms. In this paper, we introduce MASK (Multilevel Approximate Similarity search with k -means), an unconventional application of the k -means algorithm as the foundation of a multilevel index structure for approximate similarity search suitable for metric spaces. We show that this method leverages inherent properties of k -means for this purpose, like representing high-density data areas with fewer prototypes. An implementation of this new indexing procedure is evaluated using a synthetic dataset and two real-world datasets in high-dimensional and high-sparsity spaces. Experimental tests show that MASK performs better than alternative algorithms for approximate similarity search. Results are promising and underpin the applicability of this novel indexing method in multiple domains. [ABSTRACT FROM AUTHOR]
ISSN:00200255
DOI:10.1016/j.ins.2022.11.024