Title: Q-grams-imp: an improved q-grams algorithm aimed at edit similarity join

Authors: Yunxia Liu; Zhaobin Liu; Zhiyang Li

Addresses: School of Information Science and Technology, Dalian Maritime University, Dalian, 116026, China ' School of Information Science and Technology, Dalian Maritime University, Dalian, 116026, China ' School of Information Science and Technology, Dalian Maritime University, Dalian, 116026, China

Abstract: Similarity join is more and more important in many applications and has attracted wide-spread attention from scholars and communities. Similarity join has been used in many applications, such as spell checking, copy detection, entity linking, pattern recognition and so on. Actually, in many web and enterprise scenarios, where typos and misspellings often occur, we need to find an efficient algorithm to handle these situations. In this paper, we propose an improved algorithm on q-grams called q-grams-imp that is aimed at solving edit similarity join. We use this algorithm in order to reduce the number of tokens and thus reduce space costs, so it is fit best for same size strings. But for different sizes of strings, we need to handle these strings in order to fit for the algorithm. Finally, we conclude and get the results that our proposed algorithm is better than the traditional method.

Keywords: similarity join; q-grams algorithm; edit distance.

DOI: 10.1504/IJCSE.2019.098537

International Journal of Computational Science and Engineering, 2019 Vol.18 No.3, pp.269 - 278

Received: 30 Jun 2016
Accepted: 08 Sep 2016

Published online: 26 Mar 2019 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article