Title: Study on fast collection method of massive marketing data based on crawler technology

Authors: Shuiying Hu

Addresses: Department of Business Administration, Henan University of Animal Husbandry and Economy, Zhengzhou 450000, China

Abstract: In this paper, a fast collection method of massive marketing data based on crawler technology is proposed. According to the variable characteristics of marketing data, the normal distribution model is used to extract the features of marketing data, and the fusion algorithm is used to fuse the features of marketing data. The first step is to setup the network node of marketing data collection, determine the collection location of marketing data, take the fused data as the initial URL, then join the crawler queue to judge the marketing data whose similarity meets the collection requirements, and finally, crawl the data whose similarity meets the requirements again to complete the rapid collection of massive marketing data. The experimental results show that the proposed method takes less than 10 seconds to collect the experimental sample data, and the error is less than 5%.

Keywords: crawler technology; marketing data; fast acquisition; similarity; mass data.

DOI: 10.1504/IJICT.2023.134251

International Journal of Information and Communication Technology, 2023 Vol.23 No.3, pp.242 - 252

Received: 13 Jul 2021
Accepted: 17 Aug 2021
Published online: 15 Oct 2023 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article