Title: Web toolkit: an agent based scalable search engine using cellular automata based classification for duly ranked retrieved data

Authors: Anirban Kundu, Debajyoti Mukhopadhyay

Addresses: Netaji Subhash Engineering College, West Bengal University of Technology, West Bengal-700 152, India; Web Intelligence and Distributed Computing Research Lab, (WIDiCoReL), Green Tower C-9/1, Golf Green, Calcutta 700095, India. ' Calcutta Business School, Diamond Harbour Road, Bishnupur, West Bengal-743503, India; Web Intelligence and Distributed Computing Research Lab, (WIDiCoReL), Green Tower C-9/1, Golf Green, Calcutta 700095, India

Abstract: Web page classification is a major issue for categorising web documents to facilitate indexing, search and retrieval of web pages for search engine. Different crawling techniques have been utilised to accumulate web pages of different domains under separate databases depending on practical scenario. Downloaded web pages are being parsed for further processing. A classifier is designed dynamically using single cycle multiple attractor cellular automata for mapping downloaded web pages of different domains into specific structure. This paper proposes alternate technique for automatic categorisation of web pages into different domains. Retrieved web pages have been ranked automatically at the time of classifier formation. Typically, our system consists of crawling, ranking and storage parts created in a different way. Hierarchical concept has been used over parallel crawler. GF(2P) concept is introduced in ranking. The concept of SMACA has been utilised in indexing storage. Overall, a search engine module has been created using agent-based method.

Keywords: cellular automata; Galois field; hierarchical crawlers; page ranking; single cycle multiple attractor cellular automata; SMACA; agent-based systems; search engines; multi-agent systems; web page classification; web documents; indexing; information retrieval; web pages.

DOI: 10.1504/IJIIDS.2011.038971

International Journal of Intelligent Information and Database Systems, 2011 Vol.5 No.2, pp.177 - 199

Received: 21 Jan 2010
Accepted: 12 Jun 2010
Published online: 21 Oct 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article