Title: Content extraction from news web pages using tag tree

Authors: Chandrakala Arya; Sanjay K. Dwivedi

Addresses: Department of Computer Science, Babasaheb Bhimrao Ambedkar University, Lucknow, India ' Department of Computer Science, Babasaheb Bhimrao Ambedkar University, Lucknow, India

Abstract: As the web endures to develop, there is an enormous amount of information which is typically designed for its users, which makes it difficult to extract relevant data from numerous sources. In this paper, we propose an approach for extracting the main content from news web pages. Our approach is based on the concept of tokenisation of HTML page, these tokens construct the tag tree; web pages from different websites are parsed into Tag tree and generated a template from each web pages and discover matching patterns and multiple sequence alignment. It finds and removed shared token sequences from the web pages until the relevant information is extracted from them. We perform experiments on 500 web pages from ten different news websites. Experimental results show that our approach efficiently extracts the relevant information.

Keywords: content extraction; tag tree; news web page; information extraction; pattern matching.

DOI: 10.1504/IJAC.2018.092548

International Journal of Autonomic Computing, 2018 Vol.3 No.1, pp.34 - 51

Received: 13 Jun 2017
Accepted: 20 Nov 2017

Published online: 24 Jun 2018 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article