Title: A crowdsourced system for user studies in information extraction
Author: Zohreh Khojasteh-Ghamari
Address: Department of Environmental Sciences, Informatics and Statistics, Ca' Foscari University of Venice, Via Torino, 155, 30172 Venezia Mestre, Italy
Abstract: In this paper, from an entity linking (EL) system, we take a set of tweets, where some subsequence of words is annotated with possible meaning/entities and these entities are linked with several Wikipedia pages. We propose a model using crowdsourcing to disambiguate and decide about the accurate Wikipedia page that must be linked with a definite word/spot. We discuss about importance of crowdsourcing and compare different crowdsourcing systems and at the end, introduce crowdflower. We discuss about the crowdflower features in particular. Finally, we analyse output reports of the crowdflower and present a novel approach to select the reliable results. In summary, our observations show that reliable results have a confidence rate over 0.5.
Keywords: crowdsourcing; information extraction; data mining.
Int. J. of Knowledge Engineering and Soft Data Paradigms, 2017 Vol.6, No.1, pp.44 - 51
Date of acceptance: 09 May 2017
Available online: 23 Jan 2018