Title: Annotation tools for syntax and named entities in the National Corpus of Polish

Authors: Jakub Waszczuk; Katarzyna Głowińska; Agata Savary; Adam Przepiórkowski; Michał Lenart

Addresses: Institute of Computer Science, Polish Academy of Sciences, ul. Ordona 21, 01-237 Warsaw, Poland; Institute of Informatics, University of Warsaw, ul. Banacha 2, 02-097 Warsaw, Poland ' Institute of Computer Science, Polish Academy of Sciences, ul. Ordona 21, 01-237 Warsaw, Poland ' Laboratoire d'Informatique, Université François Rabelais Tours, 3 pl. Jean-Jaurés, 41000 Blois, France ' Institute of Computer Science, Polish Academy of Sciences, ul. Ordona 21, 01-237 Warsaw, Poland; Institute of Informatics, University of Warsaw, ul. Banacha 2, 02-097 Warsaw, Poland ' Institute of Computer Science, Polish Academy of Sciences, ul. Ordona 21, 01-237 Warsaw, Poland; Institute of Informatics, University of Warsaw, ul. Banacha 2, 02-097 Warsaw, Poland

Abstract: The ongoing National Corpus of Polish project assumes several levels of linguistic annotation. We present the technical environment and methodological background developed for the three upper annotation levels: the levels of syntactic words, syntactic groups and named entities. We show how knowledge-based platforms Spejd and Sprout are used for the automatic pre-annotation of the corpus and discuss some particular problems faced during the preparation of the parser grammar, which contains over 1,000 rules and is one of the largest chunking grammars for Polish. We also show how the tree editor TrEd has been customised for manual post-editing of annotations and for further revision of discrepancies. Our XML format converters and customised archiving repository ensure an automatic data flow and efficient corpus file management. We discuss the inter-annotator agreement in the manually annotated data, and present the first results of a CRF classifier trained on these data.

Keywords: corpus annotation; National Corpus of Polish; shallow parsing; chunking grammars; named entity recognition; NER; syntax; named entities; linguistic annotation; syntactic words; syntactic groups; parser grammar; XML converters; customised archiving repository; automatic data flow; file management.

DOI: 10.1504/IJDMMM.2013.053691

International Journal of Data Mining, Modelling and Management, 2013 Vol.5 No.2, pp.103 - 122

Published online: 29 Jul 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article