Harnessing the power of hugging face's multilingual transformers: unravelling the code-mixed named entity recognition enigma
by Rejuwan Shamim; Asadullah Shaikh
International Journal of Intelligent Engineering Informatics (IJIEI), Vol. 12, No. 3, 2024

Abstract: Named entity recognition (NER) in code-mixed documents, which have different languages, is hard for natural language processing. In this paper, we use hugging face's multilingual transformers to come up with a way to do code-mixed NER without any problems. Our work tries to solve the problems that come up when you try to recognise named entities in more than one language within the same text. We did thorough tests by fine-tuning the multilingual transformer model on a dataset with mixed codes. With an F1-score of 0.85, we got great results. This works better than previous methods and proves that our model can accurately find named items. We also examine at how well the model works with other language pairs and code-mixed patterns. This shows how well the model can handle different language situations. Our study helps us understand how to handle data in multiple languages, makes code-mixed NER techniques better, and shows how multilingual transformers can help break down language barriers. The research has implications for areas that need to understand more than one language, such as analysing social media, creating language-specific customer service systems, and finding information across languages. Speaking different languages can communicate more easily and effectively in these fields, which encourage inclusion.

Online publication date: Fri, 26-Jul-2024

The full text of this article is only available to individual subscribers or to users at subscribing institutions.

 
Existing subscribers:
Go to Inderscience Online Journals to access the Full Text of this article.

Pay per view:
If you are not a subscriber and you just want to read the full contents of this article, buy online access here.

Complimentary Subscribers, Editors or Members of the Editorial Board of the International Journal of Intelligent Engineering Informatics (IJIEI):
Login with your Inderscience username and password:

Username:        Password:          Forgotten your password?

Want to subscribe?
A subscription gives you complete access to all articles in the current issue, as well as to all articles in the previous three years (where applicable). See our Orders page to subscribe.

If you still need assistance, please email subs@inderscience.com