Article: An improved BERT model feature representation method for defect text classification in power system Journal: International Journal of Computational Systems Engineering (IJCSYSE) 2026 Vol.10 No.7 pp.13 - 25 Abstract: In the text representation, although the BERT model can solve problem of the multivocal word in word2vec, the masking training of single word in BERT pre-training task separates the correlation between continuous words, which makes it difficult for the model to effectively learn the semantic information of words. Therefore, a classification method of defect text based on improved BERT model for feature representation is proposed by this paper. Firstly, the input layer of the BERT model is improved by linking and extracting entities, fusing token and entities for merging features and then conducting the pre-training task. Then, the improved BERT model is used to train and generate dynamic word vector to learn the semantic information with entity knowledge. Experiment results demonstrate that the defect text representation vectors trained by the proposed BERT model can enhance the accuracy for defect text classification. Inderscience Publishers - linking academia, business and industry through research

Title: An improved BERT model feature representation method for defect text classification in power system

Authors: Ping Wang; Ning Fu; Fangkai Cai; Shiliu Yang

Addresses: School of Network and Communication Engineering, Chengdu Technological University, Chengdu 610000, China ' School of Network and Communication Engineering, Chengdu Technological University, Chengdu 610000, China ' School of Network and Communication Engineering, Chengdu Technological University, Chengdu 610000, China ' School of Network and Communication Engineering, Chengdu Technological University, Chengdu 610000, China

Abstract: In the text representation, although the BERT model can solve problem of the multivocal word in word2vec, the masking training of single word in BERT pre-training task separates the correlation between continuous words, which makes it difficult for the model to effectively learn the semantic information of words. Therefore, a classification method of defect text based on improved BERT model for feature representation is proposed by this paper. Firstly, the input layer of the BERT model is improved by linking and extracting entities, fusing token and entities for merging features and then conducting the pre-training task. Then, the improved BERT model is used to train and generate dynamic word vector to learn the semantic information with entity knowledge. Experiment results demonstrate that the defect text representation vectors trained by the proposed BERT model can enhance the accuracy for defect text classification.

Keywords: improved BERT model; defect text; feature representation; classification method.

DOI: 10.1504/IJCSYSE.2026.152922

International Journal of Computational Systems Engineering, 2026 Vol.10 No.7, pp.13 - 25

Accepted: 05 Feb 2026
Published online: 14 Apr 2026 *

Title: An improved BERT model feature representation method for defect text classification in power system

Keep up-to-date