Research on named entity recognition for combine harvester fault domain
-
摘要: 联合收割机作为一种机械化设备不可避免地会出现机械故障，为快速地找出并解决机械故障，提出一种面向联合收割机故障领域的命名实体识别模型RP-TEBC(RoBERTa-wwm-ext+PGD+Transformer-Encoder+BiGRU+CRF)。RP-TEBC使用动态编码的RoBERTa-wwm-ext预训练模型作为词嵌入层，利用自适应Transformer编码器层融合双向门控单元（BiGRU）作为上下文编码器，利用条件随机场（CRF）作为解码层，使用维特比算法找出最优的路径输出。同时，RP-TEBC模型在词嵌入层中通过添加一些扰动，生成对抗样本，经过对模型不断的训练优化，可以提高模型整体的鲁棒性和泛化性能。结果表明，在构建的联合收割机故障领域命名实体识别数据集上，相比于基线模型，该模型的准确率、召回率、F1值分别提高1.79%、1.01%、1.46%。Abstract: Combine harvesters as a kind of mechanized equipment will inevitably have mechanical failure, in order to quickly find out the relevant fault entity and solve the mechanical failure, a named entity recognition model RP-TEBC（RoBERTa-wwm-ext+PGD+Transformer-Encoder+BiGRU+CRF） for combine harvester fault field is proposed.RP-TEBC uses the dynamically encoded RoBERTa-wwm-ext pre-trained model as the word embedding layer, uses the adaptive Transformer encoder layer to fuse the Bidirectional Gating Unit（BiGRU） as the context encoder, and finally uses the conditional random field（CRF） as the decoder layer, using the Viterbi algorithm to find the optimal path output.At the same time, the RP-TEBC model generates adversarial samples by adding some perturbations in the word embedding layer. Through continuous training and optimization of the model, the overall robustness and generalization performance of the model can be improved. On the constructed named entity recognition data set in the field of combine harvester faults, experiments have shown that compared with the baseline model, the accuracy, recall rate, and F1 value of this model have increased by 1. 79%, 1. 01%, and 1. 46% respectively.
-
-
[1] Hammerton J. Named entity recognition with long short-term memory[C]. Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, 2003:172-175.
[2] Collobert R, Weston J, Bottou L, et al. Natural language processing(almost)from scratch[J]. Journal of Machine Learning Research, 2011, 12:2493-2537.
[3] Huang Z, Xu W, Yu K. Bidirectional LSTM-CRF models for sequence tagging[J]. arxiv preprint arxiv:1508. 01991, 2015.
[4] Dong C, Zhang J, Zong C, et al. Character-based LSTMCRF with radical-level features for Chinese named entity recognition[C]. Natural Language Understanding and Intelligent Applications:5th CCF Conference on Natural Language Processing and Chinese Computing, NLPCC 2016,and 24th International Conference on Computer Processing of Oriental Languages, ICCPOL 2016, Kunming, China,December 2-6, 2016, Proceedings 24. Springer International Publishing, 2016:239-250.
[5] Ma R, Peng M, Zhang Q, et al. Simplify the usage of lexicon in Chinese NER[J]. arxiv preprint arxiv:1908.05969, 2019.
[6] Zhou J T, Zhang H, Jin D, et al. Dual adversarial neural transfer for low-resource named entity recognition[C].Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019:3461-3471.
[7] Devlin J, Chang M W, Lee K, et al. Bert:Pre-training of deep bidirectional transformers for language understanding[J].arxiv preprint arxiv:1810. 04805, 2018.
[8] Cui Y, Che W, Liu T, et al. Pre-training with whole word masking for chinese bert[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29:3504-3514.
[9] Yan H, Deng B, Li X, et al. TENER:Adapting transformer encoder for named entity recognition[J]. arxiv preprint arxiv:1911. 04474, 2019.
[10] Madry A, Makelov A, Schmidt L, et al. Towards deep learning models resistant to adversarial attacks[J]. arxiv preprint arxiv:1706. 06083, 2017.
-
期刊类型引用(1)
1. 黄鑫光，王宪良，孟志军，凌琳，肖跃进，武广伟，罗长海，颜丙新. 玉米电驱精量播种机作业工况参数检测系统研究. 农业机械学报. 2025(04): 61-71 . 
百度学术
其他类型引用(0)
计量
- 文章访问数: 2
- HTML全文浏览量: 0
- PDF下载量: 0
- 被引次数: 1
下载:
