高级检索+

采用复合知识蒸馏算法的黑皮鸡枞菌图像分级方法

Image classification method for Oudemansiella raphanipes using compound knowledge distillation algorithm

  • 摘要: 为解决黑皮鸡枞菌品质分选速度慢、精度低等问题,该研究提出了一种基于复合知识蒸馏算法的鸡枞菌分级检测方法。采用4 800根黑皮鸡枞菌(共分4级,每级1 200根)图像对教师模型(Resnet50)进行预训练,然后截留教师模型的前25层输出并对学生模型(Resnet18)前9层卷积模型进行参数训练,最后将经过预训练的学生模型的前9层卷积模型与其后半部分拼接,进行整体模型知识蒸馏。经过复合蒸馏的Resnet18识别精度为96.89%,识别单幅图像所用时间为0.032 s。通过对比发现,该研究提出的复合知识蒸馏算法相比Resnet50识别单幅图像所用时间缩短68.93%,相比未经过知识蒸馏及经过单次知识蒸馏的Resnet18模型,精度分别提升了0.97个百分点和0.52个百分点。结果表明,该研究提出的复合知识蒸馏算法可在不增加运行时间的前提下,使小模型的准确率逼近大型模型训练的准确率,研究结果可为鸡枞菌品质分级生产线提供技术支持。

     

    Abstract: In order to solve the problems of time consuming and less accuracy of the Oudemansiella raphanipes, this paper proposes an Oudemansiella raphanipes classification detection method based on a compound knowledge distillation algorithm, which improves the recognition accuracy and calculation efficiency. The knowledge distillation method uses the teacher model (large parameter model) to train the student model (small parameter model). Compared with learning a single correct label, student model can learn the category weights of the predicted target by the knowledge distillation. These category weights are extracted from the teacher model which the student model cannot be obtained through training. At the same time, in order to make full use of feature information, this paper proposes a detection method based on compound knowledge distillation which uses knowledge distillation in different positions of the model. This study uses 4 800 Oudemansiella raphanipes test set images to pre-train the teacher model (Resnet50), then intercept the output of the first 25 layers of the teacher model and perform parameter training on the first 9 layers of the convolutional model of the student model (Resnet18). The student model can continuously adjust the weight information by learning the output of the teacher model during the training process to obtain the best results. Finally, the first 9-layer convolution model of the pre-trained student model and the last half are spliced together to perform knowledge distillation of the overall model. The recognition accuracy of Resnet18 after compound knowledge distillation is 96.89%, and the time to recognize a single image is 0.032 s. It is found that compared with Resnet50, the compound knowledge distillation algorithm proposed in this paper takes 68.93% less time to recognize a single image. Compared with the Resnet18 model without knowledge distillation and single knowledge distillation, the accuracy is improved by 0.97 percent and 0.52 percent points, respectively. The reason for the improvement in accuracy is that when traditional neural network backpropagation parameters are updated, there is a certain distortion phenomenon after each layer of convolutional layer. As the network structure deepens, the first few layers of the model often fail to get effective update signals. And the Resnet network in this paper makes the structure more sensitive to change parameters. At the same time, the knowledge distillation technology can provide the student model with soft label information that cannot be learned on the hard label. The compound knowledge distillation technology proposed in this article allows the first half of the student model to learn high-level feature information in advance, and then parameterize the overall model. It can make it more fully absorb the knowledge in the teacher model, and improve the phenomenon of gradient dispersion of feedback parameters in the transmission process. The results show that the compound knowledge distillation algorithm proposed in this paper can make the accuracy of small model approach large-scale network model without increasing the running time. The research results can provide technical support for the quality grading production line of Oudemansiella raphanipes, and improve the speed and sorting accuracy of agricultural products based on neural network.

     

/

返回文章
返回