基于YOLO v7-ECA模型的苹果幼果检测

宋怀波; 马宝玲; 尚钰莹; 温毓晨; 张姝瑾

基于YOLO v7-ECA模型的苹果幼果检测

宋怀波^1,2,
马宝玲^1,2,
尚钰莹^1,2,
温毓晨^1,2,
张姝瑾^1,2

1. 西北农林科技大学机械与电子工程学院;
2. 农业农村部农业物联网重点实验室

基金项目:

国家重点研发计划项目(2019YFD1002401)

国家自然科学基金项目(31701326)

详细信息

作者简介:
宋怀波(1980—),男,教授,博士生导师,主要从事图像处理理论研究,E-mail:songyangfeifei@163.com

中图分类号: S661.1;TP391.41
计量
- 文章访问数: 0
- HTML全文浏览量: 0
- PDF下载量: 0
出版历程
- 收稿日期: 2022-10-28
- 刊出日期: 2023-06-24

Detection of Young Apple Fruits Based on YOLO v7-ECA Model

1. College of Mechanical and Electronic Engineering, Northwest A&F University;
2. Key Laboratory of Agricultural Internet of Things, Ministry of Agriculture and Rural Affairs

摘要

摘要: 为实现自然环境下苹果幼果的快速准确检测，针对幼果期苹果果色与叶片颜色高度相似、体积微小、分布密集，识别难度大的问题，提出了一种融合高效通道注意力(Efficient channel attention, ECA)机制的改进YOLO v7模型(YOLO v7-ECA)。在模型的3条重参数化路径中插入ECA机制，可在不降低通道维数的前提下实现相邻通道局部跨通道交互，有效强调苹果幼果重要信息、抑制冗余无用特征，提高模型效率。采集自然环境下苹果幼果图像2 557幅作为训练样本、547幅作为验证样本、550幅作为测试样本，输入模型进行训练测试。结果表明，YOLO v7-ECA网络模型准确率为97.2%、召回率为93.6%、平均精度均值(Mean average precision, mAP)为98.2%、F1值为95.37%。与Faster R-CNN、SSD、Scaled-YOLO v4、YOLO v5、YOLO v6、YOLO v7网络模型相比，其mAP分别提高15.5、4.6、1.6、1.8、3.0、1.8个百分点，准确率分别提高49.7、0.9、18.5、1.2、0.9、1.0个百分点，F1值分别提高33.53、2.81、9.16、1.26、2.38、1.43个百分点，召回率相较于Faster R-CNN、SSD、YOLO v5、YOLO v6、YOLO v7网络模型分别提高5.0、4.5、1.3、3.7、1.8个百分点；单幅图像检测时间为28.9 ms,可实现苹果幼果的高效检测。针对幼果目标模糊、存在阴影和严重遮挡的情况，本研究采用550幅测试图像进行模型鲁棒性检验。在加噪模糊情况下，YOLO v7-ECA的mAP为91.1%,F1值为89.8%,与Faster R-CNN、SSD、Scaled-YOLO v4、YOLO v5、YOLO v6、YOLO v7网络模型相比其mAP分别提高26.3、21.0、5.4、8.0、11.5、8.9个百分点，F1值分别提高27.19、7.08、8.50、4.20、3.94、4.67个百分点；在阴影情况下，YOLO v7-ECA的mAP为97.5%,F1值为95.36%,与Faster R-CNN、SSD、Scaled-YOLO v4、YOLO v5、YOLO v6、YOLO v7网络模型相比其mAP分别提高14.8、8.8、2.1、2.4、5.4、2.5个百分点，F1值分别提高21.51、2.60、10.49、1.53、3.23、2.56个百分点；在严重遮挡情况下，YOLO v7-ECA的mAP为98.6%,F1值为94.8%,与Faster R-CNN、SSD、Scaled-YOLO v4、YOLO v5、YOLO v6、YOLO v7网络模型相比其mAP分别提高21.7、13.7、2.3、2.4、4.8、2.2个百分点，F1值分别提高28.29、3.50、6.45、0.96、1.36、1.36个百分点。该网络模型可在保证网络模型精度的同时拥有较快的检测速度，且对场景模糊、阴影和严重遮挡等影响具有较好的鲁棒性。该研究可为幼果实时检测系统提供有效借鉴。
- 苹果幼果 /
- 检测 /
- YOLO v7 /
- 高效通道注意力机制 /
- 机器视觉
Abstract: In order to detect young apple fruits quickly and accurately in the natural environment, an improved YOLO v7 model（YOLO v7-ECA） was proposed to solve the problems of high similarity, small size, dense distribution and difficult identification between young apple fruits and leaves. By inserting the ECA mechanism into the three reparameterized paths of the model, the local cross-channel interaction of adjacent channels could be carried out without reducing the channel dimension, which can effectively emphasize the important information of young apple fruits, suppress redundant and useless features, and improve the efficiency of the model. Totally 2 557 images of young apple fruits were collected as training samples, totally 547 images as validation samples, and 550 images as test samples in the natural environment, and input them into the model for training and testing. The YOLO v7-ECA model was trained to have a precision of 97.2%, a recall rate of 93.6%, an mAP of 98.2%, and F1 value of 95.37%. Compared with the Faster R-CNN, SSD, Scaled-YOLO v4, YOLO v5, YOLO v6, YOLO v7 models, its mAP was increased by 15.5, 4.6, 1.6, 1.8, 3.0 and 1.8 percentage points, its precision was increased by 49.7, 0.9, 18.5, 1.2, 0.9 and 1.0 percentage points, its F1 value was increased by 33.53, 2.81, 9.16, 1.26, 2.38 and 1.43 percentage points, and its recall rate was increased by 5.0, 4.5, 1.3, 3.7 and 1.8 percentage points for Faster R-CNN, SSD, YOLO v5, YOLO v6 and YOLO v7 models, respectively; the image detection time was 28.9 ms, which could realize efficient detection of young apple fruits. Aiming at the fuzzy, shadowing and severe occlusion of young fruit targets, totally 550 test images were used to test the robustness of the model. The mAP of YOLO v7-ECA was 91.1% and the F1 value was 89.8% under the condition of adding noise and fuzziness. Compared with the Faster R-CNN, SSD, Scaled-YOLO v4, YOLO v5, YOLO v6 and YOLO v7 models, its mAP was increased by 26.3, 21.0, 5.4, 8.0, 11.5 and 8.9 percentage points, and its F1 value was increased by 27.19, 7.08, 8.50, 4.20, 3.94 and 4.67 percentage points, respectively. The mAP of YOLO v7-ECA was 97.5% and the F1 value was 95.36% in the shadow. Compared with the Faster R-CNN, SSD, Scaled-YOLO v4, YOLO v5, YOLO v6 and YOLO v7 models, its mAP was increased by 14.8, 8.8, 2.1, 2.4, 5.4 and 2.5 percentage points, and its F1 value was increased by 21.51, 2.60, 10.49, 1.53, 3.23 and 2.56 percentage points, respectively. The mAP of YOLO v7-ECA was 98.6% and the F1 value was 94.8% under severe occlusion. Compared with that of the Faster R-CNN, SSD, Scaled-YOLO v4, YOLO v5, YOLO v6 and YOLO v7 models, its mAP was increased by 21.7, 13.7, 2.3, 2.4, 4.8 and 2.2 percentage points, and its F1 value was increased by 28.29, 3.50, 6.45, 0.96, 1.36 and 1.36 percentage points, respectively. Experiments showed that the proposed model was of high accuracy and speed, it was also robust to different interference situations such as blurred scene, shadow and severe occlusion. The research result can provide an effective reference for the detection system of apple young fruit.
- young apple fruit /
- detection /
- YOLO v7 /
- efficient channel attention mechanisms /
- machine vision

HTML全文

参考文献(29)

[1]	宋怀波,江梅,王云飞,等.融合卷积神经网络与视觉注意机制的苹果幼果高效检测方法[J].农业工程学报,2021,37(9):297-303.SONG Huaibo,JIANG Mei,WANG Yunfei,et al.Efficient detection method for young apples based on the fusion of convolutional neural network and visual attention mechanism[J].Transactions of the CSAE,2021,37(9):297-303.(in Chinese)
[2]	王丹丹,何东健.基于R-FCN深度卷积神经网络的机器人疏果前苹果目标的识别[J].农业工程学报,2019,35(3):156-163.WANG Dandan,HE Dongjian.Recognition of apple targets before fruits thinning by robot based on R-FCN deep convolution neural network[J].Transactions of the CSAE,2019,35(3):156-163.(in Chinese)
[3]	潘云飞,周艳,何磊,等.果园管理工作中疏花疏果的研究进展[J].中国农机化学报,2021,42(11):198-204.PAN Yunfei,ZHOU Yan,HE Lei,et al.Research progress of flower and fruit thinning in orchard management[J].Journal of Chinese Agricultural Mechanization,2021,42(11):198-204.(in Chinese)
[4]	JIA W,TIAN Y,LUO R,et al.Detection and segmentation of overlapped fruits based on optimized Mask R-CNN application in apple harvesting robot[J].Computers and Electronics in Agriculture,2020,172:105380.
[5]	宋怀波,尚钰莹,何东健.果实目标深度学习识别技术研究进展[J].农业机械学报,2023,54(1):1-19.SONG Huaibo,SHANG Yuying,HE Dongjian.Review on deep learning technology for fruit target recognition[J].Transactions of the Chinese Society for Agricultural Machinery,2023,54(1):1-19.(in Chinese)
[6]	WU D,LV S,JIANG M,et al.Using channel pruning-based YOLO v4 deep learning algorithm for the real-time and accurate detection of apple flowers in natural environments[J].Computers and Electronics in Agriculture,2020,178:105742.
[7]	尚钰莹,张倩如,宋怀波.基于YOLOv5s的深度学习在自然场景苹果花朵检测中的应用[J].农业工程学报,2022,38(9):222-229.SHANG Yuying,ZHANG Qianru,SONG Huaibo.Application of deep learning using YOLOv5s to apple flower detection in natural scenes[J].Transactions of the CSAE,2022,38(9):222-229.(in Chinese)
[8]	ZHANG W,WANG J,LIU Y,et al.Deep-learning-based in-field citrus fruit detection and tracking[J].Horticulture Research,2022,9:uhac003.
[9]	WANG Z,JIN L,WANG S,et al.Apple stem/calyx real-time recognition using YOLO-v5 algorithm for fruit automatic loading system[J].Postharvest Biology and Technology,2022,185:111808.
[10]	FU L,FENG Y,WU J,et al.Fast and accurate detection of kiwifruit in orchard using improved YOLOv3-tiny model[J].Precision Agriculture,2021,22(3):754-776.
[11]	宋怀波,王亚男,王云飞,等.基于YOLO v5s的自然场景油茶果识别方法[J].农业机械学报,2022,53(7):234-242.SONG Huaibo,WANG Ya'nan,WANG Yunfei,et al.Camellia oleifera fruit detection in natural scene based on YOLO v5s[J].Transactions of the Chinese Society for Agricultural Machinery,2022,53(7):234-242.(in Chinese)
[12]	刘天真,滕桂法,苑迎春,等.基于改进YOLO v3的自然场景下冬枣果实识别方法[J].农业机械学报,2021,52(5):17-25.LIU Tianzhen,TENG Guifa,YUAN Yingchun,et al.Winter jujube fruit recognition method based on improved YOLO v3 under natural scene[J].Transactions of the Chinese Society for Agricultural Machinery,2021,52(5):17-25.(in Chinese)
[13]	龙燕,李南南,高研,等.基于改进FCOS网络的自然环境下苹果检测[J].农业工程学报,2021,37(12):307-313.LONG Yan,LI Nannan,GAO Yan,et al.Apple fruit detection under natural condition using improved FCOS network[J].Transactions of the CSAE,2021,37(12):307-313.(in Chinese)
[14]	何斌,张亦博,龚健林,等.基于改进YOLO v5的夜间温室番茄果实快速识别[J].农业机械学报,2022,53(5):201-208.HE Bin,ZHANG Yibo,GONG Jianlin,et al.Fast recognition of tomato fruit in greenhouse at night based on improved YOLO v5[J].Transactions of the Chinese Society for Agricultural Machinery,2022,53(5):201-208.(in Chinese)
[15]	赵辉,乔艳军,王红君,等.基于改进YOLOv3的果园复杂环境下苹果果实识别[J].农业工程学报,2021,37(16):127-135.ZHAO Hui,QIAO Yanjun,WANG Hongjun,et al.Apple fruit recognition in complex orchard environment based on improved YOLOv3[J].Transactions of the CSAE,2021,37(16):127-135.(in Chinese)
[16]	王立舒,秦铭霞,雷洁雅,等.基于改进YOLOv4-Tiny的蓝莓成熟度识别方法[J].农业工程学报,2021,37(18):170-178.WANG Lishu,QIN Mingxia,LEI Jieya,et al.Blueberry maturity recognition method based on improved YOLOv4-Tiny[J].Transactions of the CSAE,2021,37(18):170-178.(in Chinese)
[17]	王卓,王健,王枭雄,等.基于改进YOLO v4的自然环境苹果轻量级检测方法[J].农业机械学报,2022,53(8):294-302.WANG Zhuo,WANG Jian,WANG Xiaoxiong,et al.Lightweight real-time apple detection method based on improved YOLO v4[J].Transactions of the Chinese Society for Agricultural Machinery,2022,53(8):294-302.(in Chinese)
[18]	JIANG M,SONG L,WANG Y,et al.Fusion of the YOLOv4 network model and visual attention mechanism to detect low-quality young apples in a complex environment[J].Precision Agriculture,2022,23(2):559-577.
[19]	SU F,LU Q,LUO R Z.Review of image classification based on deep learning[J].Telecommunication Science,2019,35(11):58-74.
[20]	ZAIDI S S A,ANSARI M S,ASLAM A,et al.A survey of modern deep learning based object detection models[J].Digital Signal Processing,2022,126:103514.
[21]	TONG K,WU Y,ZHOU F.Recent advances in small object detection based on deep learning:a review[J].Image and Vision Computing,2020,97:103910.
[22]	XIAO Y,TIAN Z,YU J,et al.A review of object detection based on deep learning[J].Multimedia Tools and Applications,2020,79(33):23729-23791.
[23]	WU X,SAHOO D,HOI S C H.Recent advances in deep learning for object detection[J].Neurocomputing,2020,396:39-64.
[24]	MO Y,WU Y,YANG X,et al.Review the state-of-the-art technologies of semantic segmentation based on deep learning[J].Neurocomputing,2022,493:626-646.
[25]	HU J,SHEN L,SUN G,et al.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2018:7132-7141.
[26]	WOO S,PARK J,LEE J Y,et al.Cbam:convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision (ECCV),2018:3-19.
[27]	HOU Q,ZHOU D,FENG J.Coordinate attention for efficient mobile network design[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:13713-13722.
[28]	WANG Q L,WU B G,ZHU P F,et al.ECA-Net:efficient channel attention for deep convolutional neural networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),2020:11531-11539.
[29]	WANG C Y,BOCHKOVSKIY A,LIAO H Y M.YOLOv7:trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[J].arXiv e-prints,arXiv:2207.02696,2022.