An improved lightweight network based on deep learning for grape recognition in unstructured environments

Bingpiao Liu; Yunzhi Zhang; Jinhai Wang; Lufeng Luo; Qinghua Lu; Huiling Wei; Wenbo Zhu

doi:10.1016/j.inpa.2023.02.003

Information Processing in Agriculture > 2024 > 11(2): 202-216. > DOI: 10.1016/j.inpa.2023.02.003

Bingpiao Liu, Yunzhi Zhang, Jinhai Wang, Lufeng Luo, Qinghua Lu, Huiling Wei, Wenbo Zhu. An improved lightweight network based on deep learning for grape recognition in unstructured environments[J]. Information Processing in Agriculture, 2024, 11(2): 202-216. DOI: 10.1016/j.inpa.2023.02.003

Citation:

PDF (5164 KB)

An improved lightweight network based on deep learning for grape recognition in unstructured environments

School of Mechatronic Engineering and Automation, Foshan University, Foshan 528000, China

Funds:

This work was supported by grants from the National Natural Science Foundation of Chima (32171909, 51705365), Guangdong Basic and Applied Basic Research Foundation (2020B1515120050, 2019A1515110304), National Natural Science Foundation of Guangdong (2023A1515011255), Yunfu Science and Technology Plan Project (2021A090103), and special Project in Key Fields of Universities in Guangdong Province (2022ZDZX309).

More Information

Received Date: April 17, 2022
Revised Date: August 03, 2022
Accepted Date: February 10, 2023
Published Date: February 19, 2023

Graphical Abstract

Abstract

Abstract

In unstructured environments, dense grape fruit growth and the presence of occlusion cause difficult recognition problems, which will seriously affect the performance of grape picking robots. To address these problems, this study improves the YOLOX-Tiny model and proposes a new grape detection model, YOLOX-RA, which can quickly and accurately identify densely growing and occluded grape bunches. The proposed YOLOX-RA model uses a 3 × 3 convolutional layer with a step size of 2 to replace the focal layer to reduce the computational burden. The CBS layer in the ResBlock_Body module of the second, third, and fourth layers of the backbone layer is removed, and the CSPLayer module is replaced by the ResBlock-M module to speed up the detection. An auxiliary network (AlNet) with the remaining network blocks was added after the ResBlock-M module to improve the detection accuracy. Two depth-separable convolutions (DSC) are used in the neck module layer to replace the normal convolution to reduce the computational cost. We evaluated the detection performance of SSD, YOLOv4 SSD, YOLOv4-Tiny, YOLO-Grape, YOLOv5-X, YOLOX-Tiny, and YOLOX-RA on a grape test set. The results show that the YOLOX-RA model has the best detection performance, achieving 88.75 % mAP, a recognition speed of 84.88 FPS, and model size of 17.53 MB. It can accurately detect densely grown and shaded grape bunches, which can effectively improve the performance of the grape picking robot.
- YOLOX,
- Grape recognition,
- Depthwise separable convolution,
- ResBlock-M,
- AlNet

FullText(HTML)

References (37)

References

[1]	Lin G, Tang Y, Zou X, Xiong J, Li J. In-field pose estimation of grape clusters with combined point cloud segmentation and geometric analysis. Comput. Electron. Agric 2022; 107197.
[2]	Wu G, Zhu Q, Huang M, Guo Y, Qin J. Automatic recognition of juicy peaches on trees based on 3D contour features and color data. Biosystems Engineering 2019;188, 1-13. https://doi.org/10.1016/j.biosystemseng.2019.10.002.
[3]	Liu S, Whitty M. Automatic grape bunch detection in vineyards with an SVM classifier. Journal of Applied Logic 2015;13(4), 643-653. https://doi.org/10.1016/j.jal.2015.06.001.
[4]	Pérez-Zavala R, Torres-Torriti M, Cheein F A, Troni G. A pattern recognition strategy for visual grape bunch detection in vineyards. Computers and Electronics in Agriculture 2018;151, 136-149. https://doi.org/10.1016/j.compag.2018.05.019.
[5]	Luo L, Tang Y, Lu Q, Chen X, Zhang P, Zou X. A vision methodology for harvesting robot to detect cutting points on peduncles of double overlapping grape clusters in a vineyard. Computers in Industry 2018;99, 130-139. https://doi.org/10.1016/j.compind.2018.03.017.
[6]	Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell 2017;39, 1137–1149.
[7]	Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C.-Y, et al. SSD: Single Shot MultiBox Detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (Eds.), Computer Vision – ECCV 2016. Springer International Publishing, Cham 2016; pp. 21–37.
[8]	Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition 2016; (pp. 779-788).
[9]	Redmon J, Farhadi A. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767 2018. https://doi.org/10.48550/arXiv.1804.02767.
[10]	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In Paper presented at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2015.
[11]	He K, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. In Proceedings of the IEEE international conference on computer vision 2017;(pp. 2961-2969).
[12]	Chen Z, Ting D, Newbury R, Chen C. Semantic segmentation for partially occluded apple trees based on deep learning. Computers and Electronics in Agriculture 2021;181, 105952. https://doi.org/10.1016/j.compag.2020.105952.
[13]	Yin W, Wen H, Ning Z, Ye J, Dong Z, Luo L. Fruit Detection and Pose Estimation for Grape Cluster–Harvesting Robot Using Binocular Imagery Based on Deep Neural Networks. Frontiers in Robotics and AI 2021;8. https://doi.org/10.3389/frobt.2021.626989.
[14]	Kang H, Chen C. Fast implementation of real-time fruit detection in apple orchards using deep learning. Computers and Electronics in Agriculture 2020;168, 105108. https://doi.org/10.1016/j.compag.2019.105108.
[15]	Lin G, Tang Y, Zou X, Wang C. Three-dimensional reconstruction of guava fruits and branches using instance segmentation and geometry analysis. Computers and Electronics in Agriculture 2021;184, 106107. https://doi.org/10.1016/j.compag.2021.106107.
[16]	Gené-Mola J, Sanz-Cortiella R, Rosell-Polo J R, Morros J R, Ruiz-Hidalgo J, Vilaplana V, et al. Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry. Computers and Electronics in Agriculture 2020;169, 105165. https://doi.org/10.1016/j.compag.2019.105165.
[17]	Wan S, Goudas S. Faster R-CNN for multi-class fruit detection using a robotic vision system. Computer Networks 2020;168, 107036. https://doi.org/10.1016/j.comnet.2019.107036.
[18]	Tu S, Pang J, Liu H, Zhuang N, Chen Y, Zheng C, et al. Passion fruit detection and counting based on multiple scale faster R-CNN using RGB-D images. Precision Agriculture 2020;21(5), 1072-1091.
[19]	Jia W, Tian Y, Luo R, Zhang Z, Lian J, Zheng Y. Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot. Computers and Electronics in Agriculture 2020;172, 105380. https://doi.org/10.1016/j.compag.2020.105380.
[20]	Mirhaji H, Soleymani M, Asakereh A, Mehdizadeh S A. Fruit detection and load estimation of an orange orchard using the YOLO models through simple approaches in different imaging and illumination conditions.Computers and Electronics in Agriculture 2021;191, 106533. https://doi.org/10.1016/j.compag.2021.106533.
[21]	Huang G, Liu Z, Van Der Maaten L, Weinberger K Q. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition 2017; (pp. 4700-4708).
[22]	Howard A G, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 2017. https://doi.org/10.48550/arXiv.1704.04861.
[23]	Yu Y, Zhang K, Liu H, Yang L, Zhang D. Real-time visual localization of the picking points for a ridge-planting strawberry harvesting robot.IEEE Access 2020;8, 116556-116568.
[24]	Wang D, He D. Channel pruned YOLO V5s-based deep learning approach for rapid and accurate apple fruitlet detection before fruit thinning.Biosystems Engineering 2021;210, 271-281. https://doi.org/10.1016/j.biosystemseng.2021.08.015.
[25]	Ning Z, Luo L, Ding X, Dong Z, Yang B, Cai J, et al. Recognition of sweet peppers and planning the robotic picking sequence in high-density orchards. Computers and Electronics in Agriculture 2022;196, 106878. https://doi.org/10.1016/j.compag.2022.106878.
[26]	Zhang Y, Yu J, Chen Y, Yang W, Zhang W, He Y. Real-time strawberry detection using deep neural networks on embedded system (rtsd-net): An edge AI application. Computers and Electronics in Agriculture 2022;192, 106586. https://doi.org/10.1016/j.compag.2021.106586.
[27]	Wang J, Zhang Z, Luo L, Zhu W, Chen J, Wang W. SwinGD: A Robust Grape Bunch Detection Model Based on Swin Transformer in Complex Vineyard Environment. Horticulturae 2021;7(11), 492. https://doi.org/10.3390/horticulturae7110492.
[28]	Chollet F. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition 2017;(pp. 1251-1258).
[29]	Ge Z, Liu S, Wang F, Li Z, Sun J. YOLOX: Exceeding YOLO Series in 2021. arXiv preprint arXiv:2107.08430 2021. https://doi.org/10.48550/arXiv.2107.08430.
[30]	Wang C Y, Liao H Y M, Wu Y H, Chen P Y, Hsieh J W, Yeh I H. CSPNet: A new backbone that can enhance learning capability of CNN. InnullProceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops 2020;(pp. 390-391).
[31]	He K, Zhang X, Ren S, Sun J. Spatial pyramid pooling in deep convolutional networks for visual recognition.IEEE transactions on pattern analysis and machine intelligence 2015;37(9), 1904-1916.
[32]	He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition 2016;(pp. 770-778).
[33]	Wang Q, Wu B, Zhu P, Li P, Zuo W, Hu Q. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. arXiv preprint arXiv:1910.03151 2020. https://doi.org/10.48550/arXiv.1910.03151.
[34]	Hu J, Shen L, Albanie S, Sun G, Wu E. Squeeze-and-Excitation Networks. arXiv preprint arXiv:1709.01507 2019. https://doi.org/10.48550/arXiv.1709.01507.
[35]	Selvaraju R R, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-cam: Visual explanations from deep networks via gradient-based localization. InnullProceedings of the IEEE international conference on computer visionnull2017;(pp. 618-626).
[36]	Li H, Li C, Li G, et al. A real-time table grape detection method based on improved YOLOv4-tiny network in complex background. Biosystems Engineering, 2021, 212: 347-359.
[37]	Sozzi M, Cantalamessa S, Cogato A, et al. Automatic bunch detection in white grape varieties using YOLOv3, YOLOv4, and YOLOv5 deep learning algorithms. Agronomy, 2022, 12(2): 319.