Abstract:
The synchronous detection of ripe grapes and their picking point is crucial for ensuring fruit quality and improving harvesting efficiency. To overcome the limitations of conventional asynchronous detection methods in grape detection, such as complex processes, low efficiency, high model redundancy, and high computational cost, this study proposed GrapickNet, a novel synchronous detection method for grape ripeness and picking point localizations. This method employs an improved Grapick-YOLO12n model to simultaneously detect both grape ripeness and key stem points, and then utilizes a bunch-pick point matching algorithm based on multiple constraint rules (RBPM) to effectively match ripe bunches with their corresponding pick points. First, Grapick-YOLO12n used YOLO12n-pose as the baseline model, with the innovative integration of the Edge Information Transfer (EIFT) module and the C3K2_RFAConv module. The EIFT module enhances the model's perception of object boundaries and local elongated small targets through shallow multi-scale edge feature extraction and cross-channel fusion mechanisms. The C3K2_RFAConv module expands the receptive field and strengthens global context modeling to improve the model's feature discrimination capability in complex scenes with multi-target interweaving. Second, the RBPM algorithm constructed multiple constraints to improve the successful matching rate between ripe bunches and its picking point. A comprehensive dataset comprising 4 383 RGB images of greenhouse grapes, covering two distinct environments, four different lighting conditions, and five varieties, was established and used as model training data. Experimental results shown that GrapickNet achieves 88.9% mAP@0.5 for ripeness detection and 82.7% precision for picking point detection. Compared with the baseline model (87.0% mAP@0.5 for ripeness and 77.8% precision for picking points), the proposed method improves by 1.9% and 4.9%, respectively. Furthermore, the proposed model has a computational complexity of 8.4 GFLOPs and a parameter size of merely 3.33 M, while attaining an inference speed of 94.6 FPS. In terms of positioning accuracy, the average errors in the
X and
Y directions are 2.51 pixels and 5.37 pixels, respectively. The average error in Euclidean distance is 6.39 pixels. The performance of object detection is outperformed the existing models, including YOLOv5n-pose, YOLOv8n-pose, YOLOv10n-pose, YOLO11n-pose, YOLO12n-pose, Faster R-CNN, SSD, and RT-DETR. Compared with asynchronous separate detection tasks, GrapickNet, which possesses multi-task synchronous detection capability, achieves synchronous, accurate, and efficient detection of grape ripeness and picking points for different varieties in complex greenhouse scenarios. This method maintains good detection accuracy and short detection time while reducing parameter usage, offering greater advantages in computational efficiency and deployment on mobile devices. This study has good practical value, providing an efficient and accurate technical solution for the automated grape harvesting in semi-structured greenhouses and promoting the intelligent development of grape harvesting robots.