高级检索+

基于改进YOLOv8和像素遍历法的草莓姿态估计

Strawberry pose estimation using improved YOLOv8 and pixel traversal method

  • 摘要: 针对田间环境中叶片遮挡、果实重叠及轮廓缺失导致的草莓类型识别与成熟果主轴估计不稳定问题,该研究提出一种基于改进YOLOv8s和像素遍历法的草莓姿态估计方法。采集并筛选田间草莓图像,将草莓目标按成熟草莓、未成熟草莓、叶片遮挡成熟草莓和未成熟果遮挡成熟草莓4类进行标注,构建包含1109张图像的数据集。在YOLOv8s中引入边界框回归损失(wise-inner-scylla intersection over union,Wise-Inner-SIoU)、轻量化注意力模块(C2f-faster-efficient multi-scale attention,C2f-faster-EMA)和下采样结构(ADown)改进模型,实现草莓语义分割,并依据成熟草莓掩膜外轮廓的最长连线估计主轴方向。结果表明,改进模型在交并比阈值为0.5时,平均精度均值mAP@0.5达95.4%,较原始YOLOv8s提升2.7个百分点;在101个成熟草莓主轴验证样本中,以角度误差小于15°为正确估计标准,姿态估计准确率为80.20%,平均角度误差为10.27°。该方法可为复杂场景下草莓遮挡工况判断及机械爪抓取角度调整提供视觉依据。

     

    Abstract: Leaf occlusion and fruit overlap complicate the classification of strawberries and the extraction of fruit orientation from field images. For grippers that directly grasp the fruit, the main axis provides a geometric reference for adjusting the grasping angle, but incomplete or displaced mask boundaries can affect its estimation. This study developed a method combining an improved You Only Look Once version 8 small model (YOLOv8s) with pixel traversal for strawberry instance segmentation and two-dimensional main-axis estimation. Images of 'Suizhu' strawberries were collected in Tianjin, China, using a RealSense D435i camera under different illumination conditions and viewing angles. After unusable images were excluded, 1,109 images were retained and divided into training, validation, and test sets at a ratio of 8:1:1. Four categories were annotated: ripe strawberries, unripe strawberries, ripe strawberries occluded by leaves, and ripe strawberries occluded by unripe fruits. These categories distinguished fruit maturity and the source of occlusion within the same segmentation task. Three modifications were introduced into YOLOv8s. The Wise-Inner-SIoU bounding-box regression loss combined geometric constraints, intersection over union (IoU) calculated using auxiliary boxes, and dynamic nonmonotonic focusing to adjust overlap measurement and sample weighting during training. The C2f-faster-EMA module combined partial and pointwise convolutions with efficient multi-scale attention (EMA) to aggregate local and spatial features. The ADown module used pooling and convolution branches for feature downsampling. Training used 150 epochs, a batch size of eight, and an input image resolution of 640 by 480 pixels. Module ablation and auxiliary box scale comparisons were conducted to select the final configuration, with the scale factor set to 0.9. The model jointly predicted fruit categories and individual masks. For each ripe fruit, including those in the two occlusion categories, all pairs of mask boundary pixels were evaluated, and the line joining the two pixels with the greatest Euclidean distance was taken as the estimated main axis. This geometric step did not require training a separate network for orientation prediction. The axis represented an orientation in the image plane without distinguishing its two ends. Segmentation performance was compared with Mask Region-based Convolutional Neural Network (Mask R-CNN), U-Net, YOLOv5s, the original YOLOv8s, YOLOv9, and YOLO11s-seg. The final model achieved a mask mean average precision of 95.4% at an IoU threshold of 0.5, exceeding the original YOLOv8s by 2.7 percentage points. The six comparison models had mask mean average precision values ranging from 64.6% to 93.8%. Average precision for ripe strawberries occluded by leaves reached 95.3%, an increase of 7.8 percentage points. For ripe strawberries occluded by unripe fruits, average precision was 92.4%, below the 93.1% of YOLOv9 and 94.1% of YOLO11s-seg. The model contained 10.9 million parameters and achieved an inference speed of 159.3 frames per second under the reported experimental conditions; this speed did not include subsequent main-axis computation. Main-axis estimation was evaluated using 101 ripe fruits with valid manually annotated reference axes from 60 of 62 selected validation and test images. Angular error was defined as the smaller angle between the estimated and reference axes. Of these fruits, 81 had angular errors below 15 degrees, corresponding to 80.20% of the evaluated samples. The mean angular error was 10.27 degrees, with a standard deviation of 13.52 degrees. Stratified analysis showed higher mean errors for small fruits, moderately occluded fruits, and fruits in the 30–60-degree growth-angle group than for the other groups within each respective stratification. These findings identify conditions requiring further refinement of contour-based axis estimation. The method provides category information and an image-plane orientation reference for strawberry harvesting, while its evaluation remains limited to segmentation and two-dimensional axis estimation. Future work will incorporate depth information, hand-eye calibration, and contour fitting, together with harvesting trials using an SFG-FNC3 flexible three-finger gripper to assess the relationship between axis error, grasping stability, and harvesting success.

     

/

返回文章
返回