Abstract:
Leaf occlusion and fruit overlap complicate the classification of strawberries and the extraction of fruit orientation from field images. For grippers that directly grasp the fruit, the main axis provides a geometric reference for adjusting the grasping angle, but incomplete or displaced mask boundaries can affect its estimation. This study developed a method combining an improved You Only Look Once version 8 small model (YOLOv8s) with pixel traversal for strawberry instance segmentation and two-dimensional main-axis estimation. Images of 'Suizhu' strawberries were collected in Tianjin, China, using a RealSense D435i camera under different illumination conditions and viewing angles. After unusable images were excluded, 1,109 images were retained and divided into training, validation, and test sets at a ratio of 8:1:1. Four categories were annotated: ripe strawberries, unripe strawberries, ripe strawberries occluded by leaves, and ripe strawberries occluded by unripe fruits. These categories distinguished fruit maturity and the source of occlusion within the same segmentation task. Three modifications were introduced into YOLOv8s. The Wise-Inner-SIoU bounding-box regression loss combined geometric constraints, intersection over union (IoU) calculated using auxiliary boxes, and dynamic nonmonotonic focusing to adjust overlap measurement and sample weighting during training. The C2f-faster-EMA module combined partial and pointwise convolutions with efficient multi-scale attention (EMA) to aggregate local and spatial features. The ADown module used pooling and convolution branches for feature downsampling. Training used 150 epochs, a batch size of eight, and an input image resolution of 640 by 480 pixels. Module ablation and auxiliary box scale comparisons were conducted to select the final configuration, with the scale factor set to 0.9. The model jointly predicted fruit categories and individual masks. For each ripe fruit, including those in the two occlusion categories, all pairs of mask boundary pixels were evaluated, and the line joining the two pixels with the greatest Euclidean distance was taken as the estimated main axis. This geometric step did not require training a separate network for orientation prediction. The axis represented an orientation in the image plane without distinguishing its two ends. Segmentation performance was compared with Mask Region-based Convolutional Neural Network (Mask R-CNN), U-Net, YOLOv5s, the original YOLOv8s, YOLOv9, and YOLO11s-seg. The final model achieved a mask mean average precision of 95.4% at an IoU threshold of 0.5, exceeding the original YOLOv8s by 2.7 percentage points. The six comparison models had mask mean average precision values ranging from 64.6% to 93.8%. Average precision for ripe strawberries occluded by leaves reached 95.3%, an increase of 7.8 percentage points. For ripe strawberries occluded by unripe fruits, average precision was 92.4%, below the 93.1% of YOLOv9 and 94.1% of YOLO11s-seg. The model contained 10.9 million parameters and achieved an inference speed of 159.3 frames per second under the reported experimental conditions; this speed did not include subsequent main-axis computation. Main-axis estimation was evaluated using 101 ripe fruits with valid manually annotated reference axes from 60 of 62 selected validation and test images. Angular error was defined as the smaller angle between the estimated and reference axes. Of these fruits, 81 had angular errors below 15 degrees, corresponding to 80.20% of the evaluated samples. The mean angular error was 10.27 degrees, with a standard deviation of 13.52 degrees. Stratified analysis showed higher mean errors for small fruits, moderately occluded fruits, and fruits in the 30–60-degree growth-angle group than for the other groups within each respective stratification. These findings identify conditions requiring further refinement of contour-based axis estimation. The method provides category information and an image-plane orientation reference for strawberry harvesting, while its evaluation remains limited to segmentation and two-dimensional axis estimation. Future work will incorporate depth information, hand-eye calibration, and contour fitting, together with harvesting trials using an SFG-FNC3 flexible three-finger gripper to assess the relationship between axis error, grasping stability, and harvesting success.