Abstract:
High-precision segmentation of individual breeding plots is often required for high-throughput field phenotyping in precision agriculture. However, existing instance models are limited to boundary adhesion and blurred segmentation masks in large-scale fields using unmanned aerial vehicle (UAV) imagery. Particularly, wheat breeding trials can result in dense plot arrangements, similar canopy textures, and severe canopy interlacing during the mid-to-late growth stages. This study aims to propose a high-precision network for instance segmentation, named ST-PRNet. Mask R-CNN was utilized as the baseline architecture. Four technical improvements were integrated to enhance segmentation robustness under complex agricultural scenarios. 1) A Swin transformer backbone was employed to extract global semantic features using a shifted window-based multi-head self-attention (SW-MSA) mechanism. The complex global topological structure of breeding plots was more effectively captured than by conventional convolutional backbones. Semantic confusion was effectively suppressed from blurred canopy textures at various growth stages. 2) High-resolution geometric features were converted from low- to high-level semantic maps using a path aggregation feature pyramid network (PAFPN). Thereby, multi-scale feature representation was enhanced for the breeding plots at varying sizes. The loss of spatial information was reduced during downsampling. 3) A soft-nms (non-maximum suppression) post-processing was implemented to replace conventional hard-threshold nms. Gaussian decay function was also utilized to preserve overlapping with independent bounding boxes. The missed detections were effectively solved in dense, physically overlapped plot areas. 4) The PointRend module was integrated to replace the standard mask head. An iterative point-wise sampling and rendering strategy was employed on the boundary regions with high uncertainty. The jagged edge and over-smoothing were resolved in standard bilinear upsampling. Pixel-level boundary refinement was achieved to produce masks that closely match the non-regular contours of the wheat canopy, particularly with irregular plot boundaries. The framework was evaluated on the high-quality dataset with 966 plots from 107 wheat varieties. A total of 14028 images were partitioned into training, validation, and testing sets at a ratio of 8:1:1. Experimental results demonstrate that the ST-PRNet achieved an average precision (AP) of 92.8%, indicating an improvement of 8.6 percentage points over the Mask R-CNN baseline. Furthermore, ST-PRNet outperformed the state-of-the-art (SOTA) segmentation models, including Cascade Mask R-CNN, HTC, and the recently developed oriented instance segmentation network (OSNet). Ablation studies reveal that the PointRend module was introduced to provide the most significant performance gain. The AP improved by 6.8 percentage points than before. The complex serrated contours of wheat canopies were reconstructed after evaluation. Superior robustness of segmentation was achieved at different growth stages, particularly for severe boundary adhesion and segmentation overlap. ST-PRNet masks were also validated in crop phenotyping at the jointing stage. A case study was conducted to extract the fractional vegetation cover (FVC). High-fidelity masks were taken as the spatial references to isolate pure crop canopies. Non-crop background noise was eliminated, such as aisle pathways and soil. The extracted plot-level FVC values ranged from 41.6% to 84.0%, with a mean of 69.9%, objectively reflecting canopy growth heterogeneity across different varieties. Compared with conventional methods, the evaluation errors were significantly reduced under background contamination, indicating the visual representation of plot-level growth heterogeneity. ST-PRNet can also provide a robust and reliable algorithm for large-scale wheat phenotyping. More efficient data acquisition was facilitated for genetic trait evaluation and variety selection in intelligent breeding. ST-PRNet can effectively bridge the gap between raw UAV imagery and actionable phenotypic data. The findings can also offer high-precision end-to-end solutions to serve as a powerful tool for precision phenotyping in smart agriculture.