Abstract:
Automated and high-precision extraction of individual breeding plots from large-scale unmanned aerial vehicle (UAV) imagery is a critical prerequisite for high-throughput field phenotyping and precision agriculture. In large-scale wheat breeding field trials, the challenges of dense plot arrangement, similar canopy textures, and severe canopy interlacing during the mid-to-late growth stages often result in significant boundary adhesion and blurred segmentation masks, posing substantial challenges to existing instance segmentation models. To address these limitations, this study proposes a high-precision instance segmentation network named ST-PRNet. By utilizing Mask R-CNN as the baseline architecture, the proposed method integrates four core technical improvements to enhance segmentation robustness across complex agricultural scenarios. First, a swin transformer backbone is employed to extract global semantic features through the shifted window-based multi-head self-attention (SW-MSA) mechanism, enabling the model to capture the complex global topological structure of breeding plots more effectively than traditional convolutional backbones. This architectural choice effectively suppresses the semantic confusion caused by non-distinct canopy textures at different growth stages. Second, a path aggregation feature pyramid network (PAFPN) is constructed to reinforce the transmission of high-resolution geometric details from low-level features to high-level semantic maps, thereby enhancing multi-scale feature representation for breeding plots of varying sizes and reducing the spatial information loss during downsampling. Third, a soft-nms (non maximum suppression) post-processing strategy is implemented to replace traditional hard-threshold nms. This mechanism utilizes a Gaussian decay function to preserve overlapping but independent boundary boxes, effectively solving the problem of missed detections in dense, physically overlapped plot areas. Finally, the PointRend module is integrated to replace the standard mask head, employing an iterative point-wise sampling and rendering strategy on boundary regions with high uncertainty. This mechanism resolves the jagged edge and over-smoothing issues inherent in standard bilinear upsampling, achieving pixel-level boundary refinement and producing masks that align closely with the intricate, non-regular contours of the wheat canopy, which is particularly vital when dealing with irregular plot boundaries. The proposed framework was evaluated on a high-quality dataset containing 966 plots from 107 wheat varieties, with a total of 14,028 images strictly partitioned into training, validation, and testing sets at an 8:1:1 ratio. Experimental results demonstrate ST-PRNet achieves a AP of 92.8%, representing an improvement of 8.6 percentage points over the Mask R-CNN baseline. Furthermore, ST-PRNet outperforms other state-of-the-art (SOTA) segmentation models, including Cascade Mask R-CNN, HTC, and the recently developed oriented instance segmentation network OSNet. Ablation studies reveal that introducing the PointRend module provides the most significant performance gain, further improving the AP by 6.8 percentage points compared with the model before its introduction, highlighting its critical role in reconstructing the complex, serrated contours of wheat canopies. The model exhibits superior robustness across different growth stages, particularly in handling severe boundary adhesion and segmentation overlap. Beyond its superior segmentation performance, this study also validated the practical utility of ST-PRNet masks in automated crop phenotyping, using the jointing stage as a case study for automated fractional vegetation cover (FVC) extraction. By utilizing high-fidelity masks as spatial references to isolate pure crop canopies and eliminate non-crop background noise—such as aisle pathways and soil—the framework achieved more objective and accurate phenotypic trait quantification. Compared to traditional methods, the proposed approach significantly reduced the evaluation errors caused by background contamination, providing a clear visual representation of plot-level growth heterogeneity. These results confirm that ST-PRNet provides a robust and reliable algorithmic foundation for automated, large-scale wheat phenotyping, facilitating more efficient data acquisition for genetic trait evaluation and variety selection in intelligent breeding research. By offering a high-precision, end-to-end solution, ST-PRNet effectively bridges the gap between raw UAV imagery and actionable phenotypic data, serving as a powerful tool for the future of smart agriculture and precision phenotyping.