Abstract: Precise monitoring of early-stage crop phenotypic parameters is essential for intelligent agricultural management. For economically significant crops like citrus, acquiring three-dimensional size parameters of immature green fruit is critical for yield prediction, water and fertilizer management, and harvest scheduling. Traditional manual measurement methods are labor-intensive and subjective, while existing vision techniques suffer from insufficient accuracy under complex orchard conditions due to weak textures, severe occlusions, and variable illumination. This study aimed to develop an automated method for accurately measuring the major and minor axes of immature green citrus in complex orchard environments. The proposed approach integrated instance segmentation with multi-stage point cloud matching. A high-performance segmentation model, CSF-YOLO, was developed by enhancing the YOLO11s-seg baseline with three novel components: a Cross-Stage Spatial-Frequency Adaptive Module for dual-domain feature extraction, a Pinwheel-shaped Convolution for expanded receptive fields, and an improved dynamic upsampler for semantic preservation. For measurement, point clouds were generated from segmentation masks and depth data using calibrated camera parameters, followed by LOF-based denoising. A multi-stage strategy was then employed: initial sphere fitting for template filtering, FPFH descriptor extraction, RANSAC-based correspondence establishment, and TEASER++ registration between the target and the best-fitting template. The experimental results demonstrate that, compared with the baseline model, CSF-YOLO achieves improvements in AP@50 for both object detection and instance segmentation from 87.31% and 86.92%, respectively, to 91.00% and 90.52%; similarly, AP@50–95 increases from 73.65% and 62.82% to 76.79% and 65.00%, respectively. Compared with current mainstream models, CSF-YOLO exhibits outstanding overall performance in object detection and instance segmentation while maintaining high efficiency. Ablation studies confirmed that all three proposed modules contributed complementary improvements, with the combined model achieving the best overall performance. Visualization results, including both segmentation comparisons and Grad-CAM++ heatmaps, further demonstrated that CSF-YOLO exhibited superior feature extraction and localization capabilities, accurately focusing on target instances under varying illumination conditions, small target sizes, and severe occlusions where other models showed significant missed detections. In the size measurement task, the proposed multi-stage matching strategy effectively overcame the inherent limitations of traditional geometric fitting methods. Compared to Maximum Inter-point Distance, Sphere Fitting, and Ellipsoid Fitting approaches, the proposed method achieved the best overall performance. The Root Mean Square Error for the measured major and minor axes was 3.96 mm and 3.10 mm, respectively, with corresponding Mean Absolute Percentage Errors of 5.52% and 5.41%. Residual analysis confirmed that the proposed method exhibited superior convergence characteristics and better adaptation to the local morphological features of citrus fruits, effectively addressing the challenges of incomplete point cloud data and asymmetric fruit morphology. The integrated method provides an accurate and automated solution for citrus fruit sizing in the field. By synergistically combining the enhanced segmentation model with the multi-stage matching strategy, the approach effectively addresses the two major challenges of weak-feature target segmentation and asymmetric fruit size estimation under incomplete point cloud conditions. The high performance achieved validates its capability to meet the stringent requirements of modern precision agriculture for both accuracy and automation. This work offers a reliable technical foundation for intelligent orchard management and provides a practical reference for advancing automated crop phenotyping in other fruit crops facing similar challenges.