Abstract:
Accurate and automated acquisition of plant phenotypic parameters is important for quantitative analysis of crop growth status and structural characteristics. However, traditional manual measurements are labor-intensive and subjective, while two-dimensional imaging methods are limited in describing complex three-dimensional plant architectures. Therefore, this study developed an automated three-dimensional phenotyping framework for tomato plants integrating multi-view image acquisition, three-dimensional reconstruction, point cloud semantic segmentation, leaf instance segmentation, and geometric phenotypic parameter extraction. A multi-view RGB image acquisition platform was established to collect tomato plant images under greenhouse and field conditions. After image preprocessing and camera pose estimation, neural radiance fields (NeRF) were employed to reconstruct individual tomato plants and generate corresponding point clouds. A self-constructed dataset containing 70 tomato plant point clouds from two cultivation environments and three cultivars was established, with each point manually annotated as leaf, stem, or other. An improved RandLA-Net model was developed by introducing a bilateral enhancement module (BEM), a multi-scale attention mechanism (MSAM), and a combined loss function integrating focal loss, Dice loss, and boundary loss. The BEM enhanced interactions between local geometric information and semantic features to improve the representation of fine structures and stem–leaf boundary regions. The MSAM incorporated channel attention, spatial attention, and multi-scale feature extraction to adaptively fuse local and global features and suppress interference from occlusion and background noise. The combined loss function further addressed class imbalance, region overlap, and boundary constraints. Experimental results showed that the improved model achieved mIoU, mPrecision, mRecall, and mF1-score values of 87.92%, 88.67%, 88.18%, and 88.42%, respectively, representing improvements of 3.70, 3.60, 3.55, and 3.57 percentage points over the original RandLA-Net. Independent validation on the public Pheno4D tomato dataset yielded an mIoU of 86.41%, demonstrating the applicability of the improved model to an independent tomato point cloud dataset. Following semantic segmentation, DBSCAN was used to separate individual leaf point clouds into leaf instances. An automated geometric measurement method was developed to extract plant height, leaf length, leaf width, leaf area, and leaf–stem angle. A local coordinate system was established for each leaf point cloud according to the leaf midrib direction and its orthogonal directions, and the maximum projection spans along the corresponding axes were calculated as leaf length and width. For leaf area estimation, the leaf point cloud was projected onto three mutually orthogonal coordinate planes, and the arithmetic mean of the three projected convex hull areas was used as the estimated leaf area. The main stem axis was obtained by cylindrical fitting using RANSAC, while the leaf midrib direction was determined from the leaf longitudinal axis. The extracted phenotypic parameters showed strong consistency with manual measurements, with coefficients of determination (
R²) greater than 0.95. For leaf area estimation, the proposed tri-orthogonal projection convex hull method achieved an
R² of 0.9628 and an RMSE of 1.18 cm² compared with the mesh triangular patch-based method. The results demonstrate that the proposed framework can integrate three-dimensional reconstruction, point cloud deep learning, and geometric analysis for automated tomato phenotyping, providing a technical basis for three-dimensional structural characterization and quantitative phenotypic analysis.