Abstract:
Lettuce is one of the most favorite leafy vegetable in precision cultivation. It is often required to monitor the lettuce growth for real-time robotic harvesting. Nevertheless, leafy vegetables are characterized by leaves, diverse morphologies, and thin, flexible, and deformable textures. There is a more complex structure, compared with the morphologically regular objects, such as spherical fruits, umbrella-shaped mushrooms, and conical carrots. As such, plant growth and leaf expansion can lead to mutual occlusion among lettuce plants. Furthermore, the high planting density has commonly adopted in plants, leading to the inter-leaf occlusion. Additionally, height differences between individual plants can also cause upper leaves to shade lower ones. These occlusions then result in missing data in the point cloud images, seriously affecting the accurate acquisition of key phenotypic parameters. Conventional point cloud processing has mostly developed for regularly shaped crops, making it difficult to effectively reconstruct complex structures. Therefore, it is challenging to accurately detect the complete three-dimensional phenotypic parameters of lettuce from severely incomplete point cloud data. In this study, the AdaPoinTr-ER model and progressive completion were proposed for the overall completion pipeline of incomplete lettuce images in point cloud. Since the occlusion among lettuce plants occurred at edge positions, AdaPoinTr-ER model was integrated edge attention into the AdaPoinTr framework to enhance feature extraction of geometric contour. In view of the lettuce leaves with the more complex multilayer structures, AdaPoinTr-ER model was integrated residual module into AdaPoinTr to reduce feature degradation during point cloud generation for the prediction accuracy of generated point clouds. Three sequential stages were proposed to realize the progressive completion for incomplete lettuce: (1) Mask3D was employed to segment top-view lettuce clusters with mutual occlusion, thus capturing the top-view images of incomplete lettuce plants. (2) The top-down completion model trained by AdaPoinTr-ER model was used to complete the image, and the resulting images were then input into the three-dimensional completion model for training. (3) The phenotypic parameters of the completed intact lettuce were acquired after three-dimensional completion. The experimental results demonstrated that AdaPoinTr-ER model achieved the best performance in Chamfer Distance, Earth Mover's Distance, and F1-score, compared with FoldingNet, GRNet, PCN, PoinTr, and AdaPoinTr. The ablation experiments demonstrated that the AdaPoinTr-ER model achieved a Chamfer Distance of 0.288×10
-3 cm, an Earth Mover's Distance of 0.19 cm, and an F1-score of 72.68%. Compared with the original AdaPoinTr model, the Chamfer Distance and Earth Mover's Distance decreased by 43.3% and 42.4%, respectively, while the F1-score improved by 9.52 percentage points. In the lettuce phenotypic analysis, the progressive completion yielded coefficients of determination (
R2) of 0.933, 0.917, and 0.903 for the projected area, crown width, and plant height, respectively, with the root mean square errors (RMSE) of 8.569 cm
2, 0.434 cm, and 0.591 cm, respectively. Compared with the phenotypic analysis from incomplete lettuce, the R² values increased by 45.5%, 61.4%, and 30.7%, respectively. Furthermore, the
R2 values improved by 24.9%, 29.5%, and 11.1%, respectively, whereas, the RMSE was reduced by 64.8%, 56.6%, and 37.4%, respectively, compared with direct completion using only 3D reconstruction without top-view completion. Consequently, AdaPoinTr-ER model exhibited superior performance to restore both the local geometric details and the overall shape structure of lettuce point clouds. Completing occluded lettuce point clouds is a challenging task. The occluded lettuce point clouds were accurately and effectively reconstructed to significantly improve the accuracy of phenotypic parameter extraction for leafy vegetables in densely planted environments. Thereby, the finding can provide the robust support for the intelligent and precise completion of leafy vegetable species. The completion pipeline can also be integrated with real-time robotic harvesting for fully automatic operations in plants.