高级检索+

基于AdaPoinTr-ER和递进补全策略的缺失生菜整体补全

Overall completion of incomplete lettuce using AdaPoinTr-ER model and progressive completion

  • 摘要: 为实现植物工厂生菜生产中温度、湿度、光照及营养液的精准调控,解决种植中后期生菜重叠遮挡导致点云缺失使表型参数获取不准的问题,该研究针对生菜缺失点云的整株补全提出一种基于AdaPoinTr-ER模型的递进补全策略。在AdaPoinTr基模型的几何感知Transformer模块中引入边缘注意力机制(edge aware attention,EAA),并在重建头中结合残差模块(residual block,RB),改进设计出AdaPoinTr-ER补全模型。针对生菜分割后的缺失点云,构建基于AdaPoinTr-ER训练好的俯视补全模型和三维补全模型,进行从俯视到三维的递进补全。模型试验结果表明,改进后的AdaPoinTr-ER模型倒角距离降至0.288×10−3 cm,地球移动距离降至0.19 cm,F1分数达到72.68%。表型参数回归结果表明,该方法补全后生菜投影面积、冠幅、株高的决定系数R2分别为0.933、0.917、0.903,均方根误差分别为8.569 cm2、0.434 cm、0.591 cm,对比补全前R2均提升30%以上。该研究方法可有效补全遮挡生菜点云,大幅提升了生菜遮挡条件下表型参数的提取精度,为植物工厂叶菜的智能、精准管控提供了技术支持。

     

    Abstract: Lettuce is one of the most favorite leafy vegetable in precision cultivation. It is often required to monitor the lettuce growth for real-time robotic harvesting. Nevertheless, leafy vegetables are characterized by leaves, diverse morphologies, and thin, flexible, and deformable textures. There is a more complex structure, compared with the morphologically regular objects, such as spherical fruits, umbrella-shaped mushrooms, and conical carrots. As such, plant growth and leaf expansion can lead to mutual occlusion among lettuce plants. Furthermore, the high planting density has commonly adopted in plants, leading to the inter-leaf occlusion. Additionally, height differences between individual plants can also cause upper leaves to shade lower ones. These occlusions then result in missing data in the point cloud images, seriously affecting the accurate acquisition of key phenotypic parameters. Conventional point cloud processing has mostly developed for regularly shaped crops, making it difficult to effectively reconstruct complex structures. Therefore, it is challenging to accurately detect the complete three-dimensional phenotypic parameters of lettuce from severely incomplete point cloud data. In this study, the AdaPoinTr-ER model and progressive completion were proposed for the overall completion pipeline of incomplete lettuce images in point cloud. Since the occlusion among lettuce plants occurred at edge positions, AdaPoinTr-ER model was integrated edge attention into the AdaPoinTr framework to enhance feature extraction of geometric contour. In view of the lettuce leaves with the more complex multilayer structures, AdaPoinTr-ER model was integrated residual module into AdaPoinTr to reduce feature degradation during point cloud generation for the prediction accuracy of generated point clouds. Three sequential stages were proposed to realize the progressive completion for incomplete lettuce: (1) Mask3D was employed to segment top-view lettuce clusters with mutual occlusion, thus capturing the top-view images of incomplete lettuce plants. (2) The top-down completion model trained by AdaPoinTr-ER model was used to complete the image, and the resulting images were then input into the three-dimensional completion model for training. (3) The phenotypic parameters of the completed intact lettuce were acquired after three-dimensional completion. The experimental results demonstrated that AdaPoinTr-ER model achieved the best performance in Chamfer Distance, Earth Mover's Distance, and F1-score, compared with FoldingNet, GRNet, PCN, PoinTr, and AdaPoinTr. The ablation experiments demonstrated that the AdaPoinTr-ER model achieved a Chamfer Distance of 0.288×10-3 cm, an Earth Mover's Distance of 0.19 cm, and an F1-score of 72.68%. Compared with the original AdaPoinTr model, the Chamfer Distance and Earth Mover's Distance decreased by 43.3% and 42.4%, respectively, while the F1-score improved by 9.52 percentage points. In the lettuce phenotypic analysis, the progressive completion yielded coefficients of determination (R2) of 0.933, 0.917, and 0.903 for the projected area, crown width, and plant height, respectively, with the root mean square errors (RMSE) of 8.569 cm2, 0.434 cm, and 0.591 cm, respectively. Compared with the phenotypic analysis from incomplete lettuce, the R² values increased by 45.5%, 61.4%, and 30.7%, respectively. Furthermore, the R2 values improved by 24.9%, 29.5%, and 11.1%, respectively, whereas, the RMSE was reduced by 64.8%, 56.6%, and 37.4%, respectively, compared with direct completion using only 3D reconstruction without top-view completion. Consequently, AdaPoinTr-ER model exhibited superior performance to restore both the local geometric details and the overall shape structure of lettuce point clouds. Completing occluded lettuce point clouds is a challenging task. The occluded lettuce point clouds were accurately and effectively reconstructed to significantly improve the accuracy of phenotypic parameter extraction for leafy vegetables in densely planted environments. Thereby, the finding can provide the robust support for the intelligent and precise completion of leafy vegetable species. The completion pipeline can also be integrated with real-time robotic harvesting for fully automatic operations in plants.

     

/

返回文章
返回