高级检索+

基于Kinect V3深度传感器的田间植株点云配准方法

李修华, 魏鹏, 何嘉西, 李民赞, 张木清, 温标堂

李修华, 魏鹏, 何嘉西, 李民赞, 张木清, 温标堂. 基于Kinect V3深度传感器的田间植株点云配准方法[J]. 农业工程学报, 2021, 37(21): 45-52. DOI: 10.11975/j.issn.1002-6819.2021.21.006
引用本文: 李修华, 魏鹏, 何嘉西, 李民赞, 张木清, 温标堂. 基于Kinect V3深度传感器的田间植株点云配准方法[J]. 农业工程学报, 2021, 37(21): 45-52. DOI: 10.11975/j.issn.1002-6819.2021.21.006
Li Xiuhua, Wei Peng, He Jiaxi, Li Minzan, Zhang Muqing, Wen Biaotang. Field plant point cloud registration method based on Kinect V3 depth sensors[J]. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37(21): 45-52. DOI: 10.11975/j.issn.1002-6819.2021.21.006
Citation: Li Xiuhua, Wei Peng, He Jiaxi, Li Minzan, Zhang Muqing, Wen Biaotang. Field plant point cloud registration method based on Kinect V3 depth sensors[J]. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37(21): 45-52. DOI: 10.11975/j.issn.1002-6819.2021.21.006

基于Kinect V3深度传感器的田间植株点云配准方法

基金项目: 广西科技重大专项(桂科AA18118037、桂科2018-266-Z01);国家自然科学基金项目(31760342)

Field plant point cloud registration method based on Kinect V3 depth sensors

  • 摘要: 准确建立植物的三维点云是以点云方式高通量获取植株各部位物理参数的前提。为实现田间复杂环境下的植株三维点云配准,该研究提出了一种基于多标定球的田间植株点云自动配准方法,并分别在室内简单场景及大田复杂场景下从不同角度对多种作物采集的点云数据进行验证。该方法采用随机抽样一致性算法(Random Sample Consensus, RANSAC)结合点云减法的概念从下采样后的点云中实现多标定球的自动提取,弥补了RANSAC一次只能提取单个物体的缺点。然后基于各标定球的球心距离信息实现三维点集的自动匹配。最后使用奇异值分解算法解算旋转平移矩阵,实现点云的自动配准。不同场景下各作物的配准结果表明,各植株的水平90°、180°、270°以及垂直方向上的点云配准到水平0°点云下的平均轴向误差在5.8~17.4 mm之间,平均点位误差在13.1~28.9 mm之间,与手动配准的商用同类软件LiDAR360的配准结果相当,但配准过程的自动化程度明显提高,效率提高了67%。该文所提出的方法可在田间复杂环境下对低成本深度相机获取的植株点云实现高精度的自动配准,为田间植物表型参数的提取提供了低成本的可行方案。
    Abstract: Abstract: An accurate modeling of three-dimensional (3D) point cloud has been a crucial step to extract the physical phenotyping parameters of plants, such as the plant height, leaf quantity, and leaf area. In this study, an automatic registration of point clouds with multiple calibration balls was proposed to realize the 3D modeling of plants under a complex background in the field. A low-cost depth sensor (such as Kinect V3) was also selected to capture the images. The performance of the registration was then evaluated using the multiple angles of several plants under the indoor and in-field scenes. Four procedures were included the point cloud filtering and down sampling, multiple calibration balls extraction, correspondence points matching, as well as the calculation and registration of transformation matrix. 1) The pass-through filtering with only several boundary thresholds was used to reduce the noises, while the bounding box compression was used to down sampling the point clouds. 2) A Random Sampling Consistency (RANSAC) was also used in the multiple calibration balls extraction. Furthermore, a concept of point cloud subtraction was proposed to combine with the RANSAC, in order to form an automatic extraction of multiple calibration balls. Among them, the RANSAC was utilized to extract one single object at one time. As such, the center coordinates of each calibration ball were then calculated as the featured points. 3) The distances of any two feature points were calculated in the correspondence points matching. An automatic matching of correspondence points using the distance information was also proposed to realize the self-matching of the calibration balls. 4) The singular value decomposition was adopted to solve the transformation (rotation and translation) matrix, and then the registration of two pieces of point clouds was realized eventually. The experiments were carried out in two scenes: one was the indoor scene with the flat and clean surface, and another was the field scene with the uneven and complex conditions. Five plants were chosen as the research objects, including one sugarcane (indoor), one sorghum (indoor), and three young banana plants (in-field). The Kinect V3 sensor was utilized to capture the point clouds of each object in five orientations (the horizontal orientation of 0?, 90?, 180?, 270?, and one vertical orientation). A commercial point cloud processing software named LiDAR360 was adopted for comparison with the man-aid registration. After that, the object point clouds (horizontal 90°, 180°, 270°, and vertical) of each plant was used to implement the registration into the coordinate system of the source point cloud (horizontal 0°). The registration performance was evaluated to calculate the axial errors and point errors of the transformed coordinates, as well as the source coordinates of the centers of all the calibration balls. The results showed that the registration of different point clouds was successfully implemented in the different orientations, no matter in the indoor or in-field environment. Particularly, the generated 3D model of the plant was clear in shape. Specifically, there were an average axial error of 5.8-17.4 mm and an average point position error of 13.1-28.9 mm for the registration in the different scenes, similar to the registration in the manual registration software LiDAR360. In the required calculation time, the automatic matching took about 50 s on average for the process from the extraction of calibration sphere to the registration, whereas, the LiDAR360 took about 150 s with the manual selection of the correspondence points, indicating the efficiency increased by 67%. Consequently, it can be widely expected to serve a high-precision and automatic registration of plant point clouds acquired by low-cost depth cameras in complex field environments. The finding can provide a low-cost feasible solution for the 3D modeling and the extraction of phenotyping parameters of field plants.
  • [1] 张彦雯,胡凯,王鹏盛. 三维重建算法研究综述[J]. 南京信息工程大学学报:自然科学版,2020,12(5):591-602.Zhang Yanwen, Hu Kai, Wang Pengsheng. Review of 3D reconstruction algorithms[J]. Journal of Nanjing University of Information Science & Technology: Natural Science, 2020, 12(5): 591-602. (in Chinese with English abstract)
    [2] 唐丽玉,张浩,黄洪宇,等. 基于点云数据的树木三维重建方法改进[J]. 农业机械学报,2017,48(2):186-194.Tang Liyu, Zhang Hao, Huang Hongyu, et al. Improved method for 3D reconstruction of tree model based on point cloud data[J]. Transactions of the Chinese Society for Agricultural Machinery, 2017, 48(2): 186-194. (in Chinese with English abstract)
    [3] 刘振,杨玮,李民赞,等. 基于Kinect相机的土壤表面三维点云配准方法[J]. 农业机械学报,2019,50(S1):144-149.Liu Zhen, Yang Wei, Li Minzan, et al. Three-dimensional point cloud registration method for soil surface based on Kinect camera[J]. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50(S1): 144-149. (in Chinese with English abstract)
    [4] Johnson A E, Hebert M. Using spin images for efficient object recognition in cluttered 3D scenes[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2002, 21(5): 433-449.
    [5] Frome A, Huber D, Kolluri R, et al. Recognizing objects in range data using regional point descriptors[C]// European Conference on Computer Vision. Springer, Berlin, Heidelberg, 2004.
    [6] Salti S, Tombari F, Stefano L D. SHOT: Unique signatures of histograms for surface and texture description[J]. Computer Vision & Image Understanding, 2014, 125: 251-264.
    [7] Guo Y, Ferdous S, Bennamoun M, et al. Rotational projection statistics for 3D local surface description and object recognition[J]. International Journal of Computer Vision, 2013, 105(1): 63-86.
    [8] Rusu R B, Blodow N, Beetz M. Fast point feature histograms (FPFH) for 3D registration[C]// IEEE International Conference on Robotics & Automation. IEEE, 2009.
    [9] 杨玉泽,孙英伟,林文树. 基于FPFH特征和NDT算法 的树木点云配准[J]. 西北林学院学报,2019,34(5):141-146. Yang Yuze, Sun Yingwei, Lin Wenshu. Tree point cloud registration based on FPFH feature and NDT algorithm[J]. Journal of Northwest Forestry University, 2019, 34(5): 141-146. (in Chinese with English abstract)
    [10] 郑立华,麦春艳,廖崴,等. 基于Kinect相机的苹果树三维点云配准[J]. 农业机械学报,2016,47(5):9-14.Zheng Lihua, Mai Chunyan, Liao Wei, et al. 3D point cloud registration for apple tree base on Kinect camera[J]. Transactions of the Chinese Society for Agricultural Machinery, 2016, 47(5): 9-14. (in Chinese with English abstract)
    [11] Garrido M, Paraforos D, Reiser D, et al. 3D maize plant reconstruction based on georeferenced overlapping LiDAR point clouds[J]. Remote Sensing, 2015, 7(12): 17077-17096.
    [12] Chebrolu N, Magistri F, L?be T, et al. Registration of spatio-temporal point clouds of plants for phenotyping[J]. PLoS One, 2021, 16(2): e0247243.
    [13] Yu X, Song H, Wang S. Capture of multi-view point clouds and registration of plants based on Kinect[J]. International Agricultural Engineering Journal, 2016, 25(1): 74-84.
    [14] Zhou S, Kang F, Li W, et al. Point cloud registration for agriculture and forestry crops based on calibration balls using Kinect V2[J]. International Journal of Agricultural and Biological Engineering, 2020, 13(1): 198-205.
    [15] Fischler M A, Bolles R C. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6): 381-395.
    [16] 邹际祥. 基于KD-TREE加速的点云数据配准技术研究[D]. 合肥:安徽大学,2013.Zou Jixiang. The Research of Point Cloud Data Registration Technique Based on KD-Tree Acceleration[D]. Hefei:Anhui University, 2013. (in Chinese with English abstract)
    [17] Sorkine O, Rabinovich M. Least-squares rigid motion using SVD [J]. Computing, 2017, 1(1): 1-5.
    [18] 曾清红,卢德唐. 基于移动最小二乘法的曲线曲面拟合[J]. 工程图学学报,2004,25(1):84-89.Zeng Qinghong, Lu Detang. Curve and surface fitting based on moving least square method[J]. Journal of Engineering Graphics, 2004, 25(1): 84-89. (in Chinese with English abstract)
    [19] 余秀丽. 基于Kinect的苹果树三维重建方法研究[D]. 杨凌:西北农林科技大学,2016.Yu Xiuli. Reconstruction of Apple Tree Based on Kinect[D]. Yangling: Northwest A&F University, 2016. (in Chinese with English abstract)
    [20] 邵小宁. 基于Kinect的植物三维点云获取与重建方法研究[D]. 杨凌:西北农林科技大学,2016.Shao Xiaoning. Research on 3D Point Cloud Acquisition and Reconstruction of Plants Based on Kinect[D]. Yangling: Northwest A&F University, 2016. (in Chinese with English abstract)
  • 期刊类型引用(9)

    1. 陈允琳,兰玉彬,韩鑫,王娟,王会征,傅亮. 基于Kinect V3传感器的叶菜类作物三维重建与表型参数获取. 农业机械学报. 2025(03): 101-110+197 . 百度学术
    2. 王建雄,司永胜,张艳,冯凡. 基于改进海豚回声定位算法的奶牛点云配准研究. 河北农业大学学报. 2024(06): 47-53 . 百度学术
    3. 李云飞,李振东,杨立伟,刘刚,吕树盛,宫艳晶. 基于特征点改进的4PCS樱桃树三维点云配准方法. 农业机械学报. 2024(S1): 256-262 . 百度学术
    4. 王瑜,曹大有. 虚拟现实场景图像点云大数据展示仿真. 计算机仿真. 2023(03): 228-231+353 . 百度学术
    5. 谢忠红,黄一帆,吴崇友. 基于ISS-LCG组合特征点的油菜分枝点云配准方法. 华南农业大学学报. 2023(03): 456-463 . 百度学术
    6. 何永福,张锋. 建筑施工现场基坑工程目标数据采集仿真. 计算机仿真. 2023(05): 497-500+510 . 百度学术
    7. 刘刚,尹一涵,郑智源,李云涵,梁树乐,靳晨. 基于三维点云的群体樱桃树冠层去噪和配准方法. 农业机械学报. 2022(S2): 188-196 . 百度学术
    8. 刘刚,尹一涵,郑智源,周少清,李红娟,侯冲. 基于三维点云的群体樱桃树冠层光照分布预测模型. 农业机械学报. 2022(S1): 263-269 . 百度学术
    9. 陈海波,刘圣搏,王乐乐,王朝锋,向星岚,赵英杰,兰玉彬. 基于Kinect V3的单株作物自动化三维重建与验证. 农业工程学报. 2022(16): 215-223 . 本站查看

    其他类型引用(9)

计量
  • 文章访问数:  419
  • HTML全文浏览量:  0
  • PDF下载量:  701
  • 被引次数: 18
出版历程
  • 收稿日期:  2021-07-13
  • 修回日期:  2021-10-14
  • 发布日期:  2021-10-31

目录

    /

    返回文章
    返回