Abstract:
Fruit tree pruning is one of the most important horticultural practices to regulate tree structure. Canopy ventilation and light can be obtained for high fruit yield and quality. However, conventional manual pruning cannot fully meet the large-scale production in recent years, due to the high labor intensity and cost. Furthermore, pruning standards are also required to reduce the missed or incorrect cuts. Fortunately, fruit tree information can be acquired from two-dimensional appearance to three-dimensional structural representation using computer vision, deep learning, and three-dimensional sensing. Manual observation can be transformed into machine learning in smart agriculture. Previous reviews have focused on general plant point-cloud segmentation or robotic pruning hardware. However, it is lacking in the full process from visual perception and pruning decision-making to robotic execution. Therefore, this study reviewed recent progress on intelligent pruning technology of fruit trees from perception, decision-making, and execution levels using computer vision. (1) At the perception level, the summary was given to acquire fruit tree images and three-dimensional information, together with image and point-cloud preprocessing. Existing studies focused on tree reconstruction and structural information acquisition, but no single sensing scheme was found suitable for all pruning scenarios. Fine pruning of individual trees required complete representation of small branches and topological relationships, whereas continuous online operation required a balance among modeling accuracy, acquisition efficiency, and equipment cost. (2) At the decision-making level, image- and point-cloud was reviewed for branch segmentation and recognition, structural parameter extraction, pruning branch identification, and pruning point localization. Deep learning improved branch recognition and segmentation, while three-dimensional data provided quantitative information on branch length, angle, spacing, diameter, and topology. Once branches overlapped or adhered together, it was necessary to identify the pruning targets at the individual-branch level, rather than the category-level segmentation alone. Therefore, pruning decisions needed to combine branch instance separation, three-dimensional structural parameters, and horticultural pruning. Large public datasets were lacking to quantify horticultural knowledge for model generalization and accuracy. (3) At the execution level, a summary was made on the hand-eye coordinate transformation, robotic arm path planning, and pruning end-effector design. Pruning targets were converted into robot positions and motions, where depth measurement, calibration, and kinematic errors were accumulated during coordinate transformation. Static or simplified obstacle models failed to balance wind disturbance and flexible branch deformation. Current end effectors shared limited adaptability to different branches. Overall, a technical chain was established from tree information perception to pruning decision and robotic cutting using computer vision. Two challenges were to maximize the accuracy of individual perception, segmentation, or planning algorithms. Another was to acquire and interpret tree structures at the front end, where pruning and cutting positions were converted into instructions for the robots to execute accurately. Multi-source information fusion and adaptive preprocessing can be expected to improve model generalization using horticultural knowledge. Online sensing can also be utilized to update pruning points and motion paths in dynamic canopies. Accurate, dependable, and integrated systems can promote intelligent fruit tree pruning in the orchard.