Abstract:
Agricultural machinery has increasingly integrated rapid robotics in the intelligent and mechanized operations for field crops. Agricultural robots can rely mainly on the precise extraction of the ridge navigation lines to avoid crop damage, particularly in the navigation and automated tasks of the field crops. However, traditional computer vision can confine to accurately identify the ridge navigation lines in field environments, where the crop distributions are often scattered, due to uneven planting density, missing seedlings, or misalignment. In this study, a search-and-fill model was proposed to combine with the through-domain matching. The components were also connected for the real-time and accurate extraction of the navigation lines. The RGB images were captured as the input. An OAK-D-Pro-PoE depth camera was fixed to the chassis of the agricultural robot. The OAK camera was mounted at a height of 165 cm with a 45° tilt angle, thus covering an area of approximately 3.34 m² of the ground. The images were then standardized to the uniform resolution of 960×540 pixels. Sufficient crop detail was retained in the small and scattered crop images. The standardized images were converted to the YCrCb color space during preprocessing, in order to extract the different color components. The green chrominance offset (
Cg) was enhanced by a factor of 2. The luminance component (
Y) was removed to form the 2
Cg-
Cr-
Cb component image. Better differentiation between crop and soil regions was achieved for the more effective threshold segmentation. The Otsu (the maximum between-class variance) was then used to separate the crops from the background. The binary images were generated after segmentation. Morphological operations were applied with a 17×17 convolution kernel to further reduce the noise for the high quality of the binary images. A vertical filling model was proposed to connect the scattered components within the same crop row, according to the triangular relationships. The points of crop contour were taken as the apex, in order to search along the lower third of the contour region. The computational complexity was significantly reduced to maintain the high detection accuracy. When the pixel values of the apex were matched with those in the regions of the left and right vertices, the region was filled with the same pixel value, thus achieving connectivity between the two crop regions. Upper and lower boundary thresholds were then determined to connect the components touching these boundaries, defined as the through-domains. The binary image was sliced row-wise after the extraction of the through-domains. The image was divided horizontally into multiple image strips with a height of ∆
h. The regions of interest (ROIs) were processed within each strip to fit the minimum bounding rectangles. The center points of these rectangles were output as the navigation points. Then, the through-domains were selected to fit the ridge lines of the precise navigation. The base width and height of the search-and-fill model were experimentally optimized to evaluate the accuracy and processing time of navigation line fitting. All pre-annotated navigation lines were successfully detected to validate the search-and-fill model with the through-domain matching. A comparison was made on the different clustering and line-fitting algorithms. The performance of the algorithm was first validated in the standard and complex scenarios. Then two algorithms were compared to highlight the superior extraction accuracy of navigation lines. The field images were also collected by the agricultural robots. The experiments showed that better performance was achieved with an average angular deviation of 1.52°, an average offset of 12.4 pixels, and a navigation line detection accuracy of 93.35%. Furthermore, the average processing time per image frame was only 95 ms. Experimental data demonstrated that the navigation lines were accurately detected with strong robustness and real-time performance in the complex scenarios of crop distribution. This finding can also provide reliable visual navigation of intelligent agricultural robots for crop safety in the fields.