Rice panicle tracking and length extraction based on optical flow pretreatment and StrongSORT

The panicle number and length are two of the most crucial indicators of rice yield. Accurate acquisition of the panicle traits is of great significance to rice breeding and genetic research. However, the traditional measurements of panicle traits cannot fully meet the large-scale production in recent years, due to the contact damage, subjective inefficiency, and low repeatability. Therefore, it is very urgent to develop the observation and identification for the accurate and efficient measurement of rice panicle traits. In this study, the rice panicle tracking and traits extraction were proposed using optical flow preprocessing and the StrongSORT algorithm. Initially, a series of experiments was conducted to capture the 200 rotating rice videos. The dataset was divided into the training and testing sets in a ratio of 8:2. Subsequently, the Gunnar Farneback optical flow algorithm was employed to preprocess the videos in order to reduce the occlusion. The Convolutional Block Attention Module (CBAM) attention mechanism was then integrated into the YOLOv8-seg network in order to enhance the target detection and segmentation of rice panicles. Finally, the StrongSORT algorithm was utilized to realize the multi-target tracking and the counting of rice panicles. The Zhang-Suen skeleton extraction was applied to determine the length of the rice panicle with the largest panicle after detection. Moreover, a motion prior model was constructed with the movement trajectories and velocities of the potted rice. The position of rice panicles was predicted in the next frame. The ID switches were reduced to prevent the panicle tracking failures and duplicate counting caused by occlusion. The results demonstrated that high accuracy of the tracking was achieved to detect the rice panicle. The mean average precision of the improved YOLOv8-seg model reached 81.1%, with an increase of 8.7 percentage points, compared with the original YOLOv8-seg model. Furthermore, the mAP of the YOLOv8-seg model was improved to 95.0% after optical flow preprocessing, indicating a substantial enhancement of 13.9 percentage points over the unprocessed model. In rice multi-target tracking, the combination of optical flow preprocessing, the improved YOLOv8-seg, and StrongSORT was achieved in a multi-target tracking accuracy of 85.58% and a high-order tracking accuracy of 64.06%, which were improved by 11.83 and 9.53 percentage points, respectively, compared with the combination without optical flow preprocessing. The number of ID switches was significantly reduced from 891 to 275, with a decrease of 69.2%. In terms of counting accuracy, the combination was achieved in a coefficient of determination (R²) of 0.9696, a mean absolute percentage error of 2.15%, and a root-mean-square error of 1.87, compared with the actual values. In panicle length extraction, the R² value was 0.940 8, the MAPE was 4.07%, and the RMSE was 0.47. The combination of the optical flow preprocessing, improved YOLOv8-seg, and StrongSORT effectively reduced the interference among overlapping panicles and the ID switch. The accuracy and multi-target tracking were enhanced after detection at the same time. The finding can also provide the technical pathway for the rice panicles and length measurement in rice breeding.