高级检索+

基于改进CycleGAN与YOLOv8的夜间番茄茎、枝分割方法

Segmenting tomato stems and branches at night time using improved CycleGAN and YOLOv8

  • 摘要: 为解决夜间环境下番茄枝叶识别精度低、漏检等问题,该研究提出一种基于改进CycleGAN(cycle-consistent generative adversarial network)与YOLOv8的夜间番茄茎、枝分割模型(NTS-YOLO)。通过将CycleGAN网络的低层和高层特征进行融合,并在其特征提取模块中引入ECA(efficient channel attention)注意力机制模块,解决了CycleGAN生成图片颜色失真和模糊的问题。用轻量化的主干网络StarNet替换YOLOv8主干网络以降低模型的复杂度,提升模型运行速度。用Gold-YOLO替换YOLOv8颈部网络并在其头部网络嵌入CBAM(convolutional block attention module)注意力机制以提升模型的精度。通过数据增强后,NTS-YOLO模型的平均精度均值提高了19.8个百分点。通过消融试验表明,NTS-YOLO模型的平均精度均值为93.3%,相比于原网络提升了4.5个百分点。NTS-YOLO模型的主干分割精度均值、侧枝分割精度均值和果枝分割精度均值分别为95.3%、92.4%和92.2%,相比于原网络分别提升了5.0、6.7和5.4个百分点。与主流分割模型Mask R-CNN、YOLACT、YOLOv5l-seg和YOLOv8l-seg相比,NTS-YOLO模型的平均精度均值分别提升了15.0、18.8、5.7和4.5个百分点。NTS-YOLO模型相比于其他主流分割模型,在夜间环境下分割番茄主干、侧枝和果枝更具鲁棒性。研究结果可为番茄等设施果蔬的自动化和智能化枝叶修剪提供有效的技术支持。

     

    Abstract: Tomato is one of the most widely grown vegetables in the world. However, many challenges still remain in the actual cultivation in the broad market prospects of tomato production. Among them, the pruning of branches and leaves has been one of the most important steps during tomato growth and fruiting. However, manual pruning cannot fully meet the large-scale production in recent years, due to the high labor intensity and cost. Particularly, interval pruning is often required to work consistently for long periods of time. Fortunately, the tomato pruning robot can be expected to work all the whole day and night. It is an urgent need for the tomato pruning robots to accurately and efficiently identify the tomato stems and branches. Tomato stems and branches can often be recognized well during the daytime. But the low accuracy and missed detection of tomato branches can occur in a night environment at present. In this study, a segmentation model (NTS-YOLO) was proposed for the tomato stems and branches in night environment using improved CycleGAN and YOLOv8. The feature extraction of the CycleGAN module was improved to solve the color distortion and blurring of images. The low- and high-level features were then fused to introduce the efficient channel attention (ECA) mechanism in the network. The YOLOv8 backbone network was replaced by the lightweight backbone network (StarNet), in order to reduce the complexity of the improved model. The YOLOv8 neck network was also replaced by the Gold-YOLO. The convolutional block attention module (CBAM) attention mechanism was embedded in the head network, in order to improve the accuracy of the improved model. The results showed that the FID, LPIPS of the images generated by the improved CycleGAN were reduced by 12.23 and 0.07 and PSNR increased by 2.96 dB, respectively, compared to the original CycleGAN model. The NTS-YOLO improved mAP by 19.8 percentage points using the data-enhanced datasets. The ablation experiments indicated that the mean values of precision, recall, and average accuracy of the NTS-YOLO model were achieved at 92.5%, 86.1%, and 93.3%, respectively, which were improved by 3.8, 2.4, and 4.5 percentage points, respectively, compared with the original network. The frame rate of detection increased from 70.9 to 75.3 frame per second. The effectiveness of the NTS-YOLO model was validated using ablation experiments. The NTS-YOLO model achieved 95.3%, 92.4%, and 92.2% in the AP of the stem, the lateral branch, and the fruit branch, respectively, which were improved by 4.1, 4.1, and 5.3 percentage points, respectively, compared with the original network. Furthermore, the mean average accuracy of the NTS-YOLO model reached 93.3%, which was an increase of 15.0%, 18.8%, 5.7% and 4.5%, respectively, compared with the mainstream segmentation models, such as Mask R-CNN, YOLACT, YOLOv5l-seg and YOLOv8l-seg. The leakage rate reached 4.2%, which was reduced by 15.9, 18.2, 10.5, and 5.9 percentage points, respectively. The FPS reached 75.3 frame per second, which was faster than the rest networks by 58.2, 49.4, 2.7 and 4.4 frame per second, respectively. The NTS-YOLO network was more robust and faster than the rest of mainstream segmentation in segmenting tomato stems, lateral branches, and fruit branches in a night environment. This finding can also provide technical support for automatic and intelligent pruning in the tomato-growing industry.

     

/

返回文章
返回