高级检索+

基于三视角成像的苹果表面缺陷检测方法

Three-view imaging-based method for defect detection on apple surfaces

  • 摘要: 为解决传统单视角成像在苹果表面缺陷检测中存在大量盲区的问题,该研究提出了一种基于三视角成像的苹果表面缺陷检测方法。首先,设计了一套三视角成像的苹果表面缺陷检测系统,通过三视角点云配准重建苹果模型,以准确获取系统成像区域面积;其次,基于标准球模型提出了适用于苹果三视角图像的分割方法;最后,改进了YOLOv11基础模型,将其Neck部分的C3k2模块中的C3k替换为非局部自注意力残差多层感知机模块(non-local attention residual multi-layer perceptron,NARM),构建了NARM-YOLOv11模型,从而提升检测精度和小尺度缺陷的识别能力。研究结果表明:所提出的三视角成像系统将苹果表面成像区域占比均值由单视角的34.6%提升至74.3%;苹果三视角图像分割方法平均可去除20.5%的冗余区域,平均缺陷检测重复率由初始图像的26.0%降到分割后的7.6%,平均漏检率为3.6%;NARM-YOLOv11模型相较于基础模型,精确率、召回率、平均精度均值分别提高2.7、2.5、3.4个百分点,模型的复杂度略有上升,帧率下降1.7帧/s;系统平均查准率为89.7%,平均缺陷识别率为88.1%。该研究有效克服了单视角成像检测苹果表面缺陷盲区大、三视角成像冗余高以及复杂背景下苹果缺陷特征难检测等问题,并形成一套适用于苹果表面缺陷可靠检测的系统,为苹果产业的智能化升级提供更为坚实的技术支撑。

     

    Abstract: In the field of post-harvest quality inspection of fruits, apple surface defect detection was a core link that directly affected the grading efficiency and market value of apples. Traditional single-view imaging technologies, which were widely adopted in early defect detection systems, had the inherent limitation of large detection blind areas. Due to the irregular spherical shape of apples, single-view imaging could only capture partial surface information of the fruit, leading to missed detection of defects in the uncollected areas and thus failing to meet the requirements of high-precision quality inspection in the intelligent upgrade of the apple industry. To address this critical problem, a comprehensive apple surface defect detection method based on three-view imaging was proposed in this study, and the primary objective was to overcome the drawbacks of large blind areas in single-view imaging, high redundancy in multi-view imaging, and difficulty in detecting defect features under complex backgrounds, so as to establish a reliable detection system for apple surface defects and provide solid technical support for the intelligent development of the apple industry.To achieve the above objective, three key technical methods were developed and applied in the study. First, a dedicated three-view imaging system for apple surface defect detection was designed. To accurately obtain the imaging area of the system, three Intel RealSense D415 depth cameras were deployed, and the hardware synchronization function of the cameras was used to realize simultaneous data acquisition. The acquired RGB and depth images were converted into three-dimensional point clouds through internal parameter calibration, and then a series of point cloud processing steps including preprocessing, coarse registration, fine registration, downsampling, and surface reconstruction were performed to complete the 3D reconstruction of apples and calculate the area of the joint imaging region of the three-view system. Second, a segmentation method suitable for three-view apple images was proposed based on a standard sphere model, which was used to eliminate redundant background and overlapping regions in the three-view images. Third, the basic You Only Look Once version 11 (YOLOv11) model was improved to enhance its defect detection performance. Specifically, the C3k module in the Neck part of the original model was replaced with the Non-local Attention Residual Multi-Layer Perceptron (NARM) module, and the NARM-YOLOv11 model was constructed to strengthen the model’s ability to capture long-range feature dependencies and identify small-scale defects.A series of comparative experiments and performance verification tests were carried out to evaluate the effectiveness of the proposed method and system, and the results obtained were comprehensive and convincing. In terms of the imaging area performance of the three-view system, the three-view imaging system realized the fusion of multi-angle surface information of apples through precise point cloud registration and reconstruction. The average proportion of the apple surface imaging area captured by the system was increased from 34.6% of the single-view imaging to 74.3%, which significantly reduced the detection blind area and basically covered most of the apple surface, laying a foundation for full-surface defect detection. For the three-view image segmentation method based on the standard sphere model, the method effectively removed redundant regions in the three-view images, with an average redundant region removal rate of 20.5%. After segmentation, the average defect detection repetition rate caused by overlapping imaging areas was reduced from 26.0% of the original images to 7.6%, and the average missed detection rate was controlled at 3.6%, which effectively solved the problem of high redundancy in multi-view imaging and improved the accuracy of subsequent defect identification. In the test of the improved NARM-YOLOv11 model, compared with the basic YOLOv11 model, the precision, recall, and mean average precision (mAP) of the NARM-YOLOv11 model were increased by 2.7, 2.5, and 3.4 percentage points respectively, which indicated that the introduction of the NARM module effectively enhanced the model’s feature extraction capability, especially for small-scale and low-contrast apple surface defects. Although the model complexity had a slight increase due to the addition of the attention mechanism and multi-layer perceptron structure, the frame rate only decreased by 1.7 frames per second, which still met the requirements of real-time detection in practical applications. For the overall performance of the detection system, the average precision of the system reached 89.7%, and the average defect recognition rate was 88.1%, which fully demonstrated the reliability and practicality of the integrated system in apple surface defect detection.This study successfully overcame multiple technical bottlenecks in apple surface defect detection, including the large blind area of single-view imaging, high redundancy of three-view imaging, and difficulty in detecting defect features under complex backgrounds. The developed three-view imaging system, efficient image segmentation method, and improved NARM-YOLOv11 detection model formed a complete and reliable technical system for apple surface defect detection. This system not only filled the technical gap in full-surface defect detection of spherical fruits but also provided a feasible technical scheme for the intelligent upgrading of the post-harvest quality inspection link in the apple industry, and it also had certain reference significance for the defect detection of other spherical agricultural products, which promoted the application of multi-view imaging and deep learning technologies in the field of agricultural product quality inspection.

     

/

返回文章
返回