Abstract: In order to classify and locate different walnut varieties, a walnut detection method based on deep learning was proposed. First of all, this paper took the three kinds of walnut mainly produced in the southern Xinjiang region as the object for image acquisition and made the walnut data set by flipping, clipping, denoising, lighting transformation, and other operations of the image. Then, the YOLOv5-based detection model was used for experiments and compared with the detection results of YOLOv3, YOLOv4, and Faster RCNN algorithms. The results showed that the mean average accuracy（mAP） of the YOLOv5 model for walnut detection of Xin2, Xinguang, and Wen185 was 99.5%, 98.4%, and 97.1%, respectively, and the detection time of a single image was 7 ms. Under the same data set and the same experimental environment, the detection speed of the model is 7 times that of Faster RCNN, the detection accuracy of the model is 2.8% higher than that of Yolov4, and the model size is only 1/14 of that of YOLOv4. The test results showed that the model based on YOLOv5 of walnut detection was the highest in terms of detection accuracy and speed among all the comparison algorithms, which was suitable for the detection requirements of this research, which could provide a research basis for robot automatic walnut sorting.