Abstract: It first introduces the structure of convolutional neural network, the working principle of each module and retina net target detection algorithm, and then uses the color histogram feature extraction method to realize the intelligent image recognition and classification algorithm of crops based on convolutional neural network. Experiments show that the algorithm can accurately identify and test tomatoes on Tomato branches, and the accuracy is high. It verifies the effectiveness and accuracy of the system, which meet the application needs of real-time fruit recognition.