Abstract: Fruit recognition is an important part of visual detection technology, and its recognition accuracy is easily affected by the complex growth environment and fruit state. This paper takes tomato fruits in six complex growth states, such as single, cluster, illumination, shadow, occlusion and overlapping in greenhouse environment as objects, and proposes a tomato fruit recognition method based on the combination of improved YOLOv4 network model and transfer learning. Firstly, the ImageNet dataset and the 16 convolutional layers at the front end of the VGG network model were used for model parameters pre-training, and the trained model parameters were initialized to improve the weights of the model to replace the original initialization operation. Then the tomato dataset was used for training in the new model combined with the convolutional layers of VGG19 and the backbone network of YOLOV4. The optimal weight was obtained to detect tomato fruits in complex environment. Finally, the improved model was compared with Faster RCNN, YOLOv4-Tiny, and YOLOv4 network models. The results showed that the average detection accuracy（mAP） of the improved model under six complex environments for tomato fruits was 89.07%, 92.82%, 92.48%, 93.39%, 93.20%, and 93.11%, and the F1 scores at different maturity levels of ripe, half-ripe, and immature were 84%, 77%, and 85%, respectively, and their recognition accuracy is better than that of the comparison models. The method in this paper realizes effective detection and recognition of tomato fruits in six complex environments, which provides a theoretical basis for the intelligent picking of tomato fruits.