Abstract: Crop yield has been one of the most prominent issues in the world in recent years. However, crop diseases have posed a great threat to crop yield. It is a high demand to timely and accurately detect crop disease types and the degree of damage. The manual recognition can rely only on skilled technicians. But, the visual fatigue of humans can easily lead to reduce the accuracy rate. The current machine learning cannot consider the correlation between the attributes in the data set, resulting in low recognition accuracy. In this study, a network model was proposed to identify the damage degree of multiple crop diseases using deep transfer learning and improved RegNet. The model contained four aspects as follows. Firstly, an online data enhancement was carried out at the input side of this model. Nine strategies were selected for the data enhancement, such as the HSV color variation, grayscale transformation, and Gaussian noise. The diversity of data samples increased while reducing the time and space for the data set collection expansion. As such, the over-fitting of the network was alleviated during this time. Secondly, the Efficient Channel Attention (ECA) mechanism was introduced into the feature extraction layer of the model for the cross-channel interaction. The model was then improved to extract more subtle features, particularly for the crop disease features. As such, a higher accuracy of recognition was achieved for the crop disease damage, which increased by 1.5 percentage points, compared with the original model at the same model size. In addition, a multi-scale feature fusion was introduced into the classification layer of the model. A spatial pyramid pooling was adopted to highly improve the accuracy of the model. The degree of crop disease damage was classified at different scales, especially for the fine-grained features. Correspondingly, the accuracy of crop disease damage recognition increased by 2.4 percentage points. Finally, deep transfer learning was used to optimize the overall performance of the model. The convergence speed was accelerated to improve the generalization ability of the model. The recognition accuracy was improved by 2.7 percentage points, compared with the strategy without deep transfer learning. The experimental results show that the improved RegNet network model achieved 94.5 percentage points accuracy on the dataset of crop disease damage level, which was 10.4 percent higher than the original one. The recognition accuracy of the improved model was improved by 2.1, 6.0, 3.7, and 1.6 percentage points, respectively, compared with the commonly-used classification network models, such as ResNet50, VGG16, InceptionV3, and ConvNeXt. Consequently, higher accuracy of recognition and smaller model size were achieved in the improved classification model, compared with the rest. The better performance of feature extraction and stronger classification ability were also obtained for the fine-grained features during this time. The finding can also provide a promising way to identify the crop disease types and the degree of damage.