Abstract: Since the initial symptoms of Peach Tree Yellow Leaf Disease（PTYLD） are not readily apparent, the existing deep learning-based recognition techniques for this disease suffer from issues like inaccurate recognition and limited recognition species. To address this, a recognition model of PTYLD based on Faster-RCNN（Region-based Convolutional Neural Network） is proposed. In order to enhance the recognition accuracy and diversity of PTYLD, RS-Loss function is used to replace the cross-entropy function in the Region Proposal Network（RPN）, and the Soft-NMS algorithm is used to replace the original Non-Maximum Suppression（NMS） algorithm, so as to improve Faster-RCNN. The recognition effect of the initial and improved version of Faster-RCNN models on PTYLD is compared by experiments. The experimental results demonstrate that the improved Faster-RCNN achieves a mean average precision（mAP） of 90.56%, recall rate of 94.16%, an accuracy of 92.53% for each category of yellow leaf disease, and can identify five common PTYLD.