Abstract: Aiming at the problems that target detection algorithms are prone to miss detection and misdetection of highway pavement lesions in multi-scale unmanned aerial vehicle（UAV） images and that the same lesion is repeatedly detected in consecutive frames, we propose a pavement detection method with the ability of lesion re-recognition. By introducing the true width-height loss and aspect ratio to improve the performance of the loss function, utilizing the CA（Coordinate attention） mechanism to improve the recognition ability of the model in complex backgrounds, introducing the initial features of the model into the feature fusion network to improve the robustness of the model in detecting multi-scale pavement lesions, and constructing a second-level detection mechanism based on the DeepSORT（Multi-target Tracking Algorithm for Target Detection） to realize the re-recognition of lesions. The experimental results show that the average detection accuracy of the model mAP reaches 89.19%, which is 3.11% higher than that of the benchmark model; the F1 score is 0.851 4, which is 2.49% higher than that of the original model; at the same time, it also outperforms the mainstream target detection algorithms; and the counting accuracy of the lesions in the drone image reaches 91.38%, which is 25.86% higher than that of the pre-improvement model, which provides accurate and real-time lesion data for the road pavement inspection and maintenance.