Abstract: Image-based detection has been widely applied in agricultural and forestry pest monitoring. However, challenges remain in real-world environments, such as serious size variations and visual similarities between pests from closely related families, leading to low detection accuracy. In this study, an improved model was proposed to detect pests in agriculture and forestry using YOLO-EDW. Initially, YOLOv8n was used as the baseline model to tackle the missed detections due to small targets. An efficient multi-scale attention mechanism was incorporated to improve the extraction of key features, along with a cross-space learning approach. Fusion features were generated to capture both short- and long-range dependencies using multi-scale parallel sub-networks, effectively extracting pest features. A diverse branch block detection head was introduced to reduce the misdetections caused by pests with similar appearances. This multi-branch structure increased the complexity of convolutional layers during training. High performance was achieved in the differentiate pests from closely related families. Additionally, wavelet transform convolution was applied to optimize the C2f structure for feature fusion, where the texture and shape information were extracted from forestry pests. Fine-grained details were captured under consistent experimental conditions. Ablation tests were conducted on the AFPD (agricultural and forestry pest dataset), including 71 species of agricultural and forestry pests. The results revealed that the YOLO-EDW model achieved an accuracy, recall, and mean average precision of 90.6%, 88.6%, and 92.0%, respectively, which were improved by 2.3,1.2, and 0.4 percentage points, compared with the YOLOv8n model. The YOLO-EDW model’s mAP was improved by 8.4 and 3.5 percentage points, respectively, compared with popular object models, like SSD and Faster R-CNN. Visualization results confirmed that the high effectiveness was obtained for detecting agricultural and forestry pests. A comparison was also made between the YOLO-EDW and YOLOv8n models using pest images of varying target sizes from the AFPD. The YOLO-EDW model with a feature enhancement module accurately detected both small and occluded pests under complex scenarios. YOLOv8n suffered a higher misdetection rate for pests like the Cnaphalocrocis medinalis. As such, the YOLO-EDW model effectively distinguished between morphologically similar pests, better supporting the monitoring needs of forestry pests. Furthermore, the generalization was tested on the publicly available IP102 dataset. The improved YOLOv8n model outperformed mainstream object algorithms, like Faster R-CNN, in mAP. Highly precise localization and classification were observed on both adult and larval pests in crops, indicating effective detection of small pests, like aphids in clusters, the normally missed and misdetected in dense target scenarios. Overall, the improved model adapted well to the morphological and distribution features of pests in agricultural and forestry production, fully meeting the practical demands for high accuracy and robustness. Therefore, the YOLO-EDW model with only 5.58×106 parameters, high inference speed, and easy deployment provides high accuracy for pests at varying scales and families.