Abstract: In order to improve the effect of agricultural pest image recognition, the improved support vector machine algorithm was proposed. Firstly, cross validation was optimized the penalty factor, and the Manhattan distance was used to determine the kernel function selection. Secondly, the feature model of agricultural pest image was established, including color feature, texture feature and shape feature. Thirdly, the multi feature fusion recognition of pest image was carried out, and the weight of color feature, texture feature and shape feature of various pests were calculated using Fisher, so that avoiding pest misrecognition. Experimental simulation showed that the average recognition rate of multi features pests were better than single feature and two features, average recognition rate of ISVM was 95.67%, it was 9.81%, 6.82%, 5.57%, 3.93% and 1.90% higher than NN, ACNN, FL, SVM and PPSVM, detection result of ISVM was better than other algorithms.