Abstract: It firstly constructs the frame of sorting robot system, analyzes the selection and calibration of industrial cameras, then realizes the recognition of target fruit based on BP neural network. Finally, it realized the real-time positioning of the target by using the template matching of image processing to realize the fruit sorting manipulator system. The results show that the system can classify many kinds of fruits, and the accuracy is very high, which proves the feasibility and accuracy of the system.