Abstract: Aiming at the obstacle detection problem in orchard autonomous vehicles, a binocular vision combined with YOLOv4 orchard obstacle detection and positioning system is designed. The system hardware adopts Jetson Xavier NX and is developed based on the ROS robot operating system. The ZED binocular camera obtains the left and right eye views, and the distance information is calculated based on the parallax principle; the YOLOv4 algorithm detects the position of the obstacle in the image according to the RGB image on the side of the binocular camera, Combined with the distance information of the pixel position obtained by the binocular camera to locate the obstacle. Select pedestrians and fruit trees in the orchard as obstacles to make an image data set, use and train the YOLOv4 target detection model, and complete the deployment of ROS, YOLOv4, ZED cameras and other software on the Jetson Xavier NX platform. Experiments show that the accuracy rate of the trained model is 92.15%, and the recall rate is 89.34%. Pedestrian and fruit tree positioning experiments show that the average relative error of pedestrian and fruit tree obstacle depth and distance direction is 1.75%, the maximum relative error is 3.20%, and the real-time detection can reach up to 6.4 f/s. The system is fast and real-time, and can detect and locate pedestrians and fruit trees in orchard robots.