Abstract: Agricultural robots have been increasingly recognized to improve agricultural productivity and labor saving for smart farming in modern agriculture. Compared with the structured industrial environments, the agricultural working scenarios are characterized by strong openness, high dynamics, and pronounced unstructured properties. There are great variations in the crop morphology and growth stages, complex and uneven terrain conditions, as well as multiple sources of environmental uncertainty, such as the illumination and weather fluctuations. It is significantly difficult for robotic perception, decision-making, and control. As a result, the efficiency and scalability of the agricultural robots can often severely suffer from the long experimental cycles, high costs of field trials, limited availability of large-scale real-world data, and insufficient repeatability of algorithm validation. The virtual simulation can be expected for the agricultural robotics research, testing, and deployment, due to its advantages in the safety, cost controllability, and experimental repeatability. A controllable and reproducible environment of the robotic systems can be designed and then evaluated under diverse operating conditions without the expenses of real-world agricultural experiments. The computing power, physical modeling, and graphics rendering have further promoted the high-fidelity simulation in recent years. In this study, a systematic review was presented on the virtual simulation technologies. The evolution history was traced from early applications in the training simulators to modern high-fidelity and multi-modal systems. Furthermore, the application trajectory of the virtual simulation was reviewed in the robotics field, in order to integrate computer graphics, physical modeling, artificial intelligence, and interactive technologies. Particularly, the application directions of the virtual simulation were summarized in agricultural robotics, including unstructured agricultural environments, kinematic and dynamic analysis of the robotic systems, the operational workflows and task strategies, safety evaluation of the human-robot collaboration, and operator skill training. Representative scenarios of the agricultural operation were also analyzed using mainstream simulation platforms, such as Gazebo, Unity3D, and NVIDIA Isaac Sim. These platforms were then evaluated at different stages of the agricultural robots, in terms of the physical simulation accuracy, visual realism, interaction, and integration with the robotic middleware systems. Typical agricultural tasks were examined, such as orchard harvesting, crop protection, pruning, and seeding. Application examples were also given on the domain randomization, computer vision, autonomous navigation, and procedural content generation within the virtual simulation environments. The simulation was then reduced to the real-world experimental risks in order to accelerate the algorithm iteration and efficiency. As such, several key challenges were proposed in the virtual simulation of agricultural robotics. For instance, there were some difficulties in the environment modeling and task adaptability, due to the complexity and variability of the agricultural scenes. Technical limitations and application bottlenecks of the simulation reduced the modeling accuracy of the physical interactions and contact dynamics between robots and agricultural objects, such as the crops and soil. Moreover, the gap between simulation and actual robotic performance was also attributed to the discrepancies between the simulated and real-world sensors under environmental conditions. Finally, the future trends of the virtual simulation were predicted in agricultural robotics, including the high-fidelity, multi-scale, and multi-modal simulation, the integration with artificial intelligence, digital twin technologies, and data-driven modeling. The virtual simulation can be expected to play an increasingly important role in the research tasks of the agricultural robots, such as algorithm validation, system evaluation, and methodological comparison. Overall, this review can provide a strong reference to further develop the virtual simulation in agricultural robotics for smart farming.